From fe922986f6e7d7eb37918d7903388eaf241c47cf Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Thu, 29 Aug 2024 13:53:43 +0000 Subject: [PATCH] build based on b3372d0 --- dev/.documenter-siteinfo.json | 2 +- dev/api/index.html | 2 +- dev/contributing/index.html | 2 +- dev/creating_a_cat/index.html | 2 +- .../ability_convergence_3pl/index.html | 2 +- .../ability_convergence_mirt/index.html | 2 +- ...2e848e394ca61b51-10793518835325191992.bin} | Bin 5428524 -> 5428548 bytes ...e0d4018bebcc0ac6da-891568227096890466.bin} | Bin 1234126 -> 1234160 bytes ...e7c5f0cc0abedd036b-239007990221751549.bin} | Bin 4365248 -> 4365264 bytes ...2bb899803d5c0e30-12269883709464688670.bin} | Bin 170997 -> 170986 bytes dev/examples/examples/irfs.ipynb | 8 ++++---- dev/examples/examples/irfs/index.html | 6 +++--- dev/examples/examples/perf_plots.ipynb | 4 ++-- dev/examples/examples/perf_plots/index.html | 6 +++--- dev/examples/examples/vocab_iq/index.html | 2 +- dev/examples/index.html | 2 +- dev/index.html | 2 +- dev/search_index.js | 2 +- dev/using_your_cat/index.html | 2 +- 19 files changed, 23 insertions(+), 23 deletions(-) rename dev/examples/examples/bonito/bin/{1d71275d8737830c06dfe2ab9c7c93c45e3a8a26-16059389088621710572.bin => 6ecc99aedf04897452aeb2092e848e394ca61b51-10793518835325191992.bin} (98%) rename dev/examples/examples/bonito/bin/{758178665a8875fb723ef853dedeeb7ab3560bee-5463270529454683959.bin => c047ef9b090e126fe3d894e0d4018bebcc0ac6da-891568227096890466.bin} (93%) rename dev/examples/examples/bonito/bin/{3dff6576f1026c9d50cb7a2eed1fa965d9adea99-4453480068583967657.bin => ce800911340edfcc75196ae7c5f0cc0abedd036b-239007990221751549.bin} (99%) rename dev/examples/examples/bonito/bin/{02cdd7b832e4d41edaea4f7c802d5fde9fa3ce3d-313498830422961508.bin => fd136fd258ce126b6d77ecd02bb899803d5c0e30-12269883709464688670.bin} (86%) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 8ddd3e0..1843765 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.9.4","generation_timestamp":"2024-08-28T17:38:29","documenter_version":"1.6.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.9.4","generation_timestamp":"2024-08-29T13:53:39","documenter_version":"1.6.0"}} \ No newline at end of file diff --git a/dev/api/index.html b/dev/api/index.html index 632fd3d..0393e63 100644 --- a/dev/api/index.html +++ b/dev/api/index.html @@ -1,2 +1,2 @@ -API reference · ComputerAdaptiveTesting.jl
GitHub

API reference

ComputerAdaptiveTesting.Aggregators — Module

This module takes care of integrating and optimizing over the ability/difficulty space. It includes TrackedResponses, which can store cumulative results during a test.

source
ComputerAdaptiveTesting.NextItemRules — Module

This module implements the next item selection rules, which form the main part of CAT.

Bibliography

[1] Linden, W. J., & Pashley, P. J. (2009). Item selection and ability estimation in adaptive testing. In Elements of adaptive testing (pp. 3-30). Springer, New York, NY.

source
ComputerAdaptiveTesting.NextItemRules.AbilityVarianceStateCriterion — Type
struct AbilityVarianceStateCriterion{DistEst<:ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator, IntegratorT<:ComputerAdaptiveTesting.Aggregators.AbilityIntegrator} <: ComputerAdaptiveTesting.NextItemRules.StateCriterion

  • dist_est::ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator

  • integrator::ComputerAdaptiveTesting.Aggregators.AbilityIntegrator

  • skip_zero::Bool

This StateCriterion returns the variance of the ability estimate given a set of responses.

source
ComputerAdaptiveTesting.NextItemRules.DistributionExpectationBasedItemCriterion — Type
struct DistributionExpectationBasedItemCriterion{DistributionAbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator, AbilityIntegratorT<:ComputerAdaptiveTesting.Aggregators.AbilityIntegrator, StateCriterionT<:ComputerAdaptiveTesting.NextItemRules.StateCriterion} <: ComputerAdaptiveTesting.NextItemRules.ExpectationBasedItemCriterion

  • ability_estimator::ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator

  • integrator::ComputerAdaptiveTesting.Aggregators.AbilityIntegrator

  • state_criterion::ComputerAdaptiveTesting.NextItemRules.StateCriterion

This ItemCriterion wraps a StateCriterion and looks at its expected value for a particular item 1-ply ahead by integrating over an ability curve.

source
ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule — Type
struct ItemStrategyNextItemRule{NextItemStrategyT<:ComputerAdaptiveTesting.NextItemRules.NextItemStrategy, ItemCriterionT<:ComputerAdaptiveTesting.NextItemRules.ItemCriterion} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule

  • strategy::ComputerAdaptiveTesting.NextItemRules.NextItemStrategy

  • criterion::ComputerAdaptiveTesting.NextItemRules.ItemCriterion

ItemStrategyNextItemRule which together with a NextItemStrategy acts as an adapter by which an ItemCriterion can serve as a NextItemRule.

ItemStrategyNextItemRule(bits...; ability_estimator=nothing, parallel=true)

Implicit constructor for ItemStrategyNextItemRule. Will default to ExhaustiveSearch1Ply when no NextItemStrategy is given.

source
ComputerAdaptiveTesting.NextItemRules.NextItemRule — Type
abstract type NextItemRule <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase

Abstract base type for all item selection rules. All descendants of this type are expected to implement the interface (rule::NextItemRule)(responses::TrackedResponses, items::AbstractItemBank)::Int

NextItemRule(bits...; ability_estimator=nothing, parallel=true)

Implicit constructor for NextItemRule. Uses any given NextItemRule or delegates to ItemStrategyNextItemRule.

source
ComputerAdaptiveTesting.NextItemRules.PointExpectationBasedItemCriterion — Type
struct PointExpectationBasedItemCriterion{PointAbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator, StateCriterionT<:ComputerAdaptiveTesting.NextItemRules.StateCriterion} <: ComputerAdaptiveTesting.NextItemRules.ExpectationBasedItemCriterion

  • ability_estimator::ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator

  • state_criterion::ComputerAdaptiveTesting.NextItemRules.StateCriterion

This ItemCriterion wraps a StateCriterion and looks at its expected value for a particular item 1-ply ahead based on a point ability estimate.

source
ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule — Type
struct RandomNextItemRule{RandomT<:Random.AbstractRNG} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule
  • rng::Random.AbstractRNG: Default: Xoshiro()

This is the most basic rule for choosing the next item in a CAT. It simply picks a random item from the set of items that have not yet been administered.

source
ComputerAdaptiveTesting.NextItemRules.UrryItemCriterion — Type
struct UrryItemCriterion{AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator} <: ComputerAdaptiveTesting.NextItemRules.ItemCriterion
  • ability_estimator::ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator

This item criterion just picks the item with the raw difficulty closest to the current ability estimate.

source
ComputerAdaptiveTesting.CatConfig.CatRules — Type
struct CatRules{NextItemRuleT<:ComputerAdaptiveTesting.NextItemRules.NextItemRule, TerminationConditionT<:ComputerAdaptiveTesting.TerminationConditions.TerminationCondition, AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.AbilityEstimator, AbilityTrackerT<:ComputerAdaptiveTesting.Aggregators.AbilityTracker} <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase
  • next_item::ComputerAdaptiveTesting.NextItemRules.NextItemRule: The rule to choose the next item in the CAT given the current state.
  • termination_condition::ComputerAdaptiveTesting.TerminationConditions.TerminationCondition: The rule to choose when to terminate the CAT.
  • ability_estimator::ComputerAdaptiveTesting.Aggregators.AbilityEstimator: The ability estimator, which estimates the testee's current ability.
  • ability_tracker::ComputerAdaptiveTesting.Aggregators.AbilityTracker: The ability tracker, which tracks the testee's current ability level. Default: NullAbilityTracker()

Configuration of the rules for a CAT. This all includes all the basic rules for the CAT's operation, but not the item bank, nor any of the interactivity hooks needed to actually run the CAT.

This may be more a more convenient layer to integrate than CatLoopConfig if you want to write your own CAT loop rather than using hooks.

CatRules(; next_item=..., termination_condition=..., ability_estimator=..., ability_tracker=...)

Explicit constructor for CatRules.

CatRules(bits...)

Implicit constructor for CatRules.

source
+API reference · ComputerAdaptiveTesting.jl
GitHub

API reference

ComputerAdaptiveTesting.Aggregators — Module

This module takes care of integrating and optimizing over the ability/difficulty space. It includes TrackedResponses, which can store cumulative results during a test.

source
ComputerAdaptiveTesting.NextItemRules — Module

This module implements the next item selection rules, which form the main part of CAT.

Bibliography

[1] Linden, W. J., & Pashley, P. J. (2009). Item selection and ability estimation in adaptive testing. In Elements of adaptive testing (pp. 3-30). Springer, New York, NY.

source
ComputerAdaptiveTesting.NextItemRules.AbilityVarianceStateCriterion — Type
struct AbilityVarianceStateCriterion{DistEst<:ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator, IntegratorT<:ComputerAdaptiveTesting.Aggregators.AbilityIntegrator} <: ComputerAdaptiveTesting.NextItemRules.StateCriterion

  • dist_est::ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator

  • integrator::ComputerAdaptiveTesting.Aggregators.AbilityIntegrator

  • skip_zero::Bool

This StateCriterion returns the variance of the ability estimate given a set of responses.

source
ComputerAdaptiveTesting.NextItemRules.DistributionExpectationBasedItemCriterion — Type
struct DistributionExpectationBasedItemCriterion{DistributionAbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator, AbilityIntegratorT<:ComputerAdaptiveTesting.Aggregators.AbilityIntegrator, StateCriterionT<:ComputerAdaptiveTesting.NextItemRules.StateCriterion} <: ComputerAdaptiveTesting.NextItemRules.ExpectationBasedItemCriterion

  • ability_estimator::ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator

  • integrator::ComputerAdaptiveTesting.Aggregators.AbilityIntegrator

  • state_criterion::ComputerAdaptiveTesting.NextItemRules.StateCriterion

This ItemCriterion wraps a StateCriterion and looks at its expected value for a particular item 1-ply ahead by integrating over an ability curve.

source
ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule — Type
struct ItemStrategyNextItemRule{NextItemStrategyT<:ComputerAdaptiveTesting.NextItemRules.NextItemStrategy, ItemCriterionT<:ComputerAdaptiveTesting.NextItemRules.ItemCriterion} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule

  • strategy::ComputerAdaptiveTesting.NextItemRules.NextItemStrategy

  • criterion::ComputerAdaptiveTesting.NextItemRules.ItemCriterion

ItemStrategyNextItemRule which together with a NextItemStrategy acts as an adapter by which an ItemCriterion can serve as a NextItemRule.

ItemStrategyNextItemRule(bits...; ability_estimator=nothing, parallel=true)

Implicit constructor for ItemStrategyNextItemRule. Will default to ExhaustiveSearch1Ply when no NextItemStrategy is given.

source
ComputerAdaptiveTesting.NextItemRules.NextItemRule — Type
abstract type NextItemRule <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase

Abstract base type for all item selection rules. All descendants of this type are expected to implement the interface (rule::NextItemRule)(responses::TrackedResponses, items::AbstractItemBank)::Int

NextItemRule(bits...; ability_estimator=nothing, parallel=true)

Implicit constructor for NextItemRule. Uses any given NextItemRule or delegates to ItemStrategyNextItemRule.

source
ComputerAdaptiveTesting.NextItemRules.PointExpectationBasedItemCriterion — Type
struct PointExpectationBasedItemCriterion{PointAbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator, StateCriterionT<:ComputerAdaptiveTesting.NextItemRules.StateCriterion} <: ComputerAdaptiveTesting.NextItemRules.ExpectationBasedItemCriterion

  • ability_estimator::ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator

  • state_criterion::ComputerAdaptiveTesting.NextItemRules.StateCriterion

This ItemCriterion wraps a StateCriterion and looks at its expected value for a particular item 1-ply ahead based on a point ability estimate.

source
ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule — Type
struct RandomNextItemRule{RandomT<:Random.AbstractRNG} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule
  • rng::Random.AbstractRNG: Default: Xoshiro()

This is the most basic rule for choosing the next item in a CAT. It simply picks a random item from the set of items that have not yet been administered.

source
ComputerAdaptiveTesting.NextItemRules.UrryItemCriterion — Type
struct UrryItemCriterion{AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator} <: ComputerAdaptiveTesting.NextItemRules.ItemCriterion
  • ability_estimator::ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator

This item criterion just picks the item with the raw difficulty closest to the current ability estimate.

source
ComputerAdaptiveTesting.CatConfig.CatRules — Type
struct CatRules{NextItemRuleT<:ComputerAdaptiveTesting.NextItemRules.NextItemRule, TerminationConditionT<:ComputerAdaptiveTesting.TerminationConditions.TerminationCondition, AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.AbilityEstimator, AbilityTrackerT<:ComputerAdaptiveTesting.Aggregators.AbilityTracker} <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase
  • next_item::ComputerAdaptiveTesting.NextItemRules.NextItemRule: The rule to choose the next item in the CAT given the current state.
  • termination_condition::ComputerAdaptiveTesting.TerminationConditions.TerminationCondition: The rule to choose when to terminate the CAT.
  • ability_estimator::ComputerAdaptiveTesting.Aggregators.AbilityEstimator: The ability estimator, which estimates the testee's current ability.
  • ability_tracker::ComputerAdaptiveTesting.Aggregators.AbilityTracker: The ability tracker, which tracks the testee's current ability level. Default: NullAbilityTracker()

Configuration of the rules for a CAT. This all includes all the basic rules for the CAT's operation, but not the item bank, nor any of the interactivity hooks needed to actually run the CAT.

This may be more a more convenient layer to integrate than CatLoopConfig if you want to write your own CAT loop rather than using hooks.

CatRules(; next_item=..., termination_condition=..., ability_estimator=..., ability_tracker=...)

Explicit constructor for CatRules.

CatRules(bits...)

Implicit constructor for CatRules.

source
diff --git a/dev/contributing/index.html b/dev/contributing/index.html index 7a69b78..e011c01 100644 --- a/dev/contributing/index.html +++ b/dev/contributing/index.html @@ -1,2 +1,2 @@ -Contributing · ComputerAdaptiveTesting.jl
GitHub

Contributing

Contributions are welcome. Please discuss any larger changes in the issues before making a pull request to avoid wasted work.

Running tests

You can run tests locally like so:

julia --project=test test/runtests.jl

Unfortunately, Pkg.test() does not work properly at the moment. See this issue.

+Contributing · ComputerAdaptiveTesting.jl
GitHub

Contributing

Contributions are welcome. Please discuss any larger changes in the issues before making a pull request to avoid wasted work.

Running tests

You can run tests locally like so:

julia --project=test test/runtests.jl

Unfortunately, Pkg.test() does not work properly at the moment. See this issue.

diff --git a/dev/creating_a_cat/index.html b/dev/creating_a_cat/index.html index 36c8862..eef00f0 100644 --- a/dev/creating_a_cat/index.html +++ b/dev/creating_a_cat/index.html @@ -2,4 +2,4 @@ Creating a CAT · ComputerAdaptiveTesting.jl
GitHub

Creating a CAT

This guide gives a brief overview of how to create a CAT using the configuration structs in ComputerAdaptiveTesting.jl.

API design

The configuration of a CAT is built up as a tree of configuration structs. These structs are all subtypes of CatConfigBase.

The constructors for the configuration structs in this package tend to have smart defaults. In general most constructors have two forms. The first is an explicit keyword constructor form, where all arguments are given:

ConfigurationObject(
   field1=value1,
   field1=value2,
-)

The second is an implicit form, where arguments are given in any order. If possible, they will be used together will appropriately guessed defaults to construct the configuration:

ConfigurationObject(value2, value1)

The implicit form is particularly useful for quick prototyping. Implicit form constructors are also available for some abstract types. In this case, they will return a concrete type that is a reasonable default for the abstract type.

After using the implicit form, you can print the object to see what values have been filled in. This may be useful in case you want to modify some of the defaults or switch to the explicit form.

Item banks

Item banks are the source of items for the test. The basic definitions are provided by FittedItemBanks.jl and can be fit to data using RIrtWrappers.jl. See the documentation pages of those packages for more information.

CatRules

This is the main type for configuring a CAT. It contains the item bank, the next item selection rule, and the stopping rule. CatRules has explicit and implicit constructors.

ComputerAdaptiveTesting.CatConfig.CatRules — Type
struct CatRules{NextItemRuleT<:ComputerAdaptiveTesting.NextItemRules.NextItemRule, TerminationConditionT<:ComputerAdaptiveTesting.TerminationConditions.TerminationCondition, AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.AbilityEstimator, AbilityTrackerT<:ComputerAdaptiveTesting.Aggregators.AbilityTracker} <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase
  • next_item::ComputerAdaptiveTesting.NextItemRules.NextItemRule: The rule to choose the next item in the CAT given the current state.
  • termination_condition::ComputerAdaptiveTesting.TerminationConditions.TerminationCondition: The rule to choose when to terminate the CAT.
  • ability_estimator::ComputerAdaptiveTesting.Aggregators.AbilityEstimator: The ability estimator, which estimates the testee's current ability.
  • ability_tracker::ComputerAdaptiveTesting.Aggregators.AbilityTracker: The ability tracker, which tracks the testee's current ability level. Default: NullAbilityTracker()

Configuration of the rules for a CAT. This all includes all the basic rules for the CAT's operation, but not the item bank, nor any of the interactivity hooks needed to actually run the CAT.

This may be more a more convenient layer to integrate than CatLoopConfig if you want to write your own CAT loop rather than using hooks.

CatRules(; next_item=..., termination_condition=..., ability_estimator=..., ability_tracker=...)

Explicit constructor for CatRules.

CatRules(bits...)

Implicit constructor for CatRules.

source

Next item selection with NextItemRule

The next item selection rule is the most important part of the CAT. Each rule extends the NextItemRule abstract type.

ComputerAdaptiveTesting.NextItemRules.NextItemRule — Type
abstract type NextItemRule <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase

Abstract base type for all item selection rules. All descendants of this type are expected to implement the interface (rule::NextItemRule)(responses::TrackedResponses, items::AbstractItemBank)::Int

NextItemRule(bits...; ability_estimator=nothing, parallel=true)

Implicit constructor for NextItemRule. Uses any given NextItemRule or delegates to ItemStrategyNextItemRule.

source

A sort of null hypothesis next item selection rule is RandomNextItemRule, which

ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule — Type
struct RandomNextItemRule{RandomT<:Random.AbstractRNG} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule
  • rng::Random.AbstractRNG: Default: Xoshiro()

This is the most basic rule for choosing the next item in a CAT. It simply picks a random item from the set of items that have not yet been administered.

source

Other rules are created by combining a ItemCriterion – which somehow rates items according to how good they are – with a NextItemStrategy using an ItemStrategyNextItemRule, which acts as an adapter. The default NextItemStrategy (and currently only) is ExhaustiveSearch1Ply. When using the implicit constructors, ItemCriterion can therefore be used directly without wrapping in any place an NextItemRule is expected.

ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule — Type
struct ItemStrategyNextItemRule{NextItemStrategyT<:ComputerAdaptiveTesting.NextItemRules.NextItemStrategy, ItemCriterionT<:ComputerAdaptiveTesting.NextItemRules.ItemCriterion} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule

  • strategy::ComputerAdaptiveTesting.NextItemRules.NextItemStrategy

  • criterion::ComputerAdaptiveTesting.NextItemRules.ItemCriterion

ItemStrategyNextItemRule which together with a NextItemStrategy acts as an adapter by which an ItemCriterion can serve as a NextItemRule.

ItemStrategyNextItemRule(bits...; ability_estimator=nothing, parallel=true)

Implicit constructor for ItemStrategyNextItemRule. Will default to ExhaustiveSearch1Ply when no NextItemStrategy is given.

source

Evaluating item and state merit with ItemCriterion and StateCriterion

The ItemCriterion abstract type is used to rate items according to how good they are as a candidate for the next item. A typical example is InformationItemCriterion, which using the current ability estimate $\theta$ and the item response function irf to calculate each item's information $\frac{irf_θ'^2}{irf_θ * (1 - irf_θ)}$.

Within this, there are two types of ExpectationBasedItemCriterion which act as adapters: PointExpectationBasedItemCriterion and DistributionExpectationBasedItemCriterion. These take a StateCriterion, which evaluates how good a particular state is in terms getting a good estimate of the test takers ability. They look one ply ahead to get the expected value of the $StateCriterion$ after selecting the given item. The AbilityVarianceStateCriterion looks at the variance of the ability $\theta$ estimate at that state.

Stopping rules with TerminationCondition

Currently the only TerminationCondition is FixedItemsTerminationCondition, which ends the test after a fixed number of items.

+)

The second is an implicit form, where arguments are given in any order. If possible, they will be used together will appropriately guessed defaults to construct the configuration:

ConfigurationObject(value2, value1)

The implicit form is particularly useful for quick prototyping. Implicit form constructors are also available for some abstract types. In this case, they will return a concrete type that is a reasonable default for the abstract type.

After using the implicit form, you can print the object to see what values have been filled in. This may be useful in case you want to modify some of the defaults or switch to the explicit form.

Item banks

Item banks are the source of items for the test. The basic definitions are provided by FittedItemBanks.jl and can be fit to data using RIrtWrappers.jl. See the documentation pages of those packages for more information.

CatRules

This is the main type for configuring a CAT. It contains the item bank, the next item selection rule, and the stopping rule. CatRules has explicit and implicit constructors.

ComputerAdaptiveTesting.CatConfig.CatRules — Type
struct CatRules{NextItemRuleT<:ComputerAdaptiveTesting.NextItemRules.NextItemRule, TerminationConditionT<:ComputerAdaptiveTesting.TerminationConditions.TerminationCondition, AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.AbilityEstimator, AbilityTrackerT<:ComputerAdaptiveTesting.Aggregators.AbilityTracker} <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase
  • next_item::ComputerAdaptiveTesting.NextItemRules.NextItemRule: The rule to choose the next item in the CAT given the current state.
  • termination_condition::ComputerAdaptiveTesting.TerminationConditions.TerminationCondition: The rule to choose when to terminate the CAT.
  • ability_estimator::ComputerAdaptiveTesting.Aggregators.AbilityEstimator: The ability estimator, which estimates the testee's current ability.
  • ability_tracker::ComputerAdaptiveTesting.Aggregators.AbilityTracker: The ability tracker, which tracks the testee's current ability level. Default: NullAbilityTracker()

Configuration of the rules for a CAT. This all includes all the basic rules for the CAT's operation, but not the item bank, nor any of the interactivity hooks needed to actually run the CAT.

This may be more a more convenient layer to integrate than CatLoopConfig if you want to write your own CAT loop rather than using hooks.

CatRules(; next_item=..., termination_condition=..., ability_estimator=..., ability_tracker=...)

Explicit constructor for CatRules.

CatRules(bits...)

Implicit constructor for CatRules.

source

Next item selection with NextItemRule

The next item selection rule is the most important part of the CAT. Each rule extends the NextItemRule abstract type.

ComputerAdaptiveTesting.NextItemRules.NextItemRule — Type
abstract type NextItemRule <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase

Abstract base type for all item selection rules. All descendants of this type are expected to implement the interface (rule::NextItemRule)(responses::TrackedResponses, items::AbstractItemBank)::Int

NextItemRule(bits...; ability_estimator=nothing, parallel=true)

Implicit constructor for NextItemRule. Uses any given NextItemRule or delegates to ItemStrategyNextItemRule.

source

A sort of null hypothesis next item selection rule is RandomNextItemRule, which

ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule — Type
struct RandomNextItemRule{RandomT<:Random.AbstractRNG} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule
  • rng::Random.AbstractRNG: Default: Xoshiro()

This is the most basic rule for choosing the next item in a CAT. It simply picks a random item from the set of items that have not yet been administered.

source

Other rules are created by combining a ItemCriterion – which somehow rates items according to how good they are – with a NextItemStrategy using an ItemStrategyNextItemRule, which acts as an adapter. The default NextItemStrategy (and currently only) is ExhaustiveSearch1Ply. When using the implicit constructors, ItemCriterion can therefore be used directly without wrapping in any place an NextItemRule is expected.

ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule — Type
struct ItemStrategyNextItemRule{NextItemStrategyT<:ComputerAdaptiveTesting.NextItemRules.NextItemStrategy, ItemCriterionT<:ComputerAdaptiveTesting.NextItemRules.ItemCriterion} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule

  • strategy::ComputerAdaptiveTesting.NextItemRules.NextItemStrategy

  • criterion::ComputerAdaptiveTesting.NextItemRules.ItemCriterion

ItemStrategyNextItemRule which together with a NextItemStrategy acts as an adapter by which an ItemCriterion can serve as a NextItemRule.

ItemStrategyNextItemRule(bits...; ability_estimator=nothing, parallel=true)

Implicit constructor for ItemStrategyNextItemRule. Will default to ExhaustiveSearch1Ply when no NextItemStrategy is given.

source
ComputerAdaptiveTesting.NextItemRules.ItemCriterion — Type
abstract type ItemCriterion <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase
source
ComputerAdaptiveTesting.NextItemRules.NextItemStrategy — Type
abstract type NextItemStrategy <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase
source
ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply — Type
struct ExhaustiveSearch1Ply <: ComputerAdaptiveTesting.NextItemRules.NextItemStrategy
  • parallel::Bool
source

Evaluating item and state merit with ItemCriterion and StateCriterion

The ItemCriterion abstract type is used to rate items according to how good they are as a candidate for the next item. A typical example is InformationItemCriterion, which using the current ability estimate $\theta$ and the item response function irf to calculate each item's information $\frac{irf_θ'^2}{irf_θ * (1 - irf_θ)}$.

Within this, there are two types of ExpectationBasedItemCriterion which act as adapters: PointExpectationBasedItemCriterion and DistributionExpectationBasedItemCriterion. These take a StateCriterion, which evaluates how good a particular state is in terms getting a good estimate of the test takers ability. They look one ply ahead to get the expected value of the $StateCriterion$ after selecting the given item. The AbilityVarianceStateCriterion looks at the variance of the ability $\theta$ estimate at that state.

Stopping rules with TerminationCondition

Currently the only TerminationCondition is FixedItemsTerminationCondition, which ends the test after a fixed number of items.

ComputerAdaptiveTesting.TerminationConditions.TerminationCondition — Type
abstract type TerminationCondition <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase
source
ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition — Type
struct FixedItemsTerminationCondition <: ComputerAdaptiveTesting.TerminationConditions.TerminationCondition
  • num_items::Int64
source
diff --git a/dev/examples/examples/ability_convergence_3pl/index.html b/dev/examples/examples/ability_convergence_3pl/index.html index 4c783a8..2345897 100644 --- a/dev/examples/examples/ability_convergence_3pl/index.html +++ b/dev/examples/examples/ability_convergence_3pl/index.html @@ -44,4 +44,4 @@ abs_err = abs(θ - true_θ) end

Make a plot showing how the estimated value evolves during the CAT. We also plot the 'true' values used to generate the responses.

conv_lines_fig = ability_evolution_lines(recorder; abilities = abilities)
 conv_lines_fig

Make an interactive plot, showing how the distribution of the ability likelihood evolves.

conv_dist_fig = lh_evolution_interactive(recorder; abilities = abilities)
-conv_dist_fig

This page was generated using DemoCards.jl and Literate.jl.

+conv_dist_fig

This page was generated using DemoCards.jl and Literate.jl.

diff --git a/dev/examples/examples/ability_convergence_mirt/index.html b/dev/examples/examples/ability_convergence_mirt/index.html index c6c52e9..677bc7c 100644 --- a/dev/examples/examples/ability_convergence_mirt/index.html +++ b/dev/examples/examples/ability_convergence_mirt/index.html @@ -49,4 +49,4 @@ abs_err = sum(abs.(θ .- true_θ)) @info "convergence" true_θ θ abs_err end

Make a plot showing how the estimated value converges during the CAT.

conv_lines_fig = ability_convergence_lines(recorder; abilities = abilities)
-conv_lines_fig

This page was generated using DemoCards.jl and Literate.jl.

+conv_lines_fig

This page was generated using DemoCards.jl and Literate.jl.

diff --git a/dev/examples/examples/bonito/bin/1d71275d8737830c06dfe2ab9c7c93c45e3a8a26-16059389088621710572.bin b/dev/examples/examples/bonito/bin/6ecc99aedf04897452aeb2092e848e394ca61b51-10793518835325191992.bin similarity index 98% rename from dev/examples/examples/bonito/bin/1d71275d8737830c06dfe2ab9c7c93c45e3a8a26-16059389088621710572.bin rename to dev/examples/examples/bonito/bin/6ecc99aedf04897452aeb2092e848e394ca61b51-10793518835325191992.bin index d02fcff7b8854a684b4d90e099a681b7f7f6183b..214cf36d44fd898d9b87b9da040432c580142e5c 100644 GIT binary patch delta 10948 zcma)?d6X5^wZ>o7y*+o|zPG9wnuZ7p4v}ta9)yU{2o7j~Xokg;Ng9b_LPA5-=YVDf z@I4a*o*Xfvn1tksCIYvylx0*LLK32=@ntd?O?*V6qFFJC8Vv7ustRbR`XlN=*4n%3 zp0kHYku-?+DqTQa?Ot)z3X3n!7IhSO_uLj-Yy3BKTx-o5wRZN#qUP=@A#hyZbk{tqe)d`A$?B(P-ErxSZ_Ho2ywCJZ-*r6KcU;#F z%pf!!e^>2-SlvS!*oG4ZVd%MT=-Qs+xQC^F)eAz~m5+svo0UE zYFP_B48=Y|EX*GvyWkWq8<;I^;N-~T7hHF6lx|{qycf#tV|Y~k8V_|cVz zM(X^U>&Smx+i*k6a&152cTH8it0+BrVj-UuYgA>RFg&%S+nyoA#<5H@G;Ggy3?qHR zKZ?q{wrY-CzG9GAHL);2Y^+hXReUGF?13xI$n&ljjW|!;Sr8)|m8CgC^W2ucabni_ z$~$7*M>QqgH+{qQiA0BB<2{xYL^rK!c~%jhxKLT4SF>_a8K<5u3KuCOb=xr=88?pQ z_^xI7rWyP0y6Dh;WxFD-oQf6lbdDaOb(*qS%v#-0S0MyK*Rx3qqQSIXU(A_W9vVHk zy=GHEtiD9)#gQ2u(~NGKUdMqRTc=EjTGuJfs(5fteXGWS!oW8|Cm??e$8fy#RE>Aa zQ#l#AARs%%i+3s~>AsCaozU?ElVGsPtJtlUD$d*>-I^6kpPVlCZcqj!ZpDhGPbjg0 zfS|BAlyoa^n6y#3tgV_O@8{M(ZB#b5Kiu1Mb#JHTfeMD{$(L9*Ms^$-oI_HxT(wE* zZL8)erB<1JzVa(^^FE?&)~c3P-8DiBH`|unm+5(a;HOqu{)Vyw`{pRfeSK0VEnU~Q z@J#4Bgt}u|av!myJ5<8#II(!Ox6+;j>F3@`J*)vPp<@JOIr(B+hU=uE`rCJv#ce=U zR-Q!b57#N(0_R>HJz=VwlsIrTK+iP@S$ySLKH%z#5n=6+Xrrx;QT1w$*gRJqEZ)6P zX%iiuIzS>vs~%XcXSz7c4SnAZT##6L>UNd=WM$&$D@KLtNZkWGKvF00`={Jxw2o1^>FENjnfVn9U?3k?fZR2(0eBP2HkX6-+Ekbq+Bj&}0VvGq3x0bxEF1|HS*uFVrOQG{XX%UC_-A z05})4Pj@rpSE|grtlSk_*VT=VPW-jHT4{ecuJ?{>7=B<=Z3uS>FMc5AT-)3s{{7AJ zaIx{3(x;*sFRQm zATDhwtko?Wu0voOgeKcId?2|{LSXwIQAgduD%2Y*2!rUpQo&Nhy9I` zDwm6m_g4mrL*0cQUYVzV*sG7HfEs{5md#l)CAbO>9plzh&MN?@IZFM?j6|Xl(XLwx zZz>X2%N2kbOSqs&AYE+QM}UYFzW3{ehHqH#xgdGNqUPcTRm%uXG5gAzk)pe`cz^pNapr=B zmT#NB7aD;BCnZXxDTY4u<5p_;kMv?(!!tULG;Vd=Ll+kpmUPqNStpF`1)dUSl=zsB zOqo|)S;>fn@;;Sj_;cC-q4yL8VZiBqS=+o&W}QWB2ja1w;%X`KJ<1+PX#)tM0o$;m zx$}$PP}3BH#25e>h2MmYLWL48I3zluP+L+X-gY{+UBmBwK5Y2EMqx zujYUTA0jO26Cqk5PCpErFyNuZ9BB$kfBQwl&50uh{-WVTRa|_q<^vtpX&Yf^(mB$c z1y*RqS%f3H4%MvCvf{PV4h~MT=)FTV0tmDTVjnnwSCEVW2)un`?Q~VQX_agmR?mY7 zQaeK1O6nph(o@J2OY3jy5@X%k0qqYbE|3L;H%iLDe*l7&df-vNwnyKV6-x(<5;KC@ z&EkxOH4VCLyEIHRPJk4+2>@G1y|7&&(sC4%oVb5k?GHdg*-OxSff8_h{z965;ilTR zHIs8=tx&eKkav2?BW3@)B*pUDB{?doE3V&FJ3Vp5uYXkAP4fuJrmK^+lBSd#vu%lG zc;Ta;)~?7CC)x1S?%F$a4hIN1S#?y0LX5s?il7zR2^v;u99hChYjCqJ+2%~eV~ zux3#G7M)ixF2Dy=MdFBl*%Oxxu78{7IG98?l8eI=9cc=DvAw>dm?uud|B3bGZ^hv+ z_WrrDGUXV`?C7w`Zn6-yd6YxgyUZ+4wD#1mZmVJgm`@mwu;7W1IxpJZQ-6k{liN5*b~VsgSv^B1 zNoG#+y@Gi7iu&F>o%A>@T{UZtRJc#8;=}LPA4+B>@CN7=O}q?VNs;-*B(L{x;V9Lt zT&a`+h%v?`acJDA&6b$4t^OV63WPl4Jm{ZKXHGw92+8lG-L*}(7DO>( zw>dh|D~(Ov-Qu|q>m6z4k_K!LPHaz-c4!zREFKx0^^f|gse-syg1LNL>l6n+Zx{1d}djo zPQYLb^!E%aW^Wy``($_F?F~zERMPJAm`XgfpfkoImDm?kJO~4aGSV>7Uv5~LCl>dv zYUmQbeYxRwDzMAc2O9zohHEmT!-C6y+wiUqiIt;90*BTdh?e=;FHVft@K26T8Yll4 z+33V5smbz2o8STc_P0yAZ*UHLMA^(kywn%LBaJKb#4-A;+tIj=dSL(`mNb)` z)stPJ*u10h?Q#H7vqON(pw}>LTP)kJ3`;}hipLv);Nuh=FRoL}E03WH%N22#()1X7 z--2&o0^khFCMVk)9u6az#};eAT>zr)CEbLd~s^il3bm*^N!=>q-jl* zEW$J{$+GdHqJqt7oczxvO;{>NoW#kM>zf{uLu8+RjKeYafOBw2;oRA@Z%{HOCDXtY zk{JM{FYv4*vtyF>y}O$Tl8g#7m>5$cFa=nP2c*7da7a1##S!$c z${bGqLz9pygcik=gy;WsD(rO6ADRy6ToYJCFnDUnVkWz;{-J!^8`v>NM{e?@@nX!! zo0rG%UgrJGV<=`pe6!?4H?C@VUJ+YIH!s(=9WQjnYrU0D@s-Zz^*V)K-mg$XWL==; zV^DEYSM$D@QESjt2ot3YF*(O1QOv%!d1zXEp0%2>VOA%Us4}rztY6w}$J?vONh(wo z6B5MSs4)RJPwj}3VA*$d^J3`z@hV|5e2atS&TE=ej!$PG3kUc~4~;tvnst!HD#|OV ztSE_=*LOC{@*^h*@=7eA`G-}&@J;c+lg)==1uDV40Dd5yEl19`Qeu$6{1v<8X>*j0 z!peSeYkkXS<0=HkfqOIzfyw-g!|~fPYA8cLZ*2LR42g__ z3WSGDQ*<}C{CBIxu$^!rYM?uo5$$Vk`HGTGdu;Yhtdg{hUm--kC6Z~+Go|0FVsWKq zChp4Un2gxmF41vz%b+;m@j95pJPf7`Vlce%5_fTZH&7K{ozNoZYFVZJ4m-8GCAq=T zxdmbjW?0l#Ig$t%IrNY4`zN<7mafTarHS}|D_X9t6K8a{wrT<8S6-lSM965&jj`y( z%=NMIaiV_wW#F2YKIV^%4|vd_gY!cAJi|+;X){F2(p)jqw0#qzpZ=!hg2R)bWQYNa zlK;89CcRrt$7$ZHElh%Pl+tn9fY#D5je9x37Q{5^>Op7Aj_A|wR-)gROYI69IbIwO zfJO;hD=~kD)lm85)i41dM-5&K*v!&K7sMw9mJVoH6-h+;#zhy-Exlfp)b(T0UuKkE z9H@ic4Ag0}co8idj3S=Pd|nZ3<#h4jmijZrcY8}K@)VPnbm5ZH9ZZBlGg>lv5kv_g zt86i7X=$y7ts#mMeM4Fa_UemAmzMSoPS82NAi)>xf$Bzy3go>Hjp8-`QR?M}QQn^% zD~_*yqN$hGPbox)9xLfee5qHVlVe~b`9E02CJ5H_#-~pa%9HHB>ST4|JDYew_>(6~ z>vgtCcZf^5`Qa@vZc8kAswBsQa;8L9$s=;x4~O-nQ$c0sFVq|t1>;CG3}zn7A1)47a*)DxV*EU$s##~1e39i0uP1MM$=P?Cw4 zr+>Jt4itkwEZrB+@O1hHCDba)<1AL2sYDA*i`dd@9A+G73CGF>sDCQ*l+q&dR+3<;uf!^)we zxdxGfIpEVoGZi}dYD9)HrDOSs^6VUyq;PajEe}j<9ysp0%n#@<F?-g?;$<@rue=J{I z&>5bBHn@hXA^DjJDV>~#06w6K^AD6~X<41?)hV3hj zqWjC0f#QbUTB}A$k;ToUU*S)Ayl7cG-kjW9S)qep$IIns0nfBm*$A^nIjSLB`2K~D z9K?<#mDTz-Kn9ElVG+u!MZ=F~Evck_ea=&dMzEagnbICF^iz7nH#Tiz#sii6(>fO4git=%1k}XA zl2LM1^1CgVu$mPgZuXAB*A0b)dP)8cPo>_-Z;0@*ks5Eo-zySTvvSGhEuu}>Q zO0seZt}D9;ZQ6brm;f{}PJjl-V(Jb>?7mv-$yFpgM~Ma3Xq&}5!z;BgB{Bm>&fHPX zJD8Nj%|~)?_}2AWuU5^dM8>%-4~>g9Jf(FXMb-Bj#mu=yzH0+FnShyaQD)nqzHGeX zTg0oL)mCWLj9k1nRf(?=f1qsX?gR6m1y!M<1rp9~?Pcec7O=NHHi4}Xa zHSFI8e3=E1X!7=wp6~SMwc$!tY<*GFi%t16TuDgII%E?W`fJ4%SfC zFxJOdC$NUIMzBs~ox~c+8pS%9bqecL)@iJdvrcD?W_7ZwMM)tSPJuSr@TB$(qWV KCJw$d-S~f@iB4bu delta 10852 zcma)?dz2N`mBy#)-oE|5eQ#CsXdVg?)Ck?y`=MlLYzQXGOU=W?@DLg$5+4B?jTta6 zVAd!CEgTFgG6_kHt_WP9MkYpLvO*kD&@m>)pg2j0PJH0FjL|`z->JG6+FSRJajhdP z&hF|{XMg+K-`=}^`Qyjc2UsgN92o!Q{^!Olm>-6oZO`kpe0P4QeCl z^&Q*nHE#Ldl4VzZ^M}{ZyJ_)t^A>D)>>spe{^gntzy9f6|KA6^klnrA@;%G@ZSRKU zFOTkAdDrazNplAG51Vsl>2q_$mo7QHuz%Gq#|uN#3?0`rUB`Dr&pMF1db{KLreAVQ z$M$S52t&v64cl_Ppt9-hHeK5dUCZ+w-!=o^4MIN@`xjp&$lcy2psYADe7=hxUsmTwEU4LRDD;>qwUB~u0 zjmzVVz;VQ+Es8JpZ)u)Vy5P+YwSBjlHWG8EsN?$Yd~1x((E`T`ZG&?-f#F!5+5hGV zw>0d%_mzJ2Zwrbk12OAKt)qY4-+bM1L))=K-*8;lGknK$+|rvTEVaMo-33~DQ|z3a z8Oje1>VNRv_WlRo%Im&oJAvgpA?|R^(DF|H^0}&3O6^vDb9?{36Js!G-WksNp0Jbv zLq<`wZAxZ;MjZT%vP3TdoT~k3=g#7uiDIUy-WToMq-H_?4wz%d-p0#1hOaW!6a z!(+K86fya`@w?P8NZzG!i!xTT12Z%s6x$0O3DeLz>6+oHp5Bdpa*h~u4>u`&tJ}V5 z*oJ4|8ZY!MJG3l`4@(s*?p5YPFxJYAb-xjF9MY26z{={`IfQ!eYv1fQxT`l{dO~y_(a&s23VgTmW@(!L|_* zf4EqGQ(4*$KUQ!1Ww3TzYLD3Q+3ac24pV(SakOShxVccJVcOgg4v*%oRl)3SuDS{i zOYc|cL>IF>b#VXlALcduQt z)o~2pwoGyOzT${7Z4I5KE(P2v8~-EBHEyjsMk819Akvu!ZIRPFGg^N~{ga9~cb&SX z#;(M5a+bH(sXMhGAOi;unL2Purmp9T8;-PA=<~*n>M}iTVh`lcTc1@2$8kntv&kwX zH=ag}z#SFVS^qnAHP}w?i%EZM9v@x)oVs4=cqmTqrsMfsuLnbfrUjBbUu-KijEGKq zUY(ZVQqucIwDJY@!!AJT^P6J{VFC;%1WuyAHPf$q5UP#8T%v_rCuKRnR63g9hsa-aHE-aU?Z?Bfa8a@$y=5A zWA3Li5;N)jG=ZG>=L`fQxl!Zw1H*L(BJn+`jIP%$rk&Rosm~0UvN7hB>_F2Pmmw(aH^N zuc_jeBU~{)PZ>ooY;0THC1$r|Ptz=L#p&@bLXND%3AT+a&WwM$PW&;Sy|RXhSbvi? zM%>(%y?;owwD-oPoq=H>s4O>t3_Y}wnZWMa(Z%hVP3_sG`mU6Pk;FkX;j--FJfXyk zzz6}-$ddZkE??!E1=%$nKMRkuT;)A=h3v1fM_qXMq?EPZKxh;)a;KCqWfsdFo z`~V>)$V9`U;~z9EP@|*UvU`;FQrgIkW^^`9=n=<`W%nX0xqCyZQq$zUd0|65_?PUl zvWHM4Qi}wzJ@f#14^(>BzA9amGN};ejIO%=Bzu0bv+gq)QCQxRM}op_1~d!$e1oDS zkSYoA-)q&a(sre`W#$Jm2>Ruex}+!)HHFmO zAy;6;eT=dI{Kre`DF)IuUOD4@ZA1Ovl$gW{v=9O^H$KNFFr+*%`Ym?K+)eeXAjK++ zz>8&V|nZ~X=SU*xDf@Oxs(-dQLsyySQb9_}kXJMh*M06i$J6o(hR9`A^RmgM`vLRYc zgxH+51Br_;(VwYbs#lxQVJZj7vqKv8>Mr>m5YslnA?QG#K`QC4SiaAJEHP_wL3cZ;e5Lq9h=*rDWZF=+E$dO1VMDLNKy2_f@p4Bko|dn z!)mRR+7&w{XF9~uoehJ<0}C29kqbPKf=(sf$1NggPkgSoVZDs+5S}GHQ)1}y>**EB_+mJ*!S@CnKnDmo|4`m0TKraMI zfikAoCa=r$qn&Cq+Ig5hXorA$Qll^K-PQ1hP9h9l ze1NbgbYvPru=KYM|KY=>hHS0f=y24qyhq+_L6fSABXK__IFbH}h2p1|w;WN#?DrdJ zkk~03*|IBZ=(Z0Uh;MoTuno6MawaR-5xszwsyKab!+YZB5j7`{{%f{Vyl=G(5}VY< zWwcVNet@e*vUK-|*`I9OL5xcIOu~Z^AqDsq_Is-d~=#b$H{~j(GKK!YUF`fS$kH~5)ENpWikLN z-DfxL(WnHdKSQ(v1+5{AE>wB9N;&NLLh}?=$A3s;aytczX9H(+VIs6|$u_U4wVl9ZYF+c4s4+==7V_Gn zLg7`={cqJZAB)QlT{00pG>Y^Sv0G8w?+iW+J}A)e@bri#i;()LsBx_ zr&jaI8=q-jTH`0Ac9_&7_TJgNOLxd1wAeu|IZ1ZJLPOmAU0yowib7lPs)U|YIpi^i zzE?~w;nwD*V6T<|o_V{wT=b)Fh%2s@f*W`S6=K=`U(K@UPi@D{B1VlU3>Me^v3V2R z<8o7oeu`E(u|N_G{ZsSnI*Es%qUmVTOmZ3c*cHzE-OJ5=HMe8HMCh6874hwJT2{qb z6t1T^N*a*`8VxSWfHL*HGrlE0VfFTa6aK8TWwS0NFGUWsB=}pVdczTmx>}CKvRjit zCl>)l?$F{@{3Nv7>YLYQgl4x$y`44+%c%K7UyCDdCruK3lHDmU0@H`ArHYG7JfJgE zR%Ge@cxF(ZU;OT-mZT1#D-0WR40xGF0D8?B$ThQ9#0N?n#cV?=8vAGqsmPKeEwDiM zAg<);Qj7n7wB^{)a)ZO611?Jd07}P^W>*uerc%Ya#*M`wuET7q_32HY!x zfjW!AfIid7=G)C`p;WakRxHa8PnvGm^t6_n_P|*l&2t?ZF&0_R9oTw!V`{4u>nWQ` z?_u4ot#y&Hrgd3;GG~INaWmsZ0LlzAFh%#fiDakUzHxo)cO-DCZAh=D249zNXx*bp zAEL)8ns6vBAjcGReuoJ{?ZnzXs%q_ap79A=yt^?c)b;^n3J)wK*|`sox8F3Stuhe!Y}@C_jmIh(e`<$d{n zjlMxZ@FQ~JXQY;Cix2wpm0&{wll6#Xt^=~A)ld$$pWd7&*lKJiMA*H#5Gb;$I4cu_ zMWcrTNa$41p=17KyKy^wShQ|`{(Q+W>HRpjcZ!x@@;!vS?jY#|&;h?#4m{HPo(#HgbPcOC2IC2kEE>_v+%T6KKS9v;ST4G!Xt`+4 za_u2S{Kv1_dU;^l+)KjnssnAiG$PNY`9b>g(9V3l!9bmtix!VA{xBn6eY~wtPn(oG zdC`pC;>1>QRY&1WL?^12!i7>_mci67714L?@PgE&sqG}{yb%S;U^FP123iA{GE4Kh zhxD}S3p8?Dl}QDrgT=QmE!-E&GQOISQ))CQQdxe%;y8&I;8TQ- zEC)~Tmj`JPQU8^!MsqBX(>(exRE@NZt?~fjwtEVU@>07eM=_7lIp;$O1Ta#W%Vt(> z2T?MGx%2x4rZ6?O6C{p(xo{b^oJ|)>N+p?PQ3n7h{)i$O!t}phn5(BvLW#m_y2K45 ziz79Ls9=Tsh{NK~ORShLJ_^Boiuj+?i@oT;YGYplaICm}RPo1R?$@*&CjuZOZCRqj zure3K;{;GRJlkGMZ@cA)2Z5h$72mxwo7a4pp9jc}71cL&U^&>2ezkbM+RleE)!SB5 zc3iQnD63Oo%L~yXauK2&*@fZj`-*U<>&pfYpydB_`H+`RbbntlREkg>T7!lMDwr__ z?L+x-jhq)=01B%&D}%y2?<>l9p_xw5F@+`KsW%OXoDWD~%H4`kA1Gc`)1)E_28ka% zz~#mD0a+M_Lv*w*sIaLXD)qrRJBzYDsAX`HFh7+QKY6=Y5l=Q*jN4`;FEPnAl*h}Z zcZ&40-0JQ4OCKqp28o;hUff+|M1YEy6)pt;MC754hvNJrr^iqVAEnVsVK&9r3bMW%5g|?J1)R4+lLbTCsM4+ll-;dj z_v6}Nv7)2UrjvW&89+kYmXVu_i^|{U{M!?n{MM+3L4wVdPilAS&{C(mCjrO&>lXdZO~lOJi?t}A|qYgXoh0xVQtiOG&Jo*9zna3d>=>M z$?x^t63=oN=F1TwDjH2@TKh{39GS19hxyNp8;{;^uHI(;KJ#bsz+vs!kXnyC>*M3a zq-V5_K{dYdJL}_O=F?h-Y~EIX94InRYbKu{q<&#^;nUiUiU#R%l>@&u6wmyp_MKY) z{*`C6Z;e^Ng-};}oxuXv=2J<3As-a_PUrmj&BDXdSjrm`+%eTwyI)qlLV9=xZ)O_2_{Gqfd)&AQCyY*&K%&J zK7GF`?){ek_y7L4p4xQk+nY|ka_wU?&ma8NG}8#3aAA+rX&7d2r{fuB=fZ{Oc%4R1 zuNn53iwwgHzw#e_1B<@;?PUvBTzK)q#n=Dj1KQ6wUa%Vki z&jsf#J-74Xr5CQ)dgVv%n(BH+@XOw{8$J2MLmNHE^1R1N@)z=_!P~C;z|eDd>E$b5 zKWFfo?WX7Zu4lQH%U{!RLf~A-1KlE(Nv4X&JY}0c>*Zswk+sz@%bVJi{ z93$Xxu4{*uJN#5I2!V92?H-=3%2R_zF`RSrqU#_rc&-3n)SewYTHoOBpjOZ z&=Ty?G@W^b-_53c|Hb$Zs`5-%^# zA2vAe4@XUY;KK8k_V)Fjdr|Mw{=Ux8HF2A1;1kneQ?7XD^y;L+-TNmE?fJt3Ro|8| z2!>zPHn{9hZG&I?Yqe&&94vOQW!X5?klXWyUf+K}8QlAq9(G_gd9}NpI!FY$KRB}m)=`6jwohKoYCD}wO{V0RSyY^;{~DXc_D7| zY}*#MUsjtqw0r-T@N&In=^XJDRT($9>irMaeCg#N3|yZ~vID~u^QSi-9!RU-2Pfger##1IrGaKt6BqA}fr;oxbF~yt25B<7ACqTAZIC_S~k7 z6W=T-+iL8=4nr&O2`RZ7`VryTl@n8&mCLluVi3K&v#@=ZIQwJDo5Gk~uGL&hg{(K7 z&WNKR-aPDv8O55;7`xa7>xK+D4$U91unEq)3$vlv>nePCip_&i$&2}bJZP+ zSTaxPtz))lMsueq&F2h;WmjG1D9#`~!|Kl{%eBnn@EQJQs&mDzM;j)KrKWmH z)VE&QsH|VDw$>~%GBoh5=TLyAV@5}=R(C6+c%yQSKICPVg>)&IuZt&dRHkdDC2P>& z;1s*z+eR2?ybR10YAv%Wro5rF$4~$27G;{|o0iQpzHj^N!XiI}=@ll)r`uK0eVfu# z&!je2jf47txN=9KQDX;&LmAs%NJ0By;K)Ru6d4~?@5qTI8DJ zOZ&(WV|sH{Hz^p23zm;-fg|5{(ingFwK&Fg42EN@i#fl+B}CW?1DOheWq6hm`l9wb zWztBH?^eX--zvR&*3O9EB=dE=nD>UVDL$W04f8HyV}&7jNHHe`m>%%yx0OC}uZ~rk zz9XTJ2>pIV56E3A$&<|{Ade`8^oXar>b3QjM-N!_)bUA0x)um-_zr-;%N)ls#lj~V zAu>~?Pq>Wgv7lCy*sCsr`1mbk@cgH5XsA@og zXND$KjaLGCPwE;Hh+V8XUhR=(S;s2HZ(SI()aklqa{$9I9A3en08ooDP9`Gnxp9u# zTi0gfEyvDPr)idL@e0EXJ=u_Wp?PTFCvvQeNfPteBPX4xPSYJH03B?Dz__+Wz}@6X zBk2Q1eE($jQ_nC4-rKIjJExzbZW3#5R~mGH%CjX_corTeky8AY>C>JEz_aV4xc7{B z{TrMNSO9fhUO>zP@XLtIT@`@u%{639cGXQ1nVtMd3#kCtHKbc<))lcT|NoF~SwmzRbS&6QGQQg_8yIx@Ve#jnhs%2QV z=Zg8|-1Q`z<4btAUB~pTz-2>@C;r3|yCX4h1b!f>kORm~bYmrFEA0uTu>dj0ls%hb zuqaZ~jcBy*sp8jK?rge#b{#(Bb&0Rf$&ClgE1G0V0redEh7T>G*TglMq|WTQxpq|_ zs$=uh8O<}rh7)qz09#NPW&~zgmSqsJ!L9elNJ^+Ce>B&xXDzG@%=}4K{WAAQRWlqM9|oXt08s_sQ}QtH+1w{`dUiFGv0l|x zr3gcZG=Z$cRYO}mI49R3R=W+CMY|qpeLxYnRPyKQW3437IzDKaCXQ*%Z=P^ZfA8h} zowjAW2B-qgQALEviSWfKIdT3K+6r;Mp6}O(>e!g4l<82*&dled1E=TDsEAFfQlUlJ zfE4uHvEb2$MXXZ%K523Onzo@j>(Z+albnEl5Xf|e&s$_09ar4bn_pLBe|QE~ZrD%= zA7XRXIr(*}=7X)EdjNnq1~47O?fvlRG_R1)iRl;S7wTD)Ao^xQ!?(J`aX011iz}BD ziX`j>){rt#L;i$``Q>(GDb-SL z05Iwp$;fQ%66Y@|bZB$~Xt@Q3nSd38hLeylg5s;Vm?MPG#~W z-PL#Hi+Dkuki4LZ^0LA}J>xVkzZ@!jpCdc2N8=>5NjRTc%c&LBev0+ZOIzvGUE(^Z`T>q%?$e@BA8%NxZ_6$x5%|x?8;F2I69^sKl@fv^ z=aPjCJkjuJwZ{~2&qVe!3se|078s#vM zv0>7UXW8iZwT*dIw_zEc<#@QoX1kKMrh%BczVX_6%Q6r%#c$R(3SMu(Vra*qOw?H8 z3Ek-5MB~TRRGa}7pknYoTdD(|?-;Q^lY(f!rSWWNNX8_(wV{E4SWh%|XtW85dj=rS zG)B(Z5yQnjs-9i&6F{23yYbGr8Z=ZVND4HIu4U4{O)I8$;`lYW9m9QsV%YmL8HucM zkVfM6!-`upLW=H%iWu!(xI+`SKO2#u2r=hvCr?^s+HFD_kD8m((E-v1n+ipN-^QF*2 zQ4nBenU3D#APYX^4xk6dC_Py~(ZjvPqm*QfrrE$a4LM`-SZI5Y#~)QDrHS?M`9(QK zXN@9A%Jx~&{yU4~@;cedbG(~+WHN(>4sZUQEyZiK?D9dIzh`Sv=9YXIpM{VC%P}0O zE{H|jif?H^EQ}QKg_uBAY{qJ4kcrW(54Wt&iBeST*Rv*W^g?r~yIcI?-Qo#Vh(5xa z47nFb`Xp-Kt93*dO>cf)5g$BI9H@;oaK!C5D(!L}ZxeeD6n9j4j-tb*WS0C%7`rNy z4#})r6fv`@^sZR&nbIlZOtoo18~Y5b9=JWWTTDKw>23x>Py?z#2bX&Sbp!F*Nli(L zk;^{Y1yty+0o4ww#L`Am?B71MsZWP1j5Ug0jFpYMUMy)+@;O4B`C{oF@sqDNomT@O znN3k#pcE!pX+~G=l>%06P17nZvl!2&GsQD&ngp~G!OVxrGh@^8Tw>vgf4OdCrv|g7 zOS&;$B2Gd0)9%o{u1S`Gn^}+B!)0^DoM)O|7Yp{P)tYAjNQ9Uni2iJmJ4jJ%Zo+d- zEA=5ivn-{kPVvHXO)_uDHjlv&a%)-+$|fvLtopB}x9Do{JSrExLbe{dw4Y26R}K}w zo)gx~P5t$3(!|+6t@#AnH{%D%g9a0zhsTPslt5+fd`dEa>~cJb$BA=bM)P*vq>K%^ zgOp>*Yv#o|ROyi1`RAF<>?`|Nnmi{hYu=&DSY2#6fm zNkHq6T7GJvd7zF-QpsXW+Prgj^BD;+(SMnG<;2X;3*j{V6qv%D%>YwoS!%dlLbDE2co+cy-`S@4ZlUxxA}3NF zR~`^6UR4St_(&0`0LWDddmsznEpyhmbe>G*aoSG9uRXN+YozRVV6?p(s(!diFc)wJyPC{ zkX;e?*SuDu(`T2BQA%{NB91w_Wu3;fFXaLFCHGpuiEBj2Lg-6dXSS@=v#ast%}jAw zXN%B>P)ww5j2Ag4P)RXP?P_^P1Nvx-)F+~XPwY3HY{)HrUN+4 z`Xr=>qQnpl1CjS!Y1i6!dCNdOli_QRHnCwvOOiHFV8ELJ2uPp_Pz{~{`zQY2-K*ov zEi$~>=ZrLUUU;wt401y@M*0J4FiCoVDeicvB@GHf1%>Y;gfZGs<6f8srT4KGf|6a1 zgK}`{$mX?E{A;oN`9z=(AfJdilCN_hq)lrWhITiXSJDnM24aiEp>?J51|5(=c*CU> zGp=Bow8fm3^7V2m;u67-v{g#DINGRQxm}`tcKNVao3qfUB^f3UC|x<@9jeWti7l&@3u5Km@&a-xYmG|RDURwc zZ>h3>CKglyfRme9SInE<+>!FM%_o=p^|1!*Evd-pe^!>SZDhcJ)xwRD=b5R&JPw

|wAOW+Mo)qXV=n}21l_NB0KF}H>EOR1e zxWJN*T`MKJ7A1PVTA8TmS(|ih_^?*l0`GxyApFRoi(!T9>7+o9{i4=|LOs6nCW$lF z#7!}7-no@R^od247xOXKir)G{<>~RV9YECOohN#dyeM>HjqW@}9M@lYquz3YbV~** zH^}Qyc^5*SFw%1AX}CgLURg_rpuy81Wg7)`tq`mhJC|49(4?46{sOLCs4>Kw==1Sx ze)*Rw{d#tFcr+gwLEId25WJ50nY(d0`IGD^H+=uW3OSKkP08V-4^<>KOM#HqNnOd2 ziw1AI;*y6eQW0mSfOa{C)II6rv@9bU|479eE-k1Uzef7zs-MeI&XqtMSNXqI#aTbD zP+wUad6!Rpeew|!rS#{DCT|F;nxuWCHgXwAgu;M>BvBhtz^SsOf2~Nh5&X!QjwZSG zZe?>a7RC3(a%iB4&;z-lif7rk->tOgB-FjGZdBH3rM@0$0 zG(KEp4bR*iC$_HSRdoywD!W%sY~3vGUt4bCLI;I{t13K+RE8phhB+*W_vFQlQ(Gmn zW{nOO#<#9+{cNrVkWwy$g}-p_^k_*19=Wb{QPv>Z@_g>XF7fv2)`G}C(psB%Z=zo@ zgdnIQTk_2a1c_FVOe!O4`7~8#LdM8RruUA=TJM~=GuHWNn;|>{bl{$zoG>|Ta=hce zsBO%N+a7Q22jw$X$JS(5iD?3e@i`zDKat;2wG{EvhDIcA`GpKK zH=qd6GZGA6-dr)IdbT`!)*#8?WgXRmIC@^SCWj#Tr33DR#1dPs3mA1B`Sr&Y5ehfS z`?XJ1dyy(LHp%5^0?=KjSL56*YHX8{gR++VjN9JyDlu&p+dfy7{T*UB)+!}NWbi3@ z;oe+LZkLja0A?tv8U2YtaO6UVJIP~it#bE}wHdw9X%osftJ~q43}SR?W*4XfuJT52 zbV%QmzEvHlXEU0;M?_D*T0Je7v=Ig*IH?Dw@DmcU8732QlEky#jD4Rqa?%b`eGTVN zY00<4q9ki`4A~Qz^N3^Ik46%h>mJ@#5dS{8wm^n6W1imneB{X5wi^5aipzHma1>N5 znBdDtw@iv?DBsD6UDIjUpWC5gOdd95Hmx~Y~xaSb9!SKtnmO@n-NYv-CXNs zwv&``jAeWyIq$pDlT;!k$CIOntYup^Hc0)?w$%9AW$fZXeeJA#awSR=1!+k&ge6{u zn-j(3%s0^x-5)XerR9_n$g^d-PM9oW4;ipn)C~3?YDbz>xPo zrNPZng@9BLLvj#Xr`F*bFE!xD`P)@6#xBG&1{nuHTKiT=%}Z)%enJI7aQ~O SquBm4?drMJ>VEq98u9R#x;VOphn{oV~n6UzgtzW!B_8(`3I5jzItDM z_uk)fzxV0Ar@nCSsY_QsGvT8C*AJgPPwO;0=V)z)X*g}x9Ls6*t=V(i+?wUVK5@ zB^NKeVqobp500}8$9bl)bM*#?|JHw3@Ao!1wxfB!}tQRTrI#?wXEkd!FVwx?@SAZ!YA~X_O|wIN>lsY!U>hLKWi211okigyuJVZZ_A2p zo4$>=(_GEeOxJO28rYF-t(5k*i2cofn~hB*LtOgoHsK%oD}NhC^PkNW72k3Ue7ox# zzNNXguj&1}KesDmzMLtlo0E&tryu=lc>g&v$g!a(vy_E4x28#r~td zp50ujvl@N;*{`E7mn{RiK_X1Uvo$=)pxr%6D<1+j@Qu)Kd^<0MFPj z&?#wQwpytZhAm8>kJ3U#)@%zq)bSwHoTWQfuqh+BDJAjmUa)*xzA}mKKSlV2mc5=Y zsL6N@dS*xGwQ14V`Fd)cdc>=wQ=- zH7dh%Pp{u!4Z#}9~&M6_MQ-gvb4hjl@5RisWwG6{@QS%z105C?+lzI6chI78^Tmpu_3myAQf9!*`oI9_oa)5V*+ zHjV_1hu0gbdw#}ju}(zz>7~bsao?kmQ5U{#YS;y1pL^rEl1_rQ!b%DzGmmi=FjC__Gt}))T~Ws&C^A@D6A>w6Er(O!)y(&vuBT#Cj1? z->*MWJY2=`Tc}I)iD#mK+!(SC>iPW1*hHN$T7I(lDvdcgpG6OtfNZn~sK>EQ%XQ7* zr9Sa-iS*ON#c0ra0YmupI(6KLoj4W<(DyV02cda3__A8wC-;ggmELU8HpVxKT7rgYSz;Q*`f9OMM&YZwG9ZlMitNaN@ZDRWkE@0;Qq%x4PWKyo}+>NO**3}^{Evw z0mGedFqLrWS-}@Y$rR{?L(5rk3EvAqNa9^RblRyHR> zBK8fYpPf3tNRKU+ORAw8tVe;)(3L*)r5iTnivry_H+5a}pq>nf80WCJ^22CuXNsWO zAc&9|ydG4g1!w@imgl9`$exP>2jX~^&+Lc_eF*<7suuS?YZuGWZPvt&N#K~);A$z8 zlIW_1safFD+L&PMG3gBxY0d4a;Ugbjc;UsJU2R<3p6M7iRJHB-ro&=vb-_-9lQvn`Ca7~seP(6I=FK%*DnP7j?GLK z=$K!mdemgJ72K51p3p(*_fi8uFiSI#ajtLZD`-vH{$6TNoHJi{aaI<37T;x~B0}-V zs_B;!t^6p}1p!?f;>Qs7>HWEW+BvB-Civ#Q+__@#><`jKL4|1Zl1LDvbmQ1t( zPNtfsv_zZy^!aG|I#I!A3sd(@q^IYn50!1WDKx$YH3J4VAlO5phz4*(deBd&Y1)GH zY&8+)Tb!cT6r6{nv#5iRaKNLQPlw$gjS80S%;L-zEJ}B)$q*+_i8Yvh0QCvY>^pF9 zmJ4}_w?Tb=IhfuP@fuhajLMB}TWm^n-3kU;au7UQwx+w(LGV>gK(KULe#<1fWMBH) zC?UOj(*-F+N>~UD*k-}k;OMbgsLD7WH+-93fqqH`gu8BAz05|Mac8Olr~$liVS?~Hh$@Jpi2CBJ9DjVKiz_1;73U?h zJhMrSsKfv?=#B48Q~i&iVf0Xc!Nc<;SA!mtXh#lpVV27#70qQJS$Ag zmIaL~1bSq9<^fs?(pS;i=saiy1DQdsLkh>KIrq5?SijDC$okZ}Bg_jbolQO%A9T)u z?)$;hJ2GdBu+>~5oExSV6BItwCmD%OdpR?+PDq@bZ+B$UG+)TJKpUAZfK&r0`aCOu z_@}<7N__|5Y#jYj%FYAE)CSVF1?eN`+EjKUjXATiQSmiHM|PMXG-wAGqJfo3oSs}h z+l5L*k<~;6pDjvvbkIxFvJa{_eU?EM1_CxqPiT4SKP|f_bUP|`i8}IdqDVdWmaTJk zAO_DqGrJgCxXyZ*p5U|D2*-dxSDZ8!$#J7IB_nYveq5JZOLt`1_pP3T!aB?s6`k zQt_2Y17sOiMAborf{Q-Y%0VsmvyxdVjwZ(OaDr^^@FnJu~hPwvx zmKWr>#gog?&c8#2b0ihsh%u9`fd;0+{D7QJc7L1ESMLHm$bst;R ze1|HVbDyY?GjInWY7>(;;16tQl(82oIWbjlIgsDYqkoRrmz$zsi~y_$dpZ!hC~lJt zV{7n}%bMOsZGV$nq9!9kGk!E3xj(l>0U^V|LnpxVfKVXShstHHNHcfk_tLgp)}$F% zHw~lTi1{UTKEd^KTn8;vvj}_s8bvdMis6}{z>_$fS`22D(ql9pcf}a1m|1%PZXpYM5RHx z;Z?wlRgr>9xly8j-<|Jv_i4NKJ80DvBQ zo1^KRqZ$TPs0?T`B-cXmqX~Qn7c~s1KoSq) z4={kShKBxxRG4>Z!=HJ&tn2Vy5br)90q`1E$`L)-YH~F!l-t|she5-6ArC7!d4L@| zc(4;s&dq~izjtc`xHuUTc%h?tt{ypx2xgu*qp^!ETrvn#l{Q*3y>TP;E==cT-vx27_5ez==rT!o4|WV1 zJ6S>d&cQ<~iBGtp4T{hvMNi(^IG}RDV`c=tgD}D_Fb|}aw>5@Y%E}-N<~&ppsunC9 z?>zECC*8?k0-;(H6Ph7Z(?z=)FI3?yKsqiC0Huok1Ab!IAAGlQRlQ}-%>=sg4~;NC z7$X2yz#M2N3`w9tEjsdlu{%t=axz3Rrc~G!=F%`9pmV-w2-^SH7?QP}SdVQQ>zg#j z5hx6F50CudHykihRcmJ2npW0Zj<&X=t%=wq0Q144KHvmO9|`fPXJXT53O0&KC`$nT zHyh@SK+&Uu2xrEcO-t&81m$=0Q#$CQo~9`>TOsy27$XAg!PoToBylu@&6#4I#rD0$ ziS+RmO&eRbcFmd9-QBqumK!I=bOa93K^H@AthQ2CDFd~!p)ISH$Z=0Jp+wMI@H&jC z7@}bkpgU`yY>J$j!ZwG!J%*pSnZqR6BKdMTOv0}u+V)HnPaTs%;mi*;vO8Bek1@X_ zV@?K)bufs~;CO*8ahV+5SODgO-fNblT>Pc6aI0!!TkzHxm0|7#p#w8Y>zWFE_-5!9 zY#YrDU|^|WX{-0{5I;;0atZ2sb2r<>KEPl06@Y=s<>>nttS$_yF1!+km+bGsP*ACu z^@LMWfXkMLZZ3>Z)e8wq(@J|z2o8O-aH`nyaCn6ZzX?Ku27st!fY)>$f{)^1+%yRn zEXida#9cp8LCNzM18RU0C;<VLuUCqj6o@2{9^VgR zPy`!}5oS1yxTKt#C{h3L;>SSk+K}pDp7i75E3Ha+lH7l=c^;?>qOy7`a)vUtfu2}WyoL9GkqE|WNQTE#O@|it6j!4V&?49xFcEEpbK+wn<+9>O3gFA|@a~uv z<4?9p2hW{ey0Qp)k_f5JovJqBWGVRZW5uz8f)_`|K{(I^xM@Of4rxBn;(1RLSJn%u zD!*Yg=E)-9*968vw)i10VtCHidCxsr#K;@77`|&UOqMeE4vgo5OP?wlVPEBB3>jIL zYxn^HlRxwaK;dhVg6+i_fKJ@d)UGeRQg$ZNj4z8L!b?;I9gF)Q3lB#c7{%;xHeQ`) ze=X3nUlr$Ki;1{ccl~s4@xIsxV2y8kl0*Fu7o4M@wDuj(#DpD!y&hi7vK>FE(Sn~ z4t#U0lhASCfYgpsH?&AHD84mlZz|6%D7b3^gkr{oxd~4JeE?GR!5J)_*l4R0Zjqp{ov<^lnr4-gjhNN z3)=>6B zfiuH*HV_q-!$t~o=MC%1d|jVhkDF(~o#jCV>2%;kp^LF06oMTI845{ZeR+JUPDD_g zR!VgQ$G%@aU5YsYXAdOCtKha942K6sMf&K6W#)uB%kkyR^@qsAN5&P3n!=9`_!k>E z7{HDm-1JoGOpzK6lRHsdwX4WLcop;O!{p6y`6voNA%hth8UPRpvZy@U#>+jh`L!X{ zJ1tb_qyqhVu{?qvoSkBs5*j%kb;IAl-3;L>eg*MBx4crVppg9kBJ~vaF+xlss)qF(ms(2 zSIRT%E<>bDn;88j1rNjn0LAd~&{voSauV?SiSTOh(EH_173^FaL)iBOdh&i8BJO~A z_T@oMLGR+S2xFj(@R&qX2jrf5tM%$qmgv8~kcYEmD$9^vd^H9Jgqs0HcyVpM@TJVP ziDIo;k8AVhzsvX0%$uaVj194Vhh&6o0L9rf<_4*?TAx7AH@}h5HHk3KVfd6ff_amb zvn84783vj~F8}FjH2Jco(E;icAo|;rlo=rVnh4X6YjksuGMtu{ikM?UXJZJ?9v>Ih za6BQ991t*xOGM)eg(vxm&#_`vJXHkiun2Gl<^9hfJbGnhU-=s1fGvaknr;J3hY>7HJq*BCk`(t;k5@!2SOQ+ zj3pz`&NXPtiwY}k_8MR@0IY=(jDxdu=mRXFrhQBS008(H=136ercXERRK}cMaiNBI zqKI-b*jWIjw!tkJh7LSOC*Z0KgJV=e^@n_J1xIT_)ga#txF|#0`R)Am`Ww>wG_X_I zqt^KqAT4`YX@${+O{n<=ATKo6`##FPtmtZ;?+T{9tXwWM*ZU*}vu@`|$j F{{WZl;(P!A diff --git a/dev/examples/examples/bonito/bin/3dff6576f1026c9d50cb7a2eed1fa965d9adea99-4453480068583967657.bin b/dev/examples/examples/bonito/bin/ce800911340edfcc75196ae7c5f0cc0abedd036b-239007990221751549.bin similarity index 99% rename from dev/examples/examples/bonito/bin/3dff6576f1026c9d50cb7a2eed1fa965d9adea99-4453480068583967657.bin rename to dev/examples/examples/bonito/bin/ce800911340edfcc75196ae7c5f0cc0abedd036b-239007990221751549.bin index eca4feabbb6cb0dd6fdb5f70b4a26384147a47f1..59c70a03bbcff6ab043c0004b9680e015fb7ed0c 100644 GIT binary patch delta 2963 zcma)8U5s5-74E$ktf+L_duPhb6iRDnR9dF&{~xi;Ip+~$glI%Gwexq!mi{m>6|G@h z{mJeFePSxJi#(wSn)ra7(L>S3NoZ1w0pS6ynxMWAG@9T88WI%0bMDwqxaYx#dvkKu z+UxA~t#5s6E$5HaU%`*&FW!3J>E8`awkO$CoB5Gxqo+nxyWJic*KBg6JvHUV)kJ$z zw9V1)&3~_`S(*wGIeond%k=)MuBR9 z>VX=8t_gH)pbrFE8|Z_9t_$>`KpzhDkwDi6S{JA%(E30d0yP7*0`&&!3$!uN4T1Ut z4Fnntv?1z25*-pJBwl;lQvtzX;wBwy5cP+SUTEM5xb zoL@QQ7p>q*Tgx2`mThuYa|$^c_APG$xw?kXP9JCYT=MESH6V);UYj^z0!fLeKh11yOQL+-YtFR=dR}@Vi{uCMKfRcKzY)LvNOA~etnkY3|4p(q8rR*E z{Wd=Nt7K!G=}&uFR)K960kr@T8Dm`MZ%wB!*W%MJC4@*74nBAHqi ze?6V9YiZ3CLZB@?;&=z3S5g=oOsRLV!rsYX^QIITXv4fwT6l2DNy&^ZI-)GgQZ0s? z(uqo+5`&@5Xp+<_E@)gBaQ!W7Epq=#~@&g8wB6ou{fSxMy& zXVdxomp@Kt8=V>h6e*3e3L4=w^w|Bc#yj$&#ufHXjsKFPLWT5gkV0E;l?Br^Dg}~k z{aGsN@w3mDDRr5djIX``p8x&%bX(I2W^o7wK?`FnC>z;@bgt-KRQGe|)A3ffi|GXT z$N3IPdYVvm5R?^Hzy*qdvYKjNTrN1GE1b%E{A+n7la^ypkwU|iL*H-(&aIBK`-L*0 zDlY^Y?VtheG#Y3s)KUp;Tn;gtFR0OFfr{bWj)jZRq0SKD98881T4$`ppVtLFprO;3 z(me9tk3O&wuWDMC`r36qfCZ_k8npf!2x~8C5x5HJMUyi4HvTe}9qC*&Hgj6h$ zbESB_RErB6vSU@|E7$0YUub4O7E}7;cTnvQsdj z_l<@SA)u0;L1EHK@-Nf8y?AXv48QVQ1xEvkAUvh|9GQL}L zql_j;UzmDu?AS|_qmLYTV08CW7nWqFANM?dWre_Zn47X>F`Vfpw>>&vtc zz9aZr@SVXg4SrehR|LO2_|F9Y+2F4X{;J?V7yQ-1uL!;?_?5wT2VW093BD10Pw=aP z?+t!+@N0r!8+>2z*93oU@auxVF8KAqZwP*4@ShL9Klp**2ZO&p_#1-X6#R|BZw`J- z@I%3G4gRL!N$@my7CaAL1TTYE!Rz2n@HY6HgTE#CF9g3W`0c?Dzv9y!+g87O|I^2t zOb}*_u!K-43^QEleE;_J&W_%EVti`v5T%wFB9tV`Fe<5z>)%QH@(+g7TRM}orKrx+ zThlLXjn5oOS0q$Rg>5+{!f;8k1Bn~nNp)O#(A}1A`fA#!HJVjPJLcV{`1QkSPmD*> zu7onG4WWWDVk}`?aaHUi^S>WS|I-ow{Izr>X_k}Xm*?G$@zICUfrL}b2v!Oz6a^U3 zMk4d~A4>17Rfby_jvYw{>WUYIE6Oe7hHJ&lrPJ@Nb(f-X$KiBy{MKV>e}ax0YmDK{ zFwKBLi}q3+3*i6o>1bt6>+C0w;%u}}Q)Q)=Oag^uEI%{vNG*PNEFG(e#@5o4-@xHo za>lgchA6(;vp~B-s^pD~!YxX{!yl+)z&9`32XP zKky6p{m!`hM0&8%Ea%Al%%FQRjn}^g5?f+0lFOnNZKMW`$}*Mjd@H@ZmNd(uHg)g6 z4eAIJNQU@`(M$=Aj~wLB)avC(v21m_fDeA1&L+xQa7~1U04)ZX8dmgw?IQT8-=&jP zIR*B);dr{&W%*xMxvOgNiMz6{1Vf-CCO~pUxTOYyEGl@rBVJtVe9s)TRZ3{{@qC|y z22q1%ftExi-%j_>%=gtm{e9XNtgnPVnQFwho{pyxZ}B&G|EY`eg4t& z826%r#K7ud)`p-6Az-tL79dcsiNY?*S&LYv?#c!dXbHEDg<+x@ zrkJ*tCV}PQyR&26rEt7s-mQuK_hftfUTcln5QT9wYm3Ris1r_bf3)Z0+&$T;8X#09 z?BAIUC%7Pqg~brmp`d7dX|0S{*`NP=XLh(&m6K18W`{b~yxMv?2!Sh8Y9%fBfVkmY z5zSna58Ri1v9r;vi09wDKl|5KJPHgvo8hjO7))*94J6KdH#<@Ii&B;=ya;fiQ)q;# z$W9dz1-5WcNX{*c9*!?e{0$-~6Sk=Yc!hPho;gJ6cqnG;lCv5;DQZ>d5Z*un0kb{5 z_Rc2gL;Hm-j1Gp+7$*zM_P(f)+%!gU!MRZQp}3x?RKMN(9G!#efr`P$u$jeCLh$w? zZT`^tmpBu%i;b$Jxa--hFR6Hn6@hEI%NXB#CZkCOjC{+p*&`kGGWM8%FFRiK|2s}+ G)Bgt<_FfVI diff --git a/dev/examples/examples/bonito/bin/02cdd7b832e4d41edaea4f7c802d5fde9fa3ce3d-313498830422961508.bin b/dev/examples/examples/bonito/bin/fd136fd258ce126b6d77ecd02bb899803d5c0e30-12269883709464688670.bin similarity index 86% rename from dev/examples/examples/bonito/bin/02cdd7b832e4d41edaea4f7c802d5fde9fa3ce3d-313498830422961508.bin rename to dev/examples/examples/bonito/bin/fd136fd258ce126b6d77ecd02bb899803d5c0e30-12269883709464688670.bin index 6202bdd4a12c3c59ae89d46b6add477c8e7a649b..4eab8f76e9cf1fbd56dab9a720477a0f9d870ffd 100644 GIT binary patch delta 2720 zcma)8OKeqD6y=;6O)%l}?rRH_0LBUl@W}apOpquu4WI+#hd?Q`h1zG!vz1bUp(z34 zKz=uJL|r>)d_T-h1t} z&!5TU-;&8Y4vs8-E&i!lX``65uF%oeE(%KV^#$*REyxbxY^xRA3P1Q^@0;s~KHc0p zQ0!@KJ8)$ZJ@P@C1PTvA;ari-jL(lo@!G!nc)`^vk+%5QFs-Gu$~%pF%SX3y8eiVh zk}qeV;X;u#RzkG(-w)xLQ_M5m%?uJ{M!WbxNtb`9)K+v?|?D|U9b z_qK0*y}i4yw}5J}^g_m@QVc{4m>`dnwLq1`=C_)20=2HxfGa0JLx^~P$>9EbB)++$ zHLAr~HNcj^`~y6=LKvlhO^{C6G2IC4YfokYtd?~?K|s3*C{e3vYHFe8G3Al zghsqj<1v2Tg|cSAX^=(n&fUi&%cN9jhx1x$!@SV~<(yS+*^_@L3rv15k>O>5O{F*D z=~tU0CA~q>jWo_!4-GJ%{5^w=lDJ`XFbkdFZzE9=t98;Th{++04M=@^GWTmTOyVc6 zt>8t+p=BVkMnkgM_0wnufQ>=jZ423G~Ag-*5zv9RLe8sp56Hn9wWyiLBYkQj3d)#JAu{PMA(M;~X(p$ahg&l+0+75` zQfdbmOf3*R#*=t2Av^Nr47{4fi>F9)PC-IaD-00h${4t4vj1$_i!aZRbx|z}t=_k$ zTzKUyn&~8H8#I&|BjL18yX3%G2q9OCqqPmrQ{2FjF^yKkLcD?JjgRu@B#d1lsl8_%Dw4%v+qdM{tkZq% zTl)%_W+jXf0^_6w_ZUS!uIZ2K!rf8Q4H;#esjB++eL;W=ya+-N29phEgfBtWt7yKQ z0i`uv9KO%de|caGaJ97(zGN&mmPpr3>Wkp9s*#q!BjZe!$BoePGHpo7OvTEu3>9Wr zeM$n??`BSmF|K;*W#5Xhx*mD2ND1g_^W|ODK9)@6`Ko}*V5+X`fA!svyOz- zYpHEIQ(1Wh>|Dud;65{n zHqhtlvf!lDLGLWd4|9b{aF_>eg+lt+3}O4bbZ^cnr>uqZc`OczXO#%E57Bob^a`n> zOz=?f#6DEvi$ip8lia-o#$$39MumvAcBt-n7fl&Kk3P>r|qND^usSy*2 zM47+=Nr;U?jS�cohSJ5UV5n9fY5v-jC&-M!Xc z=lmLv|05p1ZTHZlFBX5C#R|-}w=q>GTiObBs?D3amV%abYE^5&7FKy_+I{c)-LJIv z4ZPIS)3Kpt)jQwbLl1nkZud9yi~qO4Pxt)uf%VD{jVtUcb!$@xC#|x0(`Xf`IaeI* zkwT$&&X|!ZXSSlsU{6{lBPXr$+~(@gGnGt=guzx`Db9@Y#-Qnb6w3;l;K+!l6_#$D z9Wh}I6UrJVEgr_=-%$uFX=XHCx~#W4Y|CSFEt2I@0fOadg)%x>za`E;Ku(h4>AvI{ zvXas=jjz>&-aPurz4S(KCxAjJ=9EiejFvh3dQv7U31udSrWvHF*tFwV1a%3iSt3Jp zgK3B`K~ozqd59dYC|-NBGb^Rkvh!F&WUOU)8738!CY5F)-q1)dk>c-{w&%*Dp&?J^ z7LN?ij;s_`I0Xi|#&^J!F?MCNSbe26E1HPu7V48%jz!LUD9GY%EZ7o2eaH1jZwweJYt58H16t8smxe zjCr8BP2;B|3{NM`nK`JiB(jcM>!lOkKtT=yO!24W)zDmdIBDJe32Wyg=@od=TtfXW zk&xxLpS@~KRv2vUY}mV+%*r9xTfs3Po_isjb#Z4S9VTJST0C%hG#%o(;j6WzIy4WF z36bZFE5U^YbjLuU=Oe_0IUS_As+1AGO$7t35QFt`?{RYPm||ns#7Hsd-wDp0QxGwe+T9|rJ0`u*envt9Wp3b~k(oKog=El; z!k-2cT!-EvvNitg42Cf5I|cU2Q&^}x1OCky;2(Y~n1PP1)!G_7cse*+k_zDe;OV47 zMh+@0iD33ddM2^f7!4DUX%EuHH8eAv?!53EsY=H`r>&I<{jgLA?LkFxf1Yp}2F{b_ z#6Zg8F3cDL_@^#F2e3X41QOE@EhR@!!Xj25`I@v=mPNyZL!==rzX00-=P)my^bGxH zR$yZ+eLJzujMyFIFTF@UEH+&oms2n-&(JZb2QKL4e~xpgB3vCNozPKvQmW!`Z6d8n z`PUk!1@|hMa|bVgByaKmb0zIdI#8Z)2b#kOABd#K4t97~v9FH8dF`J^lO>F(GFj_b?mhDz_yM!awc!NiU5 z;k`VW77jMiS@Fj!Xg7_@GK>qNiQ3xF(}TLs87M&uZXia569Jww;s>a)msXWB;_)*r zbQ6Khq=tSaQ_eC-Nm>zrdP;n~l|EaM6^<9R)4%8CPH>!j!hty)*f>IQ*e}~dcjc_n zLSVjO?h2!2l?;FGp>IVR=IRWt4;T`N$3UDR?4`R>A9Tz!ZZyn5C3l?Q0?nI2jth%R zd;)hPTzqhMVQTbZ#7YhY6;2$q4M#OhNSp81^z(2y&|j*uZcX94eRNu7*+nT_-beGf i@}0uXeUwKTyTniIr#&QBerpSt-=%M4{l4Y^?fVyga@v^y diff --git a/dev/examples/examples/irfs.ipynb b/dev/examples/examples/irfs.ipynb index 810e844..6403a44 100644 --- a/dev/examples/examples/irfs.ipynb +++ b/dev/examples/examples/irfs.ipynb @@ -34,25 +34,25 @@ "text/plain": "Figure()", "text/html": [ "

\n", - "
\n", + "
\n", "
\n", " \n", " \n", "
\n", "
\n", "
\n", - " \n", + " \n", " \n", "
\n", "
\n", - "
\n", + "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", "
\n", diff --git a/dev/examples/examples/irfs/index.html b/dev/examples/examples/irfs/index.html index 15cf896..61dea26 100644 --- a/dev/examples/examples/irfs/index.html +++ b/dev/examples/examples/irfs/index.html @@ -8,18 +8,18 @@ @automakie()

Typically, the logistic c.d.f. is used as the transfer function in IRT. However, it in an IRT context, a scaled version intended to be close to a normal c.d.f. is often used. The main advantage is that this is usually faster to compute. ComputerAdaptiveTesting provides normalscaledlogistic, which is also used by default, for this purpose:

xs = -8:0.05:8
 lines(xs, cdf.(Normal(), xs))
 lines!(xs, cdf.(normal_scaled_logistic, xs))
-current_figure()
+current_figure()
-

This page was generated using DemoCards.jl and Literate.jl.

+

This page was generated using DemoCards.jl and Literate.jl.

diff --git a/dev/examples/examples/perf_plots.ipynb b/dev/examples/examples/perf_plots.ipynb index 65de97f..75bd9b6 100644 --- a/dev/examples/examples/perf_plots.ipynb +++ b/dev/examples/examples/perf_plots.ipynb @@ -7,14 +7,14 @@ "data": { "text/plain": "Figure()", "text/html": [ - "
\n", + "
\n", "
\n", " \n", "
\n", "
\n", "
\n", " \n", "
\n", diff --git a/dev/examples/examples/perf_plots/index.html b/dev/examples/examples/perf_plots/index.html index 4a9dba5..d39e734 100644 --- a/dev/examples/examples/perf_plots/index.html +++ b/dev/examples/examples/perf_plots/index.html @@ -105,18 +105,18 @@ fig end -plot(df)
+plot(df)
-

This page was generated using DemoCards.jl and Literate.jl.

+

This page was generated using DemoCards.jl and Literate.jl.

diff --git a/dev/examples/examples/vocab_iq/index.html b/dev/examples/examples/vocab_iq/index.html index b33758a..9e5c279 100644 --- a/dev/examples/examples/vocab_iq/index.html +++ b/dev/examples/examples/vocab_iq/index.html @@ -55,4 +55,4 @@ run_cat(loop_config, item_bank) end -run_vocab_iq_cat()

This page was generated using DemoCards.jl and Literate.jl.

+run_vocab_iq_cat()

This page was generated using DemoCards.jl and Literate.jl.

diff --git a/dev/examples/index.html b/dev/examples/index.html index 5d743be..08ad61c 100644 --- a/dev/examples/index.html +++ b/dev/examples/index.html @@ -31,4 +31,4 @@

This example shows how to run a CAT end-to-end on real data.

- + diff --git a/dev/index.html b/dev/index.html index c7d2dbd..6b47f80 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -Getting started · ComputerAdaptiveTesting.jl
GitHub

Getting started

This is the documentation for ComputerAdaptiveTesting. For information about the purpose of the package and its installation, please see the README.md.

Contents

  • Creating a CAT: A guide to creating a CAT using the configuration structs in ComputerAdaptiveTesting.jl
  • Using your CAT: A collection of mini topical guides about: simulating CATs, integrating into a user-facing applications, evaluating CATs, visualising CATs, and constructing decision trees from CATs
  • Examples: A collection of example code which you can run and modify to get a feel for how to use the package
  • API reference: The full API reference for the package
+Getting started · ComputerAdaptiveTesting.jl
GitHub

Getting started

This is the documentation for ComputerAdaptiveTesting. For information about the purpose of the package and its installation, please see the README.md.

Contents

  • Creating a CAT: A guide to creating a CAT using the configuration structs in ComputerAdaptiveTesting.jl
  • Using your CAT: A collection of mini topical guides about: simulating CATs, integrating into a user-facing applications, evaluating CATs, visualising CATs, and constructing decision trees from CATs
  • Examples: A collection of example code which you can run and modify to get a feel for how to use the package
  • API reference: The full API reference for the package
diff --git a/dev/search_index.js b/dev/search_index.js index 99e0f9b..ca0bc7e 100644 --- a/dev/search_index.js +++ b/dev/search_index.js @@ -1,3 +1,3 @@ var documenterSearchIndex = {"docs": -[{"location":"contributing/#Contributing","page":"Contributing","title":"Contributing","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Contributions are welcome. Please discuss any larger changes in the issues before making a pull request to avoid wasted work.","category":"page"},{"location":"contributing/#Running-tests","page":"Contributing","title":"Running tests","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"You can run tests locally like so:","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"julia --project=test test/runtests.jl","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Unfortunately, Pkg.test() does not work properly at the moment. See this issue.","category":"page"},{"location":"using_your_cat/#Using-your-CAT","page":"Using your CAT","title":"Using your CAT","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"Now you've created your cat according to Creating a CAT, there are a number of ways you can use it. This section covers a few. See also the Examples.","category":"page"},{"location":"using_your_cat/#Simulating-CATs","page":"Using your CAT","title":"Simulating CATs","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"TODO","category":"page"},{"location":"using_your_cat/#Integrating-into-a-user-facing-applications","page":"Using your CAT","title":"Integrating into a user-facing applications","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"TODO","category":"page"},{"location":"using_your_cat/#Evaluating-CATs","page":"Using your CAT","title":"Evaluating CATs","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"TODO","category":"page"},{"location":"using_your_cat/#Visualising-CATs","page":"Using your CAT","title":"Visualising CATs","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"TODO","category":"page"},{"location":"using_your_cat/#Constructing-decision-trees-from-CATs","page":"Using your CAT","title":"Constructing decision trees from CATs","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"TODO","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/examples/perf_plots.jl\"","category":"page"},{"location":"examples/examples/perf_plots/#Perf-plots","page":"Perf plots","title":"Perf plots","text":"","category":"section"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"(Image: Source code) (Image: notebook)","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"Plots for performance of different integrators for ability estimators","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"using Makie\nimport Pkg\nusing CSV\nusing DataFrames\nusing CategoricalArrays: levels\nusing IterTools: chain\nusing ComputerAdaptiveTesting\nusing AdaptiveTestPlots\nusing AdaptiveTestPlots: LabelledToggleGrid, MenuGrid\n\nDOCS_DATA = pkgdir(ComputerAdaptiveTesting) * \"/docs/data/\"\n\n@automakie()\n\ndf = CSV.read((DOCS_DATA * \"integrator_benchmark_results.csv\"),\n DataFrame;\n types = Dict(\"value\" => Float64,\n \"err\" => Float64,\n \"bytes\" => Int64,\n \"response\" => Int32,\n \"item_bank\" => Int32,\n \"dim\" => Int32,\n \"trial\" => Int32,\n \"time\" => Float64,\n \"gctime\" => Float64,\n \"rtol\" => Float64))\n\nfunction plot(df)\n fig = Figure()\n ax = Axis(fig[1, 1])\n\n groupables = [\"item_bank\", \"response\", \"trial\", \"dim\", \"integrator\"]\n regressables = [\"integrator\", \"dim\", \"rtol\", \"value\", \"err\", \"bytes\", \"gctime\"]\n\n Menu(fig, options = [\"none\", groupables...], default = \"none\")\n toggles = []\n sliders = []\n slider_idxs = []\n menus = []\n menu_idxs = []\n for (idx, groupable) in enumerate(groupables)\n push!(toggles, (label = groupable,))\n opts = sort(levels(df[!, groupable]))\n if eltype(opts) <: AbstractString\n push!(slider_idxs, idx)\n push!(menus, (label = groupable,\n options = opts))\n else\n push!(menu_idxs, idx)\n push!(sliders, (label = groupable,\n range = opts))\n end\n end\n tgrid = LabelledToggleGrid(fig,\n toggles...,\n width = 350,\n tellheight = false)\n lsgrid = SliderGrid(fig,\n sliders...,\n width = 350,\n tellheight = false)\n mgrid = MenuGrid(fig,\n menus...,\n width = 350,\n tellheight = false)\n\n working_df = Observable(nothing)\n function update_working_df(_)\n df_prime = df\n slider_menu_vals = Vector{Any}(undef, length(sliders) + length(menus))\n slider_menu_vals[slider_idxs] = [slider.value for slider in lsgrid.sliders]\n slider_menu_vals[menu_idxs] = [menu.selection for menu in mgrid.menus]\n for (groupable, toggle, value) in zip(groupables, tgrid.toggles, slider_menu_vals)\n if !toggle.value\n continue\n end\n df_prime = filter(working_df, groupable => value)\n end\n working_df[] = df_prime\n end\n\n for observable in chain((t.active for t in tgrid.toggles),\n (s.value for s in lsgrid.sliders),\n (m.selection for m in mgrid.menus))\n on(update_working_df, observable)\n end\n\n x_var_menu = Menu(fig, options = regressables)\n y_var_menu = Menu(fig, options = regressables)\n coded_menu = Menu(fig, options = regressables)\n\n xs = @lift($working_df !== nothing && $(x_var_menu.selection) !== nothing ?\n $working_df[$(x_var_menu.selection)] : nothing)\n ys = @lift($working_df !== nothing && $(y_var_menu.selection) !== nothing ?\n $working_df[$(y_var_menu.selection)] : nothing)\n\n scatter = @lift($xs !== nothing && $ys !== nothing &&\n $(coded_menu.selection) !== nothing ?\n lines!(ax,\n $xs,\n $ys;\n color = $(coded_menu.selection)) : nothing)\n\n fig\nend\n\nplot(df)","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"
\n
\n \n
\n
\n
\n \n
\n \n
\n
\n
","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"This page was generated using DemoCards.jl and Literate.jl.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/index.md\"","category":"page"},{"location":"examples/#demo-page","page":"Examples","title":"Examples and demos","text":"","category":"section"},{"location":"examples/","page":"Examples","title":"Examples","text":"You can browse through the examples below. Each example has code and outputs, as well as links to download a script to run locally or to view as a Jupyter notebook on NBViewer. Some of the examples have interactive plots, and these will not currently work unless the scripts are run locally. TODO: Add binder links to examples.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"If you would like to run the examples as Jupyter notebooks locally to see interactive plots inline with the code, you can clone the repository and run docs/build_nbs.jl like so:","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"$ git clone https://github.com/JuliaPsychometricsBazaar/ComputerAdaptiveTesting.jl.git\n$ cd ComputerAdaptiveTesting\n$ julia --project=. docs/build_nbs.jl","category":"page"},{"location":"examples/#Examples","page":"Examples","title":"Examples","text":"","category":"section"},{"location":"examples/","page":"Examples","title":"Examples","text":"
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n\n \n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"How abilities converge on simulated 3PL data","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"This example shows how to run a CAT based on a synthetic correct/incorrect 3PL IRT model.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n\n \n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"How abilities converge on simulated MIRT data","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"This example shows how to run a CAT based on a synthetic correct/incorrect MIRT model.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n\n \n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Item Response Functions","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Typically, the logistic c.d.f. is used as the transfer function in IRT. However, it in an IRT context, a scaled version intended to be close to a normal c.d.f. is often used. The main advantage is that this is usually faster to compute. ComputerAdaptiveTesting provides normalscaledlogistic, which is also used by default, for this purpose:","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n\n \n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Perf plots","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Perf plots","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n\n \n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Running a CAT based based on real response data","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"This example shows how to run a CAT end-to-end on real data.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"CurrentModule = ComputerAdaptiveTesting","category":"page"},{"location":"creating_a_cat/#Creating-a-CAT","page":"Creating a CAT","title":"Creating a CAT","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"This guide gives a brief overview of how to create a CAT using the configuration structs in ComputerAdaptiveTesting.jl.","category":"page"},{"location":"creating_a_cat/#API-design","page":"Creating a CAT","title":"API design","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The configuration of a CAT is built up as a tree of configuration structs. These structs are all subtypes of CatConfigBase.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.CatConfig.CatConfigBase","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.ConfigBase.CatConfigBase-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.ConfigBase.CatConfigBase","text":"abstract type CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The constructors for the configuration structs in this package tend to have smart defaults. In general most constructors have two forms. The first is an explicit keyword constructor form, where all arguments are given:","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ConfigurationObject(\n field1=value1,\n field1=value2,\n)","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The second is an implicit form, where arguments are given in any order. If possible, they will be used together will appropriately guessed defaults to construct the configuration:","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ConfigurationObject(value2, value1)","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The implicit form is particularly useful for quick prototyping. Implicit form constructors are also available for some abstract types. In this case, they will return a concrete type that is a reasonable default for the abstract type.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"After using the implicit form, you can print the object to see what values have been filled in. This may be useful in case you want to modify some of the defaults or switch to the explicit form.","category":"page"},{"location":"creating_a_cat/#Item-banks","page":"Creating a CAT","title":"Item banks","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"Item banks are the source of items for the test. The basic definitions are provided by FittedItemBanks.jl and can be fit to data using RIrtWrappers.jl. See the documentation pages of those packages for more information.","category":"page"},{"location":"creating_a_cat/#CatRules","page":"Creating a CAT","title":"CatRules","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"This is the main type for configuring a CAT. It contains the item bank, the next item selection rule, and the stopping rule. CatRules has explicit and implicit constructors.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.CatConfig.CatRules","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.CatConfig.CatRules-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.CatConfig.CatRules","text":"struct CatRules{NextItemRuleT<:ComputerAdaptiveTesting.NextItemRules.NextItemRule, TerminationConditionT<:ComputerAdaptiveTesting.TerminationConditions.TerminationCondition, AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.AbilityEstimator, AbilityTrackerT<:ComputerAdaptiveTesting.Aggregators.AbilityTracker} <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\nnext_item::ComputerAdaptiveTesting.NextItemRules.NextItemRule: The rule to choose the next item in the CAT given the current state.\n\ntermination_condition::ComputerAdaptiveTesting.TerminationConditions.TerminationCondition: The rule to choose when to terminate the CAT.\n\nability_estimator::ComputerAdaptiveTesting.Aggregators.AbilityEstimator: The ability estimator, which estimates the testee's current ability.\n\nability_tracker::ComputerAdaptiveTesting.Aggregators.AbilityTracker: The ability tracker, which tracks the testee's current ability level. Default: NullAbilityTracker()\n\nConfiguration of the rules for a CAT. This all includes all the basic rules for the CAT's operation, but not the item bank, nor any of the interactivity hooks needed to actually run the CAT.\n\nThis may be more a more convenient layer to integrate than CatLoopConfig if you want to write your own CAT loop rather than using hooks.\n\nCatRules(; next_item=..., termination_condition=..., ability_estimator=..., ability_tracker=...)\n\nExplicit constructor for CatRules.\n\nCatRules(bits...)\n\nImplicit constructor for CatRules.\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/#Next-item-selection-with-NextItemRule","page":"Creating a CAT","title":"Next item selection with NextItemRule","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The next item selection rule is the most important part of the CAT. Each rule extends the NextItemRule abstract type.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.NextItemRule","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.NextItemRule-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.NextItemRule","text":"abstract type NextItemRule <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\nAbstract base type for all item selection rules. All descendants of this type are expected to implement the interface (rule::NextItemRule)(responses::TrackedResponses, items::AbstractItemBank)::Int\n\nNextItemRule(bits...; ability_estimator=nothing, parallel=true)\n\nImplicit constructor for NextItemRule. Uses any given NextItemRule or delegates to ItemStrategyNextItemRule.\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"A sort of null hypothesis next item selection rule is RandomNextItemRule, which ","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule","text":"struct RandomNextItemRule{RandomT<:Random.AbstractRNG} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule\n\nrng::Random.AbstractRNG: Default: Xoshiro()\n\nThis is the most basic rule for choosing the next item in a CAT. It simply picks a random item from the set of items that have not yet been administered.\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"Other rules are created by combining a ItemCriterion – which somehow rates items according to how good they are – with a NextItemStrategy using an ItemStrategyNextItemRule, which acts as an adapter. The default NextItemStrategy (and currently only) is ExhaustiveSearch1Ply. When using the implicit constructors, ItemCriterion can therefore be used directly without wrapping in any place an NextItemRule is expected.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule","text":"struct ItemStrategyNextItemRule{NextItemStrategyT<:ComputerAdaptiveTesting.NextItemRules.NextItemStrategy, ItemCriterionT<:ComputerAdaptiveTesting.NextItemRules.ItemCriterion} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule\n\nstrategy::ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\ncriterion::ComputerAdaptiveTesting.NextItemRules.ItemCriterion\n\nItemStrategyNextItemRule which together with a NextItemStrategy acts as an adapter by which an ItemCriterion can serve as a NextItemRule.\n\nItemStrategyNextItemRule(bits...; ability_estimator=nothing, parallel=true)\n\nImplicit constructor for ItemStrategyNextItemRule. Will default to ExhaustiveSearch1Ply when no NextItemStrategy is given.\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.ItemCriterion","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.ItemCriterion-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.ItemCriterion","text":"abstract type ItemCriterion <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.NextItemStrategy","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.NextItemStrategy-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.NextItemStrategy","text":"abstract type NextItemStrategy <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply","text":"struct ExhaustiveSearch1Ply <: ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\n\nparallel::Bool\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/#Evaluating-item-and-state-merit-with-ItemCriterion-and-StateCriterion","page":"Creating a CAT","title":"Evaluating item and state merit with ItemCriterion and StateCriterion","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The ItemCriterion abstract type is used to rate items according to how good they are as a candidate for the next item. A typical example is InformationItemCriterion, which using the current ability estimate theta and the item response function irf to calculate each item's information fracirf_θ^2irf_θ * (1 - irf_θ).","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"Within this, there are two types of ExpectationBasedItemCriterion which act as adapters: PointExpectationBasedItemCriterion and DistributionExpectationBasedItemCriterion. These take a StateCriterion, which evaluates how good a particular state is in terms getting a good estimate of the test takers ability. They look one ply ahead to get the expected value of the StateCriterion after selecting the given item. The AbilityVarianceStateCriterion looks at the variance of the ability theta estimate at that state.","category":"page"},{"location":"creating_a_cat/#Stopping-rules-with-TerminationCondition","page":"Creating a CAT","title":"Stopping rules with TerminationCondition","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"Currently the only TerminationCondition is FixedItemsTerminationCondition, which ends the test after a fixed number of items.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.TerminationConditions.TerminationCondition","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.TerminationConditions.TerminationCondition-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.TerminationConditions.TerminationCondition","text":"abstract type TerminationCondition <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition","text":"struct FixedItemsTerminationCondition <: ComputerAdaptiveTesting.TerminationConditions.TerminationCondition\n\nnum_items::Int64\n\n\n\n\n\n","category":"type"},{"location":"api/","page":"API reference","title":"API reference","text":"CurrentModule = ComputerAdaptiveTesting","category":"page"},{"location":"api/#API-reference","page":"API reference","title":"API reference","text":"","category":"section"},{"location":"api/","page":"API reference","title":"API reference","text":"","category":"page"},{"location":"api/","page":"API reference","title":"API reference","text":"Modules = [ComputerAdaptiveTesting, ComputerAdaptiveTesting.Aggregators, ComputerAdaptiveTesting.Responses, ComputerAdaptiveTesting.Sim, ComputerAdaptiveTesting.TerminationConditions, ComputerAdaptiveTesting.NextItemRules, ComputerAdaptiveTesting.CatConfig]","category":"page"},{"location":"api/#ComputerAdaptiveTesting.Aggregators","page":"API reference","title":"ComputerAdaptiveTesting.Aggregators","text":"This module takes care of integrating and optimizing over the ability/difficulty space. It includes TrackedResponses, which can store cumulative results during a test.\n\n\n\n\n\n","category":"module"},{"location":"api/#ComputerAdaptiveTesting.Aggregators.RiemannEnumerationIntegrator","page":"API reference","title":"ComputerAdaptiveTesting.Aggregators.RiemannEnumerationIntegrator","text":"In case an item bank is enumerable (e.g. GriddedItemBank), then this method integrates over the ability likihood given a set of responses with a given coefficient function using a Riemann sum (aka the rectangle rule).\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.Aggregators.maybe_tracked_ability_estimate-Tuple{ComputerAdaptiveTesting.Aggregators.TrackedResponses, Any}","page":"API reference","title":"ComputerAdaptiveTesting.Aggregators.maybe_tracked_ability_estimate","text":"This method returns a tracked point estimate if it is has the given ability estimator, otherwise it computes it using the given ability estimator.\n\n\n\n\n\n","category":"method"},{"location":"api/#ComputerAdaptiveTesting.Sim.auto_responder-Tuple{Any}","page":"API reference","title":"ComputerAdaptiveTesting.Sim.auto_responder","text":"This function constructs a next item function which automatically responds according to responses.\n\n\n\n\n\n","category":"method"},{"location":"api/#ComputerAdaptiveTesting.Sim.prompt_response-Tuple{Any, Any}","page":"API reference","title":"ComputerAdaptiveTesting.Sim.prompt_response","text":"This response callback simply prompts \n\n\n\n\n\n","category":"method"},{"location":"api/#ComputerAdaptiveTesting.Sim.run_cat-Union{Tuple{RulesT}, Tuple{CatLoopConfig{RulesT}, FittedItemBanks.AbstractItemBank}} where RulesT<:CatRules","page":"API reference","title":"ComputerAdaptiveTesting.Sim.run_cat","text":"Run a given CatLoopConfig\n\n\n\n\n\n","category":"method"},{"location":"api/#ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition","page":"API reference","title":"ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition","text":"struct FixedItemsTerminationCondition <: ComputerAdaptiveTesting.TerminationConditions.TerminationCondition\n\nnum_items::Int64\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.TerminationConditions.TerminationCondition","page":"API reference","title":"ComputerAdaptiveTesting.TerminationConditions.TerminationCondition","text":"abstract type TerminationCondition <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules","text":"This module implements the next item selection rules, which form the main part of CAT.\n\nBibliography\n\n[1] Linden, W. J., & Pashley, P. J. (2009). Item selection and ability estimation in adaptive testing. In Elements of adaptive testing (pp. 3-30). Springer, New York, NY.\n\n\n\n\n\n","category":"module"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.catr_next_item_aliases","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.catr_next_item_aliases","text":"This mapping provides next item rules through the same names that they are available through in the catR R package. TODO compability with mirtcat\n\n\n\n\n\n","category":"constant"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.AbilityVarianceStateCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.AbilityVarianceStateCriterion","text":"struct AbilityVarianceStateCriterion{DistEst<:ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator, IntegratorT<:ComputerAdaptiveTesting.Aggregators.AbilityIntegrator} <: ComputerAdaptiveTesting.NextItemRules.StateCriterion\n\ndist_est::ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator\nintegrator::ComputerAdaptiveTesting.Aggregators.AbilityIntegrator\nskip_zero::Bool\n\nThis StateCriterion returns the variance of the ability estimate given a set of responses.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.DistributionExpectationBasedItemCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.DistributionExpectationBasedItemCriterion","text":"struct DistributionExpectationBasedItemCriterion{DistributionAbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator, AbilityIntegratorT<:ComputerAdaptiveTesting.Aggregators.AbilityIntegrator, StateCriterionT<:ComputerAdaptiveTesting.NextItemRules.StateCriterion} <: ComputerAdaptiveTesting.NextItemRules.ExpectationBasedItemCriterion\n\nability_estimator::ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator\nintegrator::ComputerAdaptiveTesting.Aggregators.AbilityIntegrator\nstate_criterion::ComputerAdaptiveTesting.NextItemRules.StateCriterion\n\nThis ItemCriterion wraps a StateCriterion and looks at its expected value for a particular item 1-ply ahead by integrating over an ability curve.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply","text":"struct ExhaustiveSearch1Ply <: ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\n\nparallel::Bool\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.ItemCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.ItemCriterion","text":"abstract type ItemCriterion <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule","text":"struct ItemStrategyNextItemRule{NextItemStrategyT<:ComputerAdaptiveTesting.NextItemRules.NextItemStrategy, ItemCriterionT<:ComputerAdaptiveTesting.NextItemRules.ItemCriterion} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule\n\nstrategy::ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\ncriterion::ComputerAdaptiveTesting.NextItemRules.ItemCriterion\n\nItemStrategyNextItemRule which together with a NextItemStrategy acts as an adapter by which an ItemCriterion can serve as a NextItemRule.\n\nItemStrategyNextItemRule(bits...; ability_estimator=nothing, parallel=true)\n\nImplicit constructor for ItemStrategyNextItemRule. Will default to ExhaustiveSearch1Ply when no NextItemStrategy is given.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.KernelAbstractionsExhaustiveSearch","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.KernelAbstractionsExhaustiveSearch","text":"struct KernelAbstractionsExhaustiveSearch{KernelT} <: ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\n\nkernel::Any\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.KernelAbstractionsExhaustiveSearchConfig","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.KernelAbstractionsExhaustiveSearchConfig","text":"struct KernelAbstractionsExhaustiveSearchConfig{ArgsT} <: ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\n\nkernel_args::Any\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.NextItemRule","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.NextItemRule","text":"abstract type NextItemRule <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\nAbstract base type for all item selection rules. All descendants of this type are expected to implement the interface (rule::NextItemRule)(responses::TrackedResponses, items::AbstractItemBank)::Int\n\nNextItemRule(bits...; ability_estimator=nothing, parallel=true)\n\nImplicit constructor for NextItemRule. Uses any given NextItemRule or delegates to ItemStrategyNextItemRule.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.NextItemStrategy","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.NextItemStrategy","text":"abstract type NextItemStrategy <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.PointExpectationBasedItemCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.PointExpectationBasedItemCriterion","text":"struct PointExpectationBasedItemCriterion{PointAbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator, StateCriterionT<:ComputerAdaptiveTesting.NextItemRules.StateCriterion} <: ComputerAdaptiveTesting.NextItemRules.ExpectationBasedItemCriterion\n\nability_estimator::ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator\nstate_criterion::ComputerAdaptiveTesting.NextItemRules.StateCriterion\n\nThis ItemCriterion wraps a StateCriterion and looks at its expected value for a particular item 1-ply ahead based on a point ability estimate.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule","text":"struct RandomNextItemRule{RandomT<:Random.AbstractRNG} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule\n\nrng::Random.AbstractRNG: Default: Xoshiro()\n\nThis is the most basic rule for choosing the next item in a CAT. It simply picks a random item from the set of items that have not yet been administered.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.StateCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.StateCriterion","text":"abstract type StateCriterion <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.UrryItemCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.UrryItemCriterion","text":"struct UrryItemCriterion{AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator} <: ComputerAdaptiveTesting.NextItemRules.ItemCriterion\n\nability_estimator::ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator\n\nThis item criterion just picks the item with the raw difficulty closest to the current ability estimate.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.CatConfig.CatLoopConfig","page":"API reference","title":"ComputerAdaptiveTesting.CatConfig.CatLoopConfig","text":"Configuration for a simulatable CAT.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.CatConfig.CatRules","page":"API reference","title":"ComputerAdaptiveTesting.CatConfig.CatRules","text":"struct CatRules{NextItemRuleT<:ComputerAdaptiveTesting.NextItemRules.NextItemRule, TerminationConditionT<:ComputerAdaptiveTesting.TerminationConditions.TerminationCondition, AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.AbilityEstimator, AbilityTrackerT<:ComputerAdaptiveTesting.Aggregators.AbilityTracker} <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\nnext_item::ComputerAdaptiveTesting.NextItemRules.NextItemRule: The rule to choose the next item in the CAT given the current state.\n\ntermination_condition::ComputerAdaptiveTesting.TerminationConditions.TerminationCondition: The rule to choose when to terminate the CAT.\n\nability_estimator::ComputerAdaptiveTesting.Aggregators.AbilityEstimator: The ability estimator, which estimates the testee's current ability.\n\nability_tracker::ComputerAdaptiveTesting.Aggregators.AbilityTracker: The ability tracker, which tracks the testee's current ability level. Default: NullAbilityTracker()\n\nConfiguration of the rules for a CAT. This all includes all the basic rules for the CAT's operation, but not the item bank, nor any of the interactivity hooks needed to actually run the CAT.\n\nThis may be more a more convenient layer to integrate than CatLoopConfig if you want to write your own CAT loop rather than using hooks.\n\nCatRules(; next_item=..., termination_condition=..., ability_estimator=..., ability_tracker=...)\n\nExplicit constructor for CatRules.\n\nCatRules(bits...)\n\nImplicit constructor for CatRules.\n\n\n\n\n\n","category":"type"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/examples/ability_convergence_3pl.jl\"","category":"page"},{"location":"examples/examples/ability_convergence_3pl/#ability_convergence_3pl","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"","category":"section"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"(Image: Source code) (Image: notebook)","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"How abilities converge on simulated 3PL data","category":"page"},{"location":"examples/examples/ability_convergence_3pl/#Running-a-CAT-based-on-a-synthetic-correct/incorrect-3PL-IRT-model","page":"How abilities converge on simulated 3PL data","title":"Running a CAT based on a synthetic correct/incorrect 3PL IRT model","text":"","category":"section"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"This example shows how to run a CAT based on a synthetic correct/incorrect 3PL IRT model.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"Import order is important. We put ComputerAdaptiveTesting last so we get the extra dependencies.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"using Makie\nimport Pkg\nimport Random\nusing Distributions: Normal, cdf\nusing AlgebraOfGraphics\nusing ComputerAdaptiveTesting\nusing ComputerAdaptiveTesting.Sim: auto_responder\nusing ComputerAdaptiveTesting.NextItemRules: AbilityVarianceStateCriterion\nusing ComputerAdaptiveTesting.TerminationConditions: FixedItemsTerminationCondition\nusing ComputerAdaptiveTesting.Aggregators: PriorAbilityEstimator,\n MeanAbilityEstimator, LikelihoodAbilityEstimator\nusing FittedItemBanks\nusing ComputerAdaptiveTesting.Responses: BooleanResponse\nimport PsychometricsBazaarBase.IntegralCoeffs\nusing PsychometricsBazaarBase.Integrators\nusing PsychometricsBazaarBase.ConstDistributions: normal_scaled_logistic\nusing AdaptiveTestPlots\n\n@automakie()","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"Now we are read to generate our synthetic data using the supplied DummyData module. We generate an item bank with 100 items and fake responses for 3 testees.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"using FittedItemBanks.DummyData: dummy_full, std_normal, SimpleItemBankSpec, StdModel3PL\n(item_bank, abilities, responses) = dummy_full(Random.default_rng(42),\n SimpleItemBankSpec(StdModel3PL(), OneDimContinuousDomain(), BooleanResponse());\n num_questions = 100,\n num_testees = 3)","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"Simulate a CAT for each testee and record it using CatRecorder. CatRecorder collects information which can be used to draw different types of plots.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"max_questions = 99\nintegrator = FixedGKIntegrator(-6, 6, 80)\ndist_ability_est = PriorAbilityEstimator(std_normal)\nability_estimator = MeanAbilityEstimator(dist_ability_est, integrator)\nrules = CatRules(ability_estimator,\n AbilityVarianceStateCriterion(dist_ability_est, integrator),\n FixedItemsTerminationCondition(max_questions))\n\npoints = 500\nxs = range(-2.5, 2.5, length = points)\nraw_estimator = LikelihoodAbilityEstimator()\nrecorder = CatRecorder(xs, responses, integrator, raw_estimator, ability_estimator)\nfor testee_idx in axes(responses, 2)\n tracked_responses, θ = run_cat(CatLoopConfig(rules = rules,\n get_response = auto_responder(@view responses[:, testee_idx]),\n new_response_callback = (tracked_responses, terminating) -> recorder(tracked_responses,\n testee_idx,\n terminating)),\n item_bank)\n true_θ = abilities[testee_idx]\n abs_err = abs(θ - true_θ)\nend","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"Make a plot showing how the estimated value evolves during the CAT. We also plot the 'true' values used to generate the responses.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"conv_lines_fig = ability_evolution_lines(recorder; abilities = abilities)\nconv_lines_fig","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"Make an interactive plot, showing how the distribution of the ability likelihood evolves.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"conv_dist_fig = lh_evolution_interactive(recorder; abilities = abilities)\nconv_dist_fig","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"This page was generated using DemoCards.jl and Literate.jl.","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/examples/irfs.jl\"","category":"page"},{"location":"examples/examples/irfs/#Item-Response-Functions","page":"Item Response Functions","title":"Item Response Functions","text":"","category":"section"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"(Image: Source code) (Image: notebook)","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"Item Response Functions","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"using Makie\nimport Pkg\nusing Distributions: Normal, cdf\nusing PsychometricsBazaarBase.ConstDistributions: normal_scaled_logistic\nusing AdaptiveTestPlots\n\n@automakie()","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"Typically, the logistic c.d.f. is used as the transfer function in IRT. However, it in an IRT context, a scaled version intended to be close to a normal c.d.f. is often used. The main advantage is that this is usually faster to compute. ComputerAdaptiveTesting provides normalscaledlogistic, which is also used by default, for this purpose:","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"xs = -8:0.05:8\nlines(xs, cdf.(Normal(), xs))\nlines!(xs, cdf.(normal_scaled_logistic, xs))\ncurrent_figure()","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"
\n
\n \n
\n
\n
\n \n
\n \n
\n
\n
","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"This page was generated using DemoCards.jl and Literate.jl.","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/examples/vocab_iq.jl\"","category":"page"},{"location":"examples/examples/vocab_iq/#vocab_iq","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"","category":"section"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"(Image: Source code) (Image: notebook)","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"Running a CAT based based on real response data","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"This example shows how to run a CAT end-to-end on real data.","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"First a 1-dimensional IRT model is fitted based on open response data to the vocabulary IQ test using the IRTSupport package which internally, this uses the mirt R package. Next, the model is used to administer the test interactively.","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"using Base.Filesystem\nusing ComputerAdaptiveTesting\nusing FittedItemBanks.DummyData: std_normal\nusing ComputerAdaptiveTesting.Sim\nusing ComputerAdaptiveTesting.NextItemRules\nusing ComputerAdaptiveTesting.TerminationConditions\nusing ComputerAdaptiveTesting.Aggregators\nusing FittedItemBanks\nusing FittedItemBanks: item_params\nimport PsychometricsBazaarBase.IntegralCoeffs\nusing PsychometricsBazaarBase.Integrators\nusing PsychometricsBazaarBase.Optimizers\nusing ItemResponseDatasets: prompt_readline\nusing ItemResponseDatasets.VocabIQ\nusing RIrtWrappers.Mirt\n\nfunction get_item_bank()\n fit_4pl(get_marked_df_cached(); TOL = 1e-2)\nend\n\nfunction run_vocab_iq_cat()\n item_bank, labels = get_item_bank()\n integrator = FixedGKIntegrator(-6, 6, 61)\n ability_integrator = AbilityIntegrator(integrator)\n dist_ability_est = PriorAbilityEstimator(std_normal)\n optimizer = AbilityOptimizer(OneDimOptimOptimizer(-6.0, 6.0, NelderMead()))\n ability_estimator = ModeAbilityEstimator(dist_ability_est, optimizer)\n @info \"run_cat\" ability_estimator\n rules = CatRules(ability_estimator,\n AbilityVarianceStateCriterion(dist_ability_est, ability_integrator),\n FixedItemsTerminationCondition(45))\n function get_response(response_idx, response_name)\n params = item_params(item_bank, response_idx)\n println(\"Parameters for next question: $params\")\n prompt_readline(VocabIQ.questions[response_idx])\n end\n function new_response_callback(tracked_responses, terminating)\n if tracked_responses.responses.values[end] > 0\n println(\"Correct\")\n else\n println(\"Wrong\")\n end\n ability = ability_estimator(tracked_responses)\n var = variance_given_mean(ability_integrator,\n dist_ability_est,\n tracked_responses,\n ability)\n println(\"Got ability estimate: $ability ± $var\")\n println(\"\")\n end\n loop_config = CatLoopConfig(rules = rules,\n get_response = get_response,\n new_response_callback = new_response_callback)\n run_cat(loop_config, item_bank)\nend\n\nrun_vocab_iq_cat()","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"This page was generated using DemoCards.jl and Literate.jl.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/examples/ability_convergence_mirt.jl\"","category":"page"},{"location":"examples/examples/ability_convergence_mirt/#ability_convergence_mirt","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"","category":"section"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"(Image: Source code) (Image: notebook)","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"How abilities converge on simulated MIRT data","category":"page"},{"location":"examples/examples/ability_convergence_mirt/#Running-a-CAT-based-on-a-synthetic-correct/incorrect-MIRT-model","page":"How abilities converge on simulated MIRT data","title":"Running a CAT based on a synthetic correct/incorrect MIRT model","text":"","category":"section"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"This example shows how to run a CAT based on a synthetic correct/incorrect MIRT model.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"Import order is important. We put ComputerAdaptiveTesting last so we get the extra dependencies.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"using Makie\nimport Pkg\nimport Random\nusing Distributions: Normal, cdf\nusing AlgebraOfGraphics\nusing ComputerAdaptiveTesting\nusing ComputerAdaptiveTesting.Sim: auto_responder\nusing ComputerAdaptiveTesting.NextItemRules: DRuleItemCriterion\nusing ComputerAdaptiveTesting.TerminationConditions: FixedItemsTerminationCondition\nusing ComputerAdaptiveTesting.Aggregators: PriorAbilityEstimator,\n MeanAbilityEstimator, LikelihoodAbilityEstimator\nusing FittedItemBanks\nimport PsychometricsBazaarBase.IntegralCoeffs\nusing PsychometricsBazaarBase.Integrators\nusing PsychometricsBazaarBase.ConstDistributions: normal_scaled_logistic\nusing AdaptiveTestPlots\n\n@automakie()","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"Now we are read to generate our synthetic data using the supplied DummyData module. We generate an item bank with 100 items and fake responses for 3 testees.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"dims = 2\nusing FittedItemBanks.DummyData: dummy_full, std_mv_normal, SimpleItemBankSpec, StdModel4PL\nusing ComputerAdaptiveTesting.Responses: BooleanResponse","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"TODO: pass in dims","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"(item_bank, abilities, responses) = dummy_full(Random.default_rng(42),\n SimpleItemBankSpec(StdModel4PL(), VectorContinuousDomain(), BooleanResponse()),\n dims;\n num_questions = 10,\n num_testees = 2)","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"Simulate a CAT for each testee and record it using CatRecorder. CatRecorder collects information which can be used to draw different types of plots.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"max_questions = 9\nintegrator = CubaIntegrator([-6.0, -6.0], [6.0, 6.0], CubaVegas(); rtol = 1e-2)\nability_estimator = MeanAbilityEstimator(PriorAbilityEstimator(std_mv_normal(dims)),\n integrator)\nrules = CatRules(ability_estimator,\n DRuleItemCriterion(ability_estimator),\n FixedItemsTerminationCondition(max_questions))","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"XXX: We shouldn't need to specify xs here since the distributions are not used – rework","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"points = 3\nxs = repeat(range(-2.5, 2.5, length = points)', dims, 1)\nraw_estimator = LikelihoodAbilityEstimator()\nrecorder = CatRecorder(xs,\n responses,\n integrator,\n raw_estimator,\n ability_estimator,\n abilities)\nfor testee_idx in axes(responses, 2)\n @debug \"Running for testee\" testee_idx\n tracked_responses, θ = run_cat(CatLoopConfig(rules = rules,\n get_response = auto_responder(@view responses[:, testee_idx]),\n new_response_callback = (tracked_responses, terminating) -> recorder(tracked_responses,\n testee_idx,\n terminating)),\n item_bank)\n true_θ = abilities[:, testee_idx]\n abs_err = sum(abs.(θ .- true_θ))\n @info \"convergence\" true_θ θ abs_err\nend","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"Make a plot showing how the estimated value converges during the CAT.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"conv_lines_fig = ability_convergence_lines(recorder; abilities = abilities)\nconv_lines_fig","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"This page was generated using DemoCards.jl and Literate.jl.","category":"page"},{"location":"#Getting-started","page":"Getting started","title":"Getting started","text":"","category":"section"},{"location":"","page":"Getting started","title":"Getting started","text":"This is the documentation for ComputerAdaptiveTesting. For information about the purpose of the package and its installation, please see the README.md.","category":"page"},{"location":"#Contents","page":"Getting started","title":"Contents","text":"","category":"section"},{"location":"","page":"Getting started","title":"Getting started","text":"Creating a CAT: A guide to creating a CAT using the configuration structs in ComputerAdaptiveTesting.jl\nUsing your CAT: A collection of mini topical guides about: simulating CATs, integrating into a user-facing applications, evaluating CATs, visualising CATs, and constructing decision trees from CATs\nExamples: A collection of example code which you can run and modify to get a feel for how to use the package\nAPI reference: The full API reference for the package","category":"page"}] +[{"location":"contributing/#Contributing","page":"Contributing","title":"Contributing","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Contributions are welcome. Please discuss any larger changes in the issues before making a pull request to avoid wasted work.","category":"page"},{"location":"contributing/#Running-tests","page":"Contributing","title":"Running tests","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"You can run tests locally like so:","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"julia --project=test test/runtests.jl","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Unfortunately, Pkg.test() does not work properly at the moment. See this issue.","category":"page"},{"location":"using_your_cat/#Using-your-CAT","page":"Using your CAT","title":"Using your CAT","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"Now you've created your cat according to Creating a CAT, there are a number of ways you can use it. This section covers a few. See also the Examples.","category":"page"},{"location":"using_your_cat/#Simulating-CATs","page":"Using your CAT","title":"Simulating CATs","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"TODO","category":"page"},{"location":"using_your_cat/#Integrating-into-a-user-facing-applications","page":"Using your CAT","title":"Integrating into a user-facing applications","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"TODO","category":"page"},{"location":"using_your_cat/#Evaluating-CATs","page":"Using your CAT","title":"Evaluating CATs","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"TODO","category":"page"},{"location":"using_your_cat/#Visualising-CATs","page":"Using your CAT","title":"Visualising CATs","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"TODO","category":"page"},{"location":"using_your_cat/#Constructing-decision-trees-from-CATs","page":"Using your CAT","title":"Constructing decision trees from CATs","text":"","category":"section"},{"location":"using_your_cat/","page":"Using your CAT","title":"Using your CAT","text":"TODO","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/examples/perf_plots.jl\"","category":"page"},{"location":"examples/examples/perf_plots/#Perf-plots","page":"Perf plots","title":"Perf plots","text":"","category":"section"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"(Image: Source code) (Image: notebook)","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"Plots for performance of different integrators for ability estimators","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"using Makie\nimport Pkg\nusing CSV\nusing DataFrames\nusing CategoricalArrays: levels\nusing IterTools: chain\nusing ComputerAdaptiveTesting\nusing AdaptiveTestPlots\nusing AdaptiveTestPlots: LabelledToggleGrid, MenuGrid\n\nDOCS_DATA = pkgdir(ComputerAdaptiveTesting) * \"/docs/data/\"\n\n@automakie()\n\ndf = CSV.read((DOCS_DATA * \"integrator_benchmark_results.csv\"),\n DataFrame;\n types = Dict(\"value\" => Float64,\n \"err\" => Float64,\n \"bytes\" => Int64,\n \"response\" => Int32,\n \"item_bank\" => Int32,\n \"dim\" => Int32,\n \"trial\" => Int32,\n \"time\" => Float64,\n \"gctime\" => Float64,\n \"rtol\" => Float64))\n\nfunction plot(df)\n fig = Figure()\n ax = Axis(fig[1, 1])\n\n groupables = [\"item_bank\", \"response\", \"trial\", \"dim\", \"integrator\"]\n regressables = [\"integrator\", \"dim\", \"rtol\", \"value\", \"err\", \"bytes\", \"gctime\"]\n\n Menu(fig, options = [\"none\", groupables...], default = \"none\")\n toggles = []\n sliders = []\n slider_idxs = []\n menus = []\n menu_idxs = []\n for (idx, groupable) in enumerate(groupables)\n push!(toggles, (label = groupable,))\n opts = sort(levels(df[!, groupable]))\n if eltype(opts) <: AbstractString\n push!(slider_idxs, idx)\n push!(menus, (label = groupable,\n options = opts))\n else\n push!(menu_idxs, idx)\n push!(sliders, (label = groupable,\n range = opts))\n end\n end\n tgrid = LabelledToggleGrid(fig,\n toggles...,\n width = 350,\n tellheight = false)\n lsgrid = SliderGrid(fig,\n sliders...,\n width = 350,\n tellheight = false)\n mgrid = MenuGrid(fig,\n menus...,\n width = 350,\n tellheight = false)\n\n working_df = Observable(nothing)\n function update_working_df(_)\n df_prime = df\n slider_menu_vals = Vector{Any}(undef, length(sliders) + length(menus))\n slider_menu_vals[slider_idxs] = [slider.value for slider in lsgrid.sliders]\n slider_menu_vals[menu_idxs] = [menu.selection for menu in mgrid.menus]\n for (groupable, toggle, value) in zip(groupables, tgrid.toggles, slider_menu_vals)\n if !toggle.value\n continue\n end\n df_prime = filter(working_df, groupable => value)\n end\n working_df[] = df_prime\n end\n\n for observable in chain((t.active for t in tgrid.toggles),\n (s.value for s in lsgrid.sliders),\n (m.selection for m in mgrid.menus))\n on(update_working_df, observable)\n end\n\n x_var_menu = Menu(fig, options = regressables)\n y_var_menu = Menu(fig, options = regressables)\n coded_menu = Menu(fig, options = regressables)\n\n xs = @lift($working_df !== nothing && $(x_var_menu.selection) !== nothing ?\n $working_df[$(x_var_menu.selection)] : nothing)\n ys = @lift($working_df !== nothing && $(y_var_menu.selection) !== nothing ?\n $working_df[$(y_var_menu.selection)] : nothing)\n\n scatter = @lift($xs !== nothing && $ys !== nothing &&\n $(coded_menu.selection) !== nothing ?\n lines!(ax,\n $xs,\n $ys;\n color = $(coded_menu.selection)) : nothing)\n\n fig\nend\n\nplot(df)","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"
\n
\n \n
\n
\n
\n \n
\n \n
\n
\n
","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"","category":"page"},{"location":"examples/examples/perf_plots/","page":"Perf plots","title":"Perf plots","text":"This page was generated using DemoCards.jl and Literate.jl.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/index.md\"","category":"page"},{"location":"examples/#demo-page","page":"Examples","title":"Examples and demos","text":"","category":"section"},{"location":"examples/","page":"Examples","title":"Examples","text":"You can browse through the examples below. Each example has code and outputs, as well as links to download a script to run locally or to view as a Jupyter notebook on NBViewer. Some of the examples have interactive plots, and these will not currently work unless the scripts are run locally. TODO: Add binder links to examples.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"If you would like to run the examples as Jupyter notebooks locally to see interactive plots inline with the code, you can clone the repository and run docs/build_nbs.jl like so:","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"$ git clone https://github.com/JuliaPsychometricsBazaar/ComputerAdaptiveTesting.jl.git\n$ cd ComputerAdaptiveTesting\n$ julia --project=. docs/build_nbs.jl","category":"page"},{"location":"examples/#Examples","page":"Examples","title":"Examples","text":"","category":"section"},{"location":"examples/","page":"Examples","title":"Examples","text":"
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n\n \n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"How abilities converge on simulated 3PL data","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"This example shows how to run a CAT based on a synthetic correct/incorrect 3PL IRT model.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n\n \n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"How abilities converge on simulated MIRT data","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"This example shows how to run a CAT based on a synthetic correct/incorrect MIRT model.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n\n \n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Item Response Functions","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Typically, the logistic c.d.f. is used as the transfer function in IRT. However, it in an IRT context, a scaled version intended to be close to a normal c.d.f. is often used. The main advantage is that this is usually faster to compute. ComputerAdaptiveTesting provides normalscaledlogistic, which is also used by default, for this purpose:","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n\n \n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Perf plots","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Perf plots","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n\n \n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"Running a CAT based based on real response data","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"This example shows how to run a CAT end-to-end on real data.","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
\n
\n
","category":"page"},{"location":"examples/","page":"Examples","title":"Examples","text":"
","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"CurrentModule = ComputerAdaptiveTesting","category":"page"},{"location":"creating_a_cat/#Creating-a-CAT","page":"Creating a CAT","title":"Creating a CAT","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"This guide gives a brief overview of how to create a CAT using the configuration structs in ComputerAdaptiveTesting.jl.","category":"page"},{"location":"creating_a_cat/#API-design","page":"Creating a CAT","title":"API design","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The configuration of a CAT is built up as a tree of configuration structs. These structs are all subtypes of CatConfigBase.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.CatConfig.CatConfigBase","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.ConfigBase.CatConfigBase-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.ConfigBase.CatConfigBase","text":"abstract type CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The constructors for the configuration structs in this package tend to have smart defaults. In general most constructors have two forms. The first is an explicit keyword constructor form, where all arguments are given:","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ConfigurationObject(\n field1=value1,\n field1=value2,\n)","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The second is an implicit form, where arguments are given in any order. If possible, they will be used together will appropriately guessed defaults to construct the configuration:","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ConfigurationObject(value2, value1)","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The implicit form is particularly useful for quick prototyping. Implicit form constructors are also available for some abstract types. In this case, they will return a concrete type that is a reasonable default for the abstract type.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"After using the implicit form, you can print the object to see what values have been filled in. This may be useful in case you want to modify some of the defaults or switch to the explicit form.","category":"page"},{"location":"creating_a_cat/#Item-banks","page":"Creating a CAT","title":"Item banks","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"Item banks are the source of items for the test. The basic definitions are provided by FittedItemBanks.jl and can be fit to data using RIrtWrappers.jl. See the documentation pages of those packages for more information.","category":"page"},{"location":"creating_a_cat/#CatRules","page":"Creating a CAT","title":"CatRules","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"This is the main type for configuring a CAT. It contains the item bank, the next item selection rule, and the stopping rule. CatRules has explicit and implicit constructors.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.CatConfig.CatRules","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.CatConfig.CatRules-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.CatConfig.CatRules","text":"struct CatRules{NextItemRuleT<:ComputerAdaptiveTesting.NextItemRules.NextItemRule, TerminationConditionT<:ComputerAdaptiveTesting.TerminationConditions.TerminationCondition, AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.AbilityEstimator, AbilityTrackerT<:ComputerAdaptiveTesting.Aggregators.AbilityTracker} <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\nnext_item::ComputerAdaptiveTesting.NextItemRules.NextItemRule: The rule to choose the next item in the CAT given the current state.\n\ntermination_condition::ComputerAdaptiveTesting.TerminationConditions.TerminationCondition: The rule to choose when to terminate the CAT.\n\nability_estimator::ComputerAdaptiveTesting.Aggregators.AbilityEstimator: The ability estimator, which estimates the testee's current ability.\n\nability_tracker::ComputerAdaptiveTesting.Aggregators.AbilityTracker: The ability tracker, which tracks the testee's current ability level. Default: NullAbilityTracker()\n\nConfiguration of the rules for a CAT. This all includes all the basic rules for the CAT's operation, but not the item bank, nor any of the interactivity hooks needed to actually run the CAT.\n\nThis may be more a more convenient layer to integrate than CatLoopConfig if you want to write your own CAT loop rather than using hooks.\n\nCatRules(; next_item=..., termination_condition=..., ability_estimator=..., ability_tracker=...)\n\nExplicit constructor for CatRules.\n\nCatRules(bits...)\n\nImplicit constructor for CatRules.\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/#Next-item-selection-with-NextItemRule","page":"Creating a CAT","title":"Next item selection with NextItemRule","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The next item selection rule is the most important part of the CAT. Each rule extends the NextItemRule abstract type.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.NextItemRule","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.NextItemRule-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.NextItemRule","text":"abstract type NextItemRule <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\nAbstract base type for all item selection rules. All descendants of this type are expected to implement the interface (rule::NextItemRule)(responses::TrackedResponses, items::AbstractItemBank)::Int\n\nNextItemRule(bits...; ability_estimator=nothing, parallel=true)\n\nImplicit constructor for NextItemRule. Uses any given NextItemRule or delegates to ItemStrategyNextItemRule.\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"A sort of null hypothesis next item selection rule is RandomNextItemRule, which ","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule","text":"struct RandomNextItemRule{RandomT<:Random.AbstractRNG} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule\n\nrng::Random.AbstractRNG: Default: Xoshiro()\n\nThis is the most basic rule for choosing the next item in a CAT. It simply picks a random item from the set of items that have not yet been administered.\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"Other rules are created by combining a ItemCriterion – which somehow rates items according to how good they are – with a NextItemStrategy using an ItemStrategyNextItemRule, which acts as an adapter. The default NextItemStrategy (and currently only) is ExhaustiveSearch1Ply. When using the implicit constructors, ItemCriterion can therefore be used directly without wrapping in any place an NextItemRule is expected.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule","text":"struct ItemStrategyNextItemRule{NextItemStrategyT<:ComputerAdaptiveTesting.NextItemRules.NextItemStrategy, ItemCriterionT<:ComputerAdaptiveTesting.NextItemRules.ItemCriterion} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule\n\nstrategy::ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\ncriterion::ComputerAdaptiveTesting.NextItemRules.ItemCriterion\n\nItemStrategyNextItemRule which together with a NextItemStrategy acts as an adapter by which an ItemCriterion can serve as a NextItemRule.\n\nItemStrategyNextItemRule(bits...; ability_estimator=nothing, parallel=true)\n\nImplicit constructor for ItemStrategyNextItemRule. Will default to ExhaustiveSearch1Ply when no NextItemStrategy is given.\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.ItemCriterion","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.ItemCriterion-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.ItemCriterion","text":"abstract type ItemCriterion <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.NextItemStrategy","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.NextItemStrategy-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.NextItemStrategy","text":"abstract type NextItemStrategy <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply","text":"struct ExhaustiveSearch1Ply <: ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\n\nparallel::Bool\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/#Evaluating-item-and-state-merit-with-ItemCriterion-and-StateCriterion","page":"Creating a CAT","title":"Evaluating item and state merit with ItemCriterion and StateCriterion","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"The ItemCriterion abstract type is used to rate items according to how good they are as a candidate for the next item. A typical example is InformationItemCriterion, which using the current ability estimate theta and the item response function irf to calculate each item's information fracirf_θ^2irf_θ * (1 - irf_θ).","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"Within this, there are two types of ExpectationBasedItemCriterion which act as adapters: PointExpectationBasedItemCriterion and DistributionExpectationBasedItemCriterion. These take a StateCriterion, which evaluates how good a particular state is in terms getting a good estimate of the test takers ability. They look one ply ahead to get the expected value of the StateCriterion after selecting the given item. The AbilityVarianceStateCriterion looks at the variance of the ability theta estimate at that state.","category":"page"},{"location":"creating_a_cat/#Stopping-rules-with-TerminationCondition","page":"Creating a CAT","title":"Stopping rules with TerminationCondition","text":"","category":"section"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"Currently the only TerminationCondition is FixedItemsTerminationCondition, which ends the test after a fixed number of items.","category":"page"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.TerminationConditions.TerminationCondition","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.TerminationConditions.TerminationCondition-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.TerminationConditions.TerminationCondition","text":"abstract type TerminationCondition <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"creating_a_cat/","page":"Creating a CAT","title":"Creating a CAT","text":"ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition","category":"page"},{"location":"creating_a_cat/#ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition-creating_a_cat","page":"Creating a CAT","title":"ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition","text":"struct FixedItemsTerminationCondition <: ComputerAdaptiveTesting.TerminationConditions.TerminationCondition\n\nnum_items::Int64\n\n\n\n\n\n","category":"type"},{"location":"api/","page":"API reference","title":"API reference","text":"CurrentModule = ComputerAdaptiveTesting","category":"page"},{"location":"api/#API-reference","page":"API reference","title":"API reference","text":"","category":"section"},{"location":"api/","page":"API reference","title":"API reference","text":"","category":"page"},{"location":"api/","page":"API reference","title":"API reference","text":"Modules = [ComputerAdaptiveTesting, ComputerAdaptiveTesting.Aggregators, ComputerAdaptiveTesting.Responses, ComputerAdaptiveTesting.Sim, ComputerAdaptiveTesting.TerminationConditions, ComputerAdaptiveTesting.NextItemRules, ComputerAdaptiveTesting.CatConfig]","category":"page"},{"location":"api/#ComputerAdaptiveTesting.Aggregators","page":"API reference","title":"ComputerAdaptiveTesting.Aggregators","text":"This module takes care of integrating and optimizing over the ability/difficulty space. It includes TrackedResponses, which can store cumulative results during a test.\n\n\n\n\n\n","category":"module"},{"location":"api/#ComputerAdaptiveTesting.Aggregators.RiemannEnumerationIntegrator","page":"API reference","title":"ComputerAdaptiveTesting.Aggregators.RiemannEnumerationIntegrator","text":"In case an item bank is enumerable (e.g. GriddedItemBank), then this method integrates over the ability likihood given a set of responses with a given coefficient function using a Riemann sum (aka the rectangle rule).\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.Aggregators.maybe_tracked_ability_estimate-Tuple{ComputerAdaptiveTesting.Aggregators.TrackedResponses, Any}","page":"API reference","title":"ComputerAdaptiveTesting.Aggregators.maybe_tracked_ability_estimate","text":"This method returns a tracked point estimate if it is has the given ability estimator, otherwise it computes it using the given ability estimator.\n\n\n\n\n\n","category":"method"},{"location":"api/#ComputerAdaptiveTesting.Sim.auto_responder-Tuple{Any}","page":"API reference","title":"ComputerAdaptiveTesting.Sim.auto_responder","text":"This function constructs a next item function which automatically responds according to responses.\n\n\n\n\n\n","category":"method"},{"location":"api/#ComputerAdaptiveTesting.Sim.prompt_response-Tuple{Any, Any}","page":"API reference","title":"ComputerAdaptiveTesting.Sim.prompt_response","text":"This response callback simply prompts \n\n\n\n\n\n","category":"method"},{"location":"api/#ComputerAdaptiveTesting.Sim.run_cat-Union{Tuple{RulesT}, Tuple{CatLoopConfig{RulesT}, FittedItemBanks.AbstractItemBank}} where RulesT<:CatRules","page":"API reference","title":"ComputerAdaptiveTesting.Sim.run_cat","text":"Run a given CatLoopConfig\n\n\n\n\n\n","category":"method"},{"location":"api/#ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition","page":"API reference","title":"ComputerAdaptiveTesting.TerminationConditions.FixedItemsTerminationCondition","text":"struct FixedItemsTerminationCondition <: ComputerAdaptiveTesting.TerminationConditions.TerminationCondition\n\nnum_items::Int64\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.TerminationConditions.TerminationCondition","page":"API reference","title":"ComputerAdaptiveTesting.TerminationConditions.TerminationCondition","text":"abstract type TerminationCondition <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules","text":"This module implements the next item selection rules, which form the main part of CAT.\n\nBibliography\n\n[1] Linden, W. J., & Pashley, P. J. (2009). Item selection and ability estimation in adaptive testing. In Elements of adaptive testing (pp. 3-30). Springer, New York, NY.\n\n\n\n\n\n","category":"module"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.catr_next_item_aliases","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.catr_next_item_aliases","text":"This mapping provides next item rules through the same names that they are available through in the catR R package. TODO compability with mirtcat\n\n\n\n\n\n","category":"constant"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.AbilityVarianceStateCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.AbilityVarianceStateCriterion","text":"struct AbilityVarianceStateCriterion{DistEst<:ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator, IntegratorT<:ComputerAdaptiveTesting.Aggregators.AbilityIntegrator} <: ComputerAdaptiveTesting.NextItemRules.StateCriterion\n\ndist_est::ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator\nintegrator::ComputerAdaptiveTesting.Aggregators.AbilityIntegrator\nskip_zero::Bool\n\nThis StateCriterion returns the variance of the ability estimate given a set of responses.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.DistributionExpectationBasedItemCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.DistributionExpectationBasedItemCriterion","text":"struct DistributionExpectationBasedItemCriterion{DistributionAbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator, AbilityIntegratorT<:ComputerAdaptiveTesting.Aggregators.AbilityIntegrator, StateCriterionT<:ComputerAdaptiveTesting.NextItemRules.StateCriterion} <: ComputerAdaptiveTesting.NextItemRules.ExpectationBasedItemCriterion\n\nability_estimator::ComputerAdaptiveTesting.Aggregators.DistributionAbilityEstimator\nintegrator::ComputerAdaptiveTesting.Aggregators.AbilityIntegrator\nstate_criterion::ComputerAdaptiveTesting.NextItemRules.StateCriterion\n\nThis ItemCriterion wraps a StateCriterion and looks at its expected value for a particular item 1-ply ahead by integrating over an ability curve.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.ExhaustiveSearch1Ply","text":"struct ExhaustiveSearch1Ply <: ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\n\nparallel::Bool\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.ItemCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.ItemCriterion","text":"abstract type ItemCriterion <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.ItemStrategyNextItemRule","text":"struct ItemStrategyNextItemRule{NextItemStrategyT<:ComputerAdaptiveTesting.NextItemRules.NextItemStrategy, ItemCriterionT<:ComputerAdaptiveTesting.NextItemRules.ItemCriterion} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule\n\nstrategy::ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\ncriterion::ComputerAdaptiveTesting.NextItemRules.ItemCriterion\n\nItemStrategyNextItemRule which together with a NextItemStrategy acts as an adapter by which an ItemCriterion can serve as a NextItemRule.\n\nItemStrategyNextItemRule(bits...; ability_estimator=nothing, parallel=true)\n\nImplicit constructor for ItemStrategyNextItemRule. Will default to ExhaustiveSearch1Ply when no NextItemStrategy is given.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.KernelAbstractionsExhaustiveSearch","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.KernelAbstractionsExhaustiveSearch","text":"struct KernelAbstractionsExhaustiveSearch{KernelT} <: ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\n\nkernel::Any\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.KernelAbstractionsExhaustiveSearchConfig","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.KernelAbstractionsExhaustiveSearchConfig","text":"struct KernelAbstractionsExhaustiveSearchConfig{ArgsT} <: ComputerAdaptiveTesting.NextItemRules.NextItemStrategy\n\nkernel_args::Any\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.NextItemRule","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.NextItemRule","text":"abstract type NextItemRule <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\nAbstract base type for all item selection rules. All descendants of this type are expected to implement the interface (rule::NextItemRule)(responses::TrackedResponses, items::AbstractItemBank)::Int\n\nNextItemRule(bits...; ability_estimator=nothing, parallel=true)\n\nImplicit constructor for NextItemRule. Uses any given NextItemRule or delegates to ItemStrategyNextItemRule.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.NextItemStrategy","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.NextItemStrategy","text":"abstract type NextItemStrategy <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.PointExpectationBasedItemCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.PointExpectationBasedItemCriterion","text":"struct PointExpectationBasedItemCriterion{PointAbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator, StateCriterionT<:ComputerAdaptiveTesting.NextItemRules.StateCriterion} <: ComputerAdaptiveTesting.NextItemRules.ExpectationBasedItemCriterion\n\nability_estimator::ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator\nstate_criterion::ComputerAdaptiveTesting.NextItemRules.StateCriterion\n\nThis ItemCriterion wraps a StateCriterion and looks at its expected value for a particular item 1-ply ahead based on a point ability estimate.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.RandomNextItemRule","text":"struct RandomNextItemRule{RandomT<:Random.AbstractRNG} <: ComputerAdaptiveTesting.NextItemRules.NextItemRule\n\nrng::Random.AbstractRNG: Default: Xoshiro()\n\nThis is the most basic rule for choosing the next item in a CAT. It simply picks a random item from the set of items that have not yet been administered.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.StateCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.StateCriterion","text":"abstract type StateCriterion <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.NextItemRules.UrryItemCriterion","page":"API reference","title":"ComputerAdaptiveTesting.NextItemRules.UrryItemCriterion","text":"struct UrryItemCriterion{AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator} <: ComputerAdaptiveTesting.NextItemRules.ItemCriterion\n\nability_estimator::ComputerAdaptiveTesting.Aggregators.PointAbilityEstimator\n\nThis item criterion just picks the item with the raw difficulty closest to the current ability estimate.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.CatConfig.CatLoopConfig","page":"API reference","title":"ComputerAdaptiveTesting.CatConfig.CatLoopConfig","text":"Configuration for a simulatable CAT.\n\n\n\n\n\n","category":"type"},{"location":"api/#ComputerAdaptiveTesting.CatConfig.CatRules","page":"API reference","title":"ComputerAdaptiveTesting.CatConfig.CatRules","text":"struct CatRules{NextItemRuleT<:ComputerAdaptiveTesting.NextItemRules.NextItemRule, TerminationConditionT<:ComputerAdaptiveTesting.TerminationConditions.TerminationCondition, AbilityEstimatorT<:ComputerAdaptiveTesting.Aggregators.AbilityEstimator, AbilityTrackerT<:ComputerAdaptiveTesting.Aggregators.AbilityTracker} <: ComputerAdaptiveTesting.ConfigBase.CatConfigBase\n\nnext_item::ComputerAdaptiveTesting.NextItemRules.NextItemRule: The rule to choose the next item in the CAT given the current state.\n\ntermination_condition::ComputerAdaptiveTesting.TerminationConditions.TerminationCondition: The rule to choose when to terminate the CAT.\n\nability_estimator::ComputerAdaptiveTesting.Aggregators.AbilityEstimator: The ability estimator, which estimates the testee's current ability.\n\nability_tracker::ComputerAdaptiveTesting.Aggregators.AbilityTracker: The ability tracker, which tracks the testee's current ability level. Default: NullAbilityTracker()\n\nConfiguration of the rules for a CAT. This all includes all the basic rules for the CAT's operation, but not the item bank, nor any of the interactivity hooks needed to actually run the CAT.\n\nThis may be more a more convenient layer to integrate than CatLoopConfig if you want to write your own CAT loop rather than using hooks.\n\nCatRules(; next_item=..., termination_condition=..., ability_estimator=..., ability_tracker=...)\n\nExplicit constructor for CatRules.\n\nCatRules(bits...)\n\nImplicit constructor for CatRules.\n\n\n\n\n\n","category":"type"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/examples/ability_convergence_3pl.jl\"","category":"page"},{"location":"examples/examples/ability_convergence_3pl/#ability_convergence_3pl","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"","category":"section"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"(Image: Source code) (Image: notebook)","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"How abilities converge on simulated 3PL data","category":"page"},{"location":"examples/examples/ability_convergence_3pl/#Running-a-CAT-based-on-a-synthetic-correct/incorrect-3PL-IRT-model","page":"How abilities converge on simulated 3PL data","title":"Running a CAT based on a synthetic correct/incorrect 3PL IRT model","text":"","category":"section"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"This example shows how to run a CAT based on a synthetic correct/incorrect 3PL IRT model.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"Import order is important. We put ComputerAdaptiveTesting last so we get the extra dependencies.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"using Makie\nimport Pkg\nimport Random\nusing Distributions: Normal, cdf\nusing AlgebraOfGraphics\nusing ComputerAdaptiveTesting\nusing ComputerAdaptiveTesting.Sim: auto_responder\nusing ComputerAdaptiveTesting.NextItemRules: AbilityVarianceStateCriterion\nusing ComputerAdaptiveTesting.TerminationConditions: FixedItemsTerminationCondition\nusing ComputerAdaptiveTesting.Aggregators: PriorAbilityEstimator,\n MeanAbilityEstimator, LikelihoodAbilityEstimator\nusing FittedItemBanks\nusing ComputerAdaptiveTesting.Responses: BooleanResponse\nimport PsychometricsBazaarBase.IntegralCoeffs\nusing PsychometricsBazaarBase.Integrators\nusing PsychometricsBazaarBase.ConstDistributions: normal_scaled_logistic\nusing AdaptiveTestPlots\n\n@automakie()","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"Now we are read to generate our synthetic data using the supplied DummyData module. We generate an item bank with 100 items and fake responses for 3 testees.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"using FittedItemBanks.DummyData: dummy_full, std_normal, SimpleItemBankSpec, StdModel3PL\n(item_bank, abilities, responses) = dummy_full(Random.default_rng(42),\n SimpleItemBankSpec(StdModel3PL(), OneDimContinuousDomain(), BooleanResponse());\n num_questions = 100,\n num_testees = 3)","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"Simulate a CAT for each testee and record it using CatRecorder. CatRecorder collects information which can be used to draw different types of plots.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"max_questions = 99\nintegrator = FixedGKIntegrator(-6, 6, 80)\ndist_ability_est = PriorAbilityEstimator(std_normal)\nability_estimator = MeanAbilityEstimator(dist_ability_est, integrator)\nrules = CatRules(ability_estimator,\n AbilityVarianceStateCriterion(dist_ability_est, integrator),\n FixedItemsTerminationCondition(max_questions))\n\npoints = 500\nxs = range(-2.5, 2.5, length = points)\nraw_estimator = LikelihoodAbilityEstimator()\nrecorder = CatRecorder(xs, responses, integrator, raw_estimator, ability_estimator)\nfor testee_idx in axes(responses, 2)\n tracked_responses, θ = run_cat(CatLoopConfig(rules = rules,\n get_response = auto_responder(@view responses[:, testee_idx]),\n new_response_callback = (tracked_responses, terminating) -> recorder(tracked_responses,\n testee_idx,\n terminating)),\n item_bank)\n true_θ = abilities[testee_idx]\n abs_err = abs(θ - true_θ)\nend","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"Make a plot showing how the estimated value evolves during the CAT. We also plot the 'true' values used to generate the responses.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"conv_lines_fig = ability_evolution_lines(recorder; abilities = abilities)\nconv_lines_fig","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"Make an interactive plot, showing how the distribution of the ability likelihood evolves.","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"conv_dist_fig = lh_evolution_interactive(recorder; abilities = abilities)\nconv_dist_fig","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"","category":"page"},{"location":"examples/examples/ability_convergence_3pl/","page":"How abilities converge on simulated 3PL data","title":"How abilities converge on simulated 3PL data","text":"This page was generated using DemoCards.jl and Literate.jl.","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/examples/irfs.jl\"","category":"page"},{"location":"examples/examples/irfs/#Item-Response-Functions","page":"Item Response Functions","title":"Item Response Functions","text":"","category":"section"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"(Image: Source code) (Image: notebook)","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"Item Response Functions","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"using Makie\nimport Pkg\nusing Distributions: Normal, cdf\nusing PsychometricsBazaarBase.ConstDistributions: normal_scaled_logistic\nusing AdaptiveTestPlots\n\n@automakie()","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"Typically, the logistic c.d.f. is used as the transfer function in IRT. However, it in an IRT context, a scaled version intended to be close to a normal c.d.f. is often used. The main advantage is that this is usually faster to compute. ComputerAdaptiveTesting provides normalscaledlogistic, which is also used by default, for this purpose:","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"xs = -8:0.05:8\nlines(xs, cdf.(Normal(), xs))\nlines!(xs, cdf.(normal_scaled_logistic, xs))\ncurrent_figure()","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"
\n
\n \n
\n
\n
\n \n
\n \n
\n
\n
","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"","category":"page"},{"location":"examples/examples/irfs/","page":"Item Response Functions","title":"Item Response Functions","text":"This page was generated using DemoCards.jl and Literate.jl.","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/examples/vocab_iq.jl\"","category":"page"},{"location":"examples/examples/vocab_iq/#vocab_iq","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"","category":"section"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"(Image: Source code) (Image: notebook)","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"Running a CAT based based on real response data","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"This example shows how to run a CAT end-to-end on real data.","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"First a 1-dimensional IRT model is fitted based on open response data to the vocabulary IQ test using the IRTSupport package which internally, this uses the mirt R package. Next, the model is used to administer the test interactively.","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"using Base.Filesystem\nusing ComputerAdaptiveTesting\nusing FittedItemBanks.DummyData: std_normal\nusing ComputerAdaptiveTesting.Sim\nusing ComputerAdaptiveTesting.NextItemRules\nusing ComputerAdaptiveTesting.TerminationConditions\nusing ComputerAdaptiveTesting.Aggregators\nusing FittedItemBanks\nusing FittedItemBanks: item_params\nimport PsychometricsBazaarBase.IntegralCoeffs\nusing PsychometricsBazaarBase.Integrators\nusing PsychometricsBazaarBase.Optimizers\nusing ItemResponseDatasets: prompt_readline\nusing ItemResponseDatasets.VocabIQ\nusing RIrtWrappers.Mirt\n\nfunction get_item_bank()\n fit_4pl(get_marked_df_cached(); TOL = 1e-2)\nend\n\nfunction run_vocab_iq_cat()\n item_bank, labels = get_item_bank()\n integrator = FixedGKIntegrator(-6, 6, 61)\n ability_integrator = AbilityIntegrator(integrator)\n dist_ability_est = PriorAbilityEstimator(std_normal)\n optimizer = AbilityOptimizer(OneDimOptimOptimizer(-6.0, 6.0, NelderMead()))\n ability_estimator = ModeAbilityEstimator(dist_ability_est, optimizer)\n @info \"run_cat\" ability_estimator\n rules = CatRules(ability_estimator,\n AbilityVarianceStateCriterion(dist_ability_est, ability_integrator),\n FixedItemsTerminationCondition(45))\n function get_response(response_idx, response_name)\n params = item_params(item_bank, response_idx)\n println(\"Parameters for next question: $params\")\n prompt_readline(VocabIQ.questions[response_idx])\n end\n function new_response_callback(tracked_responses, terminating)\n if tracked_responses.responses.values[end] > 0\n println(\"Correct\")\n else\n println(\"Wrong\")\n end\n ability = ability_estimator(tracked_responses)\n var = variance_given_mean(ability_integrator,\n dist_ability_est,\n tracked_responses,\n ability)\n println(\"Got ability estimate: $ability ± $var\")\n println(\"\")\n end\n loop_config = CatLoopConfig(rules = rules,\n get_response = get_response,\n new_response_callback = new_response_callback)\n run_cat(loop_config, item_bank)\nend\n\nrun_vocab_iq_cat()","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"","category":"page"},{"location":"examples/examples/vocab_iq/","page":"Running a CAT based based on real response data","title":"Running a CAT based based on real response data","text":"This page was generated using DemoCards.jl and Literate.jl.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"EditURL = \"/home/runner/work/ComputerAdaptiveTesting.jl/ComputerAdaptiveTesting.jl/docs/examples/examples/ability_convergence_mirt.jl\"","category":"page"},{"location":"examples/examples/ability_convergence_mirt/#ability_convergence_mirt","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"","category":"section"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"(Image: Source code) (Image: notebook)","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"How abilities converge on simulated MIRT data","category":"page"},{"location":"examples/examples/ability_convergence_mirt/#Running-a-CAT-based-on-a-synthetic-correct/incorrect-MIRT-model","page":"How abilities converge on simulated MIRT data","title":"Running a CAT based on a synthetic correct/incorrect MIRT model","text":"","category":"section"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"This example shows how to run a CAT based on a synthetic correct/incorrect MIRT model.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"Import order is important. We put ComputerAdaptiveTesting last so we get the extra dependencies.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"using Makie\nimport Pkg\nimport Random\nusing Distributions: Normal, cdf\nusing AlgebraOfGraphics\nusing ComputerAdaptiveTesting\nusing ComputerAdaptiveTesting.Sim: auto_responder\nusing ComputerAdaptiveTesting.NextItemRules: DRuleItemCriterion\nusing ComputerAdaptiveTesting.TerminationConditions: FixedItemsTerminationCondition\nusing ComputerAdaptiveTesting.Aggregators: PriorAbilityEstimator,\n MeanAbilityEstimator, LikelihoodAbilityEstimator\nusing FittedItemBanks\nimport PsychometricsBazaarBase.IntegralCoeffs\nusing PsychometricsBazaarBase.Integrators\nusing PsychometricsBazaarBase.ConstDistributions: normal_scaled_logistic\nusing AdaptiveTestPlots\n\n@automakie()","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"Now we are read to generate our synthetic data using the supplied DummyData module. We generate an item bank with 100 items and fake responses for 3 testees.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"dims = 2\nusing FittedItemBanks.DummyData: dummy_full, std_mv_normal, SimpleItemBankSpec, StdModel4PL\nusing ComputerAdaptiveTesting.Responses: BooleanResponse","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"TODO: pass in dims","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"(item_bank, abilities, responses) = dummy_full(Random.default_rng(42),\n SimpleItemBankSpec(StdModel4PL(), VectorContinuousDomain(), BooleanResponse()),\n dims;\n num_questions = 10,\n num_testees = 2)","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"Simulate a CAT for each testee and record it using CatRecorder. CatRecorder collects information which can be used to draw different types of plots.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"max_questions = 9\nintegrator = CubaIntegrator([-6.0, -6.0], [6.0, 6.0], CubaVegas(); rtol = 1e-2)\nability_estimator = MeanAbilityEstimator(PriorAbilityEstimator(std_mv_normal(dims)),\n integrator)\nrules = CatRules(ability_estimator,\n DRuleItemCriterion(ability_estimator),\n FixedItemsTerminationCondition(max_questions))","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"XXX: We shouldn't need to specify xs here since the distributions are not used – rework","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"points = 3\nxs = repeat(range(-2.5, 2.5, length = points)', dims, 1)\nraw_estimator = LikelihoodAbilityEstimator()\nrecorder = CatRecorder(xs,\n responses,\n integrator,\n raw_estimator,\n ability_estimator,\n abilities)\nfor testee_idx in axes(responses, 2)\n @debug \"Running for testee\" testee_idx\n tracked_responses, θ = run_cat(CatLoopConfig(rules = rules,\n get_response = auto_responder(@view responses[:, testee_idx]),\n new_response_callback = (tracked_responses, terminating) -> recorder(tracked_responses,\n testee_idx,\n terminating)),\n item_bank)\n true_θ = abilities[:, testee_idx]\n abs_err = sum(abs.(θ .- true_θ))\n @info \"convergence\" true_θ θ abs_err\nend","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"Make a plot showing how the estimated value converges during the CAT.","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"conv_lines_fig = ability_convergence_lines(recorder; abilities = abilities)\nconv_lines_fig","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"","category":"page"},{"location":"examples/examples/ability_convergence_mirt/","page":"How abilities converge on simulated MIRT data","title":"How abilities converge on simulated MIRT data","text":"This page was generated using DemoCards.jl and Literate.jl.","category":"page"},{"location":"#Getting-started","page":"Getting started","title":"Getting started","text":"","category":"section"},{"location":"","page":"Getting started","title":"Getting started","text":"This is the documentation for ComputerAdaptiveTesting. For information about the purpose of the package and its installation, please see the README.md.","category":"page"},{"location":"#Contents","page":"Getting started","title":"Contents","text":"","category":"section"},{"location":"","page":"Getting started","title":"Getting started","text":"Creating a CAT: A guide to creating a CAT using the configuration structs in ComputerAdaptiveTesting.jl\nUsing your CAT: A collection of mini topical guides about: simulating CATs, integrating into a user-facing applications, evaluating CATs, visualising CATs, and constructing decision trees from CATs\nExamples: A collection of example code which you can run and modify to get a feel for how to use the package\nAPI reference: The full API reference for the package","category":"page"}] } diff --git a/dev/using_your_cat/index.html b/dev/using_your_cat/index.html index c77c740..2bcfa3b 100644 --- a/dev/using_your_cat/index.html +++ b/dev/using_your_cat/index.html @@ -1,2 +1,2 @@ -Using your CAT · ComputerAdaptiveTesting.jl
GitHub
+Using your CAT · ComputerAdaptiveTesting.jl
GitHub