Adaptive testing simulations with CATSim
Designed specifically to help you convert from traditional linear tests to adaptive tests
Improve your assessments by making them adaptive!
Computerized adaptive testing (CAT) is a sophisticated approach for delivering tests, surveys, and other measurements. It is based on complex computer algorithms that adapt the test to each examinee while also controlling for practical issues such as content distribution, item exposure, and test length. CATs have been shown to reduce test length by up to 90% without a loss of precision. However, to achieve such a reduction and make use of the advantages of CAT, it is absolutely essential that the CAT developer perform research studies to simulate the performance of CATs (see Thompson & Weiss, 2011).
Establish validity with adaptive testing simulations
CATSim implements three types of adaptive testing simulations: post-hoc (real-data) simulations, hybrid simulations, and monte-carlo simulations. These should be used in various stages of developing a CAT by documenting validity evidence for CAT options such as starting theta, theta estimation method, item selection method, item exposure control, and termination criterion.
CATSim implements simulations for all three dichotomous item response theory (IRT) models and five polytomous IRT models. CATSim includes all of the CAT options in FastTest, our secure platform for online testing, so that the results of using CATSim can easily be implemented in your testing program.
Using CATSim
CATSim is designed to present options that directly represent specifications that must be made for CAT delivery. CAT requires the specification of 5 components:
- Item bank: import your own, or generate feasible data directly in CATSim.
- Initial theta: fixed, randomized, or read from existing data.
- Three primary item selection algorithms, with subalgorithms representing practical concerns such as content distribution, item exposure, and enemy items.
- Scoring: maximum likelihood, weighted maximum likelihood, or Bayesian (EAP or MAP) estimation.
- Termination criteria, such as target SEM or information.
CATSim Output
CATSim records the adaptive testing simulations with item-by-item detail, but also provides a summary report describing the dependent variables necessary to adequately evaluate a CAT, such as average test length, accuracy, and item exposure.
