CATSim - Computerized Adaptive Testing Simulation

Computerized adaptive testing (CAT) is a sophisticated approach for delivering tests, surveys, and other measurements. It is based on complex computer algorithms that adapt the test to each examinee while also controlling for practical issues such as content distribution, item exposure, and test length. CATs have been shown to reduce test length by up to 90% without a loss of precision. However, to achieve such a reduction and make use of the advantages of CAT, it is absolutely essential that the CAT developer perform research studies to simulate the performance of CATs. CATSim is the only software program that completely fills this role.

For further information about CAT, visit CAT Central or watch a brief video of ASC’s President and Chief Psychometrician, and internationally known CAT pioneer researcher, David J. Weiss, Ph.D., present a brief introduction to CAT. Papers on CAT by ASC psychometricians are also available at the ASC research page and the FastTest Web white papers page.

CATSim implements three types of simulations for computerized adaptive testing (CAT): post-hoc (real-data) simulations, hybrid simulations, and monte-carlo simulations. In implementing a CAT program, all three types of simulation can be used at various stages of the CAT development process. CATSim options allow you to implement all three types of simulations, varying CAT starting thetas, theta estimation methods, item selection methods, item exposure controls, and termination criteria. CATSim will implement simulations for item banks of up to 999 items, with no limit on the number of examinees for both post-hoc and hybrid simulations, and a limit of 10,000 examinees for monte-carlo simulations. However, CAT simulations can be done with as few as 200 examinees or fewer if they adequately represent the population to which the CAT will be applied. CATSim implements simulations for all three dichotomous item response theory (IRT) models and five polytomous IRT models. CATSim includes all of the CAT options in the soon to be released version 3 of FastCAT (formerly the FastTEST Professional Testing System) so that the results of using CATSim can easily be implemented in your testing program.

CATSim was previously known as POSTSIM 2.0. Enhancements in the current version include:

• Addition of monte-carlo and hybrid simulations
• Five polytomous models: graded, generalized rating scale, Rasch rating scale, Rasch partial credit, generalized partial credit
• Constrained CAT options: content balancing, exposure control, enemy items
• Beta family of distributions for monte-carlo simulations
• Expanded termination options
• Multiple termination criteria
• Weighted maximum likelihood theta estimation
• Expanded output options

Using CATSim

CATSim is designed to present options that directly represent specifications that must be made for CAT delivery. CAT requires the specification of 5 components.

Use parameters from your own bank, or in monte-carlo simulation generate a new bank: IRT parameters can be fixed to desired values, read from a file, or randomly generated from a wide variety of distributions

Examinees must be assigned an initial theta when starting a test. This can be fixed, randomized, or read from existing data.

There are three primary item selection algorithms, with subalgorithms representing practical concerns such as content distribution, item exposure, and enemy items.

Examinees can be scored by maximum likelihood, weighted maximum likelihood, or Bayesian (EAP or MAP) estimation.

A number of termination criteria options are presented, allowing you to match the test simulations to your specific needs.

CATSim records the simulations with item-by-item detail, but also provides a summary report describing the dependent variables necessary to adequately evaluate a CAT, such as average test length, accuracy, and item exposure.

System Requirements

Windows ME, 2000, XP, Vista, 7

Free Demo/Student Version and Manual

A free fully-functioning demo/student version is available, limited to 50 examinees and 50 items. This version can be converted to the commercial version by email, fax, or phone after payment arrangements have been completed. Please see Step 1 below for more information.

Licenses

CATSim licenses are determined by the use of the program:

Academic (university research only): $499 one-time

Government: $499 per year, or $998 for 3 years

Commercial or Non-profit: (contact us for price) - Please let us know how you intend to use the software.

Contact us if you qualify for an upgrade. Volume discounts are also available. Contact us for more information.

Purchasing CATSim

CATSim is available via download from the link below. It includes all the same files that would be sent on a physical CD, so CD shipping is not necessary. If you wish, you are free put the ZIP file onto a CD or USB drive as a backup, which saves expensive shipping costs if ASC were to produce a CD and send it to you. In special cases, such as a University that requires a physical CD, please contact us.

CATSim is downloaded as a demonstration version limited to 50 items and 50 examinees. This allows you to get familiar with the program immediately. The program is then "unlocked" to the full version once the payment process is completed. Please follow these steps to purchase CATSim:

1. Download the demonstration version by clicking here (7.7 MB), and install it on your computer. Note: if you have Windows 7 or Vista, you will need to right-click and "Run as administrator."
2. Select the appropriate license/quantity from the lists at the bottom of this page, and click "Add to Cart." Then complete your purchase.
3. After payment is completed (that is, we receive your check/wire or run your credit card), please open the program (again, if you have Windows 7 or Vista, you will need to right-click and "Run as administrator"), and click the "Unlock" button. You will then be presented with two unlock codes. Send those to ASC (contact us with the unlock codes or call 651-647-9220 ), along with your name and Invoice number. We will send back an Activation code that will unlock the program into the full version.

The demonstration version of CATSim is available for evaluation of the program with no cost and no obligation to purchase. While limited to 50 items and 50 examinees, there is no limit on time. This makes it useful as a completely free student version. If you wish to purchase the full version at any time, simply complete Steps 2 and 3 in the list above or contact us for more information.