A cutscore that is not empircally evidenced is not legally defensible. There are several accepted methods for setting a valid cutscore, and one of these is the contrasting groups method. It has the advantage over others in that it can be used in a purely data-driven way, in addition to being used with subject matter expert (SME) ratings.
The goal of the contrasting groups method is to evaluate how test scores predict some sort of “gold standard” in classifying examinees. One example is a practice test. Suppose you are delivering a practice test for a national certification exam. You have 100 people take your test, and you also gather their Pass/Fail results from the national exam. Naturally, people who score higher on your exam are more likely to pass. We want to find the cutscore on your exam that best predicts this.
The Contrasting Groups Tool allows you to paste in the test scores and the “gold standard” and then recommends a cutscore from both a mimimized-error and smoothed-distribution perspective.
This “gold standard” can be a range of other classifications too. Here are some more plausible situations:
- You are developing a pre-employment test, and you have supervisor ratings of whether each person had acceptable job performance
- You are developing an admissions test, and have data on whether each person graduated university within the expected 4 years
- You are developing a clinical survey, and have clinical diagnoses of each person
- You are developing a certification, and have SME ratings on whether each person should be certified (for example, based on a body of work)
We provide this tool at no cost, to help practitioners better establish cutscores. Our mission at Assessment Systems is to improve assessment. If you would like to leverage our expertise to help your organization develop higher-quality assessments, please contact us!