Methods like the Angoff approach are necessary to be defensible or achieve NCCA/ANSI Accreditation and follow AERA/APA/NCME guidelines
I’M NEW TO THIS. TELL ME MORE.
What is standard setting?
Standard setting is the process of setting a cutscore or passing point on a exam, typically to indicate some sort of mastery. For example, by passing the test you might be considered certified to work in a profession or eligible to interview for a certain job. Obviously, there are important stakes associated with such exams, so it is not defensible to pick an arbitrary score as the cutscore. For example, you can’t just pick 70% because that’s what was considered passing on your 5th grade spelling tests. If you are making a criterion-referenced interpretation of the scores, you need a criterion-referenced method of setting the cutscore.
What is a criterion?
Here, it means some well-defined area of knowledge that is being assessed, such as knowledge of accounting practices or essential aspects of being a nurse. Unfortunately, criterion has another meaning in assessment too; it sometimes refers to an external variable used to provide corroborative evidence for test score interpretations, historically known as criterion-related validity.
How can I set a defensible cutscore?
There are several methods available, including modified-Angoff, Bookmark, Hofstee, Body Of Work, Borderline, and Contrasting Groups. This spreadsheet helps you implement the most common method, modified-Angoff, with or without the common add-on of the Beuk Compromise. It also supports the Hofstee method.
If you are a credentialing organization, using methods like these is a requirement of NCCA and ANSI accreditation.
How can I implement the Angoff method?
A step-by-step overview is described in this blog post. You will need a pool of high-quality items, preferably ones that are already pretested. You will then need a panel of at least 6 subject matter experts (SMEs). Need an enterprise-class item banker platform to implement an Angoff study as part of the item authoring/review process? Sign up for a free account in FastTest.