Entries by Nathan Thompson, PhD

What is the Hofstee method for setting cutscores?

Have you heard about standard setting approaches such as the Hofstee method, or perhaps the Angoff, Ebel, Nedelsky, or Bookmark methods?  There are certainly various ways to set a defensible cutscore or a professional credentialing or pre-employment test.  Today, we are going to discuss the Hofstee method. Why Standard Setting? Certification organizations that care about […]

What is the Spearman-Brown prediction formula?

The Spearman-Brown Prediction Formula, also known as the Spearman-Brown Prophecy Formula or Correction, is a method used in evaluating test reliability.  It is based on the idea that split-half reliability has better assumptions than coefficient alpha, but only estimates reliability for a half-length test, so we need to implement a correction that steps it up […]

What can artificial intelligence and machine learning tell us about item banks?

Artificial intelligence (AI) and machine learning (ML) have become buzzwords over the past few years.  As I already wrote about, they are actually old news in the field of psychometrics.   Factor analysis is a classical example of ML, and item response theory also qualifies as ML.  Computerized adaptive testing is actually an application of AI […]

What is monte carlo simulation in adaptive testing?

Simulation studies are an essential step in the development of an computerized adaptive test (CAT) that is defensible and meets the needs of your organization or other stakeholders.  There are three types of simulations: monte carlo, real data (post hoc), and hybrid.  Monte carlo simulation is the most general-purpose approach, and the one most often […]

What is decision consistency?

If you are involved with certification testing and are accredited by the National Commission of Certifying Agencies (NCCA), you have come across the term decision consistency.  NCCA requires you to submit a report of 11 important statistics each year, each for all active test forms.  These 11 provide a high level summary of the psychometric […]

The Standard Error of Measurement

The standard error of measurement is one of the core concepts in psychometrics.  One of the primary assumptions of any assessment is that it is accurately and consistently measuring whatever it is we want to measure.  We therefore need to demonstrate that it is doing so.  There are a number of ways of quantifying this, […]