Entries by Nathan Thompson, PhD

All Psychometric Models Are Wrong

The British statistician George Box is credited with the quote, “All models are wrong but some are useful.”  As psychometricians, it is important that we never forget this perspective.  We cannot be so haughty as to think that our models actually represent the true underlying phenomena and any data that does not fit nicely is just […]

Want to learn more about adaptive testing? Attend IACAT.

Computerized adaptive testing (CAT) is an incredibly important innovation in the world of assessment.  It’s a psychometric paradigm that applies machine learning principles to personalize millions and millions of assessments, from K12 education to university admissions to professional certification to employment screening to medical surveys.  While invented in the 1970s, primarily at as part of […]

What are cognitive diagnostic models?

Cognitive diagnostic models are an area of psychometric research that has seen substantial growth in the past decade, though the mathematics behind them, dating back to MacReady and Dayton (1977).  The reason that they have been receiving more attention is that in many assessment situations, a simple overall score does not serve our purposes and […]

How do I implement item response theory?

I recently received a email from a researcher that wanted to implement item response theory, but was not sure where to start.  It occurred to me that there are plenty of resources out there which describe IRT but few, if any, that provide guidance for how someone new to the topic could apply IRT.  That is, plenty […]

What is a rubric?

What is a rubric? It’s a rule for converting unstructured responses on an assessment into structured data that we can use psychometrically. Why do we need rubrics? Measurement is a quantitative endeavor.  In psychometrics, we are trying to measure things like knowledge, achievement, aptitude, or skills.  So we need a way to convert qualitative data […]

Identifying and Dealing With Threats To Test Security

Test security is an increasingly important topic. There are several causes, including globalization, technological enhancements, and the move to a gig-based economy driven by credentials. Any organization that sponsors assessments that have any stakes tied to them must be concerned with security, as the greater the stakes, the greater the incentive to cheat. And any […]

What is psychometrics supposed to be doing?

Today I read an article in The Industrial-Organizational Psychologist (the colloquial journal published by the Society for Industrial Organizational Psychology) that really resonated with me. Has Industrial-Organizational Psychology Lost Its Way? -Deniz S. Ones, Robert B. Kaiser, Tomas Chamorro-Premuzic, Cicek Svensson Why?  Because I think a lot of the points they are making are also true […]

ASC Drives Thought Leadership at 2017 Innovations in Testing Conference

I just returned from the 2017 Innovations in Testing conference in Scottsdale, AZ, organized by the Association of Test Publishers.  This is one of my favorite conferences because it contains a compelling blend of ingredients – quality psychometrics, innovative technology, and the business of assessment – and the 2017 edition definitely did not disappoint. This year I was […]