Computerized adaptive testing (CAT) is a powerful paradigm for delivering tests that are smarter, faster, and fairer than the traditional linear approach. However, CAT is not without its challenges. One is that it is a greedy algorithm which always selects your best items from the pool if it can. The way that CAT researchers address […]
About Nathan Thompson, PhD
I am a psychometrician, software developer, author, and researcher, currently serving as Chief Product Officer for Assessment Systems Corporation (ASC). My mission is to elevate the profession of psychometrics by using software to automate the menial stuff like job analysis and Angoff studies, so we can focus on more innovative work. My core goal is to improve assessment throughout the world.
I was originally trained as a psychometrician, doing an undergrad at Luther College in Math/Psych/Latin and then a PhD in Psychometrics at the University of Minnesota. I then worked multiple roles in the testing industry, including item writer, test development manager, essay test marker, consulting psychometrician, software developer, project manager, and business leader.
Research and innovation are incredibly important to me. In addition to my own research, I am cofounder and Membership Director at the International Association for Computerized Adaptive Testing, You can often find me at other important conferences like ATP, ICE, CLEAR, and NCME. I've published many papers and presentations, and my favorite remains http://pareonline.net/getvn.asp?v=16&n=1.
Entries by Nathan Thompson, PhD
Since the first tests were developed 2000 years ago for entry into civil service of Imperial China, test security has been a concern. The reason is quite straightforward: most threats to test security are also threats to validity, and the decisions we make with test scores could therefore be invalid, or at least suboptimal. It is […]
The standard error of measurement is one of the core concepts in psychometrics. One of the primary assumptions of any assessment is that it is accurately and consistently measuring whatever it is we want to measure. We therefore need to demonstrate that it is doing so. There are a number of ways of quantifying this, […]
The standard error of the mean is one of the three main standard errors in psychometrics and psychology. Its purpose is to help conceptualize the error in estimating the mean of some population based on a sample. The SEM is a well-known concept from the general field of statistics, used in an untold number of […]
One of my graduate school mentors once said in class that there are three standard errors that everyone in the assessment or I/I Psych field needs to know: mean, error, and estimate. They are quite distinct in concept and application, but easily confused by someone with minimal training. I’ve personally seen the standard error of […]
One of the best aspects of my position is the opportunity to travel the world and talk with many experts about psychometrics and educational assessment. In December 2017, I was lucky enough to travel to Monterrey, Mexico, with the dual purpose of a conference and a Psychometrics Seminar. It was an exciting week that taught […]
Validity, in its modern conceptualization, refers to evidence that supports our intended interpretations of test scores (see Chapter 1 of APA/AERA/NCME Standards for full treatment). Validity threats are issues that issues that hinder the interpretations and use of scores. The word “interpretation” is key because test scores can be interpreted in different ways, including ways […]
One of the core concepts in psychometrics is item difficulty. This refers to the probability that examinees will get the item correct for educational/cognitive assessments or respond in the keyed direction with psychological/survey assessments (more on that later). Difficulty is important for evaluating the characteristics of an item and whether it should continue to be part of […]
It’s October 30, 2017, and collusion is all over the news today… but I want to talk about a different kind of collusion. That is, non-independent test taking. In the field of psychometric forensics, examinee collusion refers to cases where an examinee takes a test with some sort of external help in obtaining the correct […]
In the past decade, terms like machine learning, artificial intelligence, and data mining are becoming greater buzzwords as computing power, APIs, and the massively increased availability of data enable new technologies like self-driving cars. However, we’ve been using methodologies like machine learning in psychometrics for decades. So much of the hype is just hype. So, what […]