What is a rubric? It’s a rule for converting unstructured responses on an assessment into structured data that we can use psychometrically.
Why do we need rubrics?
Measurement is a quantitative endeavor. In psychometrics, we are trying to measure things like knowledge, achievement, aptitude, or skills. So we need a way to convert qualitative data into quantitative data. We can still keep the qualitative data on hand for certain uses, but typically need the quantitative data for the primary use. For example, writing essays in school will need to be converted to a score, but the teacher might also want to talk to the student to provide a learning opportunity.
How many rubrics do I need?
In some cases, a single rubric will suffice. This is typical in mathematics, where the goal is a single correct answer. In writing, the goal is often more complex. You might be assessing writing and argumentative ability at the same time you are assessing language skills. For example, you might have rubrics for spelling, grammar, paragraph structure, and argument structure – all on the same essay.
Spelling rubric for an essay
|0||Essay contains 5 or more spelling mistakes|
|1||Essay contains 1 to 4 spelling mistakes|
|2||Essay does not contain any spelling mistakes|
Argument rubric for an essay
“Your school is considering the elimination of organized sports. Write an essay to provide to the School Board that provides 3 reasons to keep sports, with a supporting explanation for each.”
|0||Student does not include any reasons with explanation (includes providing 3 reasons but no explanations)|
|1||Student provides 1 reason with a clear explanation|
|2||Student provides 2 reasons with clear explanations|
|3||Student provides 3 reasons with clear explanations|
Answer rubric for math
|0||Student provides no response or a response that does not indicate understanding of the problem.|
|1||Student provides a response that indicates understanding of the problem, but does not arrive at correct answer OR provides the correct answer but no supporting work.|
|2||Student provides a response with the correct answer and supporting work that explains the process.|
How do I score tests with rubrics?
Well, the traditional approach is to just take the integers supplied by the rubric and add them to the number-correct score. This is consistent with classical test theory, and therefore fits with conventional statistics such as coefficient alpha for reliability and Pearson correlation for discrimination. However, the modern paradigm of assessment is item response theory, which analyzes the rubric data much more deeply and applies advanced mathematical modeling like the generalized partial credit model (Muraki, 1992; resources on that here and here).
How can I efficiently implement rubrics?
It is much easier to implement rubrics if your online assessment platform supports them in an online marking module, especially if the platform already has integrated psychometrics like the generalized partial credit model. Check out this blog post to learn more.
Latest posts by Nathan Thompson, PhD (see all)
- All Psychometric Models Are Wrong - August 8, 2017
- Want to learn more about adaptive testing? Attend IACAT. - July 18, 2017
- What are cognitive diagnostic models? - June 28, 2017