Exam Reliability Study
Independent assessment reliability research

PSIA assessment consistency

Assessment consistency asks a simple question: are comparable performances being scored in comparable ways across time, modules, and evaluator groups?

Consistency is not sameness

Not every group, condition, or task will be identical, and certification is never fully mechanical. But consistency still requires that variation stay within a defensible range. If outcomes swing sharply because of examiner differences, rubric interpretation, or module framing, that is a reliability concern.

Variables worth tracking

This project tracks season, discipline, level, module, group size, conditions, rubric version, score source, and optional form context such as highlighted fundamentals and assessment activities performed. Those details help separate true performance differences from process differences.

Why candidates care

Candidates need to know whether a retake reflects improved performance, improved clarity, different scoring emphasis, or some combination of the three. Consistency data helps answer that question more honestly.