Exam Reliability Study
Independent assessment reliability research

PSIA Level 2 fail rate

“Fail rate” is a common search phrase, but it is a poor analytical endpoint by itself. A Level 2 assessment often involves multiple modules, each with its own rubric, scoreable rows, and section-average thresholds.

Why a single fail rate can mislead

Two candidates can both be reported as unsuccessful while failing for very different reasons. One may miss the standard in Movement Analysis, while another may struggle in Teaching or Skiing Performance. Without module-level separation, the number alone does not explain what actually happened.

What should be compared instead

A better comparison looks at attempt number, module, section averages, item-level scores, conditions, and retake outcomes. That is especially important when candidates later pass under different circumstances or with a different examiner group.

How this project approaches Level 2 data

This project separates Level 2 submissions by module and season, and stores individual rubric rows when they are available. That makes it possible to evaluate whether certain modules or score categories cluster more heavily around borderline outcomes.