Read-aloud scoring calibration protocol

Where this applies

Recorded read-alouds are optional and off until a parent turns them on. When they are on, the score shown to adults is words correct per minute. This protocol says when a machine may produce that score and when a person must.

Gates

1. Agreement with human raters. On a held-out set of at least 300 recordings scored independently by two trained people, the machine's words-correct count must be within three words of the human mean on 90 percent of recordings, and the correlation with the human mean must be at least 0.95.
2. No large group differences. The same agreement figures are computed separately by age band, by the main regional and community accents in the sample, by recording device class and by background-noise level. A group whose agreement falls below the overall figure by more than five points fails the gate for everyone until it is fixed.
3. Calibration over time. The held-out set is refreshed each term and the gates re-run. A model update that fails any gate is not deployed.
4. Honest display. A machine score is labeled as such and shown as a check-in next to published grade ranges. It is never shown as a reading age, never as a percentile and never as a diagnosis.

Until the gates pass

The machine does not score. A teacher or parent marks words as read or missed on the device, and that count is what is shown.

Sample for the audit

Recordings come only from children whose families consented to research use of recordings, are kept for the audit no longer than the retention period in the Children's notice, and are never sent to a third party for scoring.

Reporting

The audit's overall and per-group figures are posted on the Evidence page with the date and the model version. A failure is posted too.