Proof-of-teaching pilot protocol

Question

Do kindergarten and first-grade children who use TaleTykes four times a week for twelve weeks read more words correctly per minute, and decode more untaught words, than matched children who do not yet have access?

Design

A waitlist comparison in partner schools. Classes are randomized to start in the fall or in the spring. Children in spring-start classes are the comparison group during the fall and receive the program afterward. Nobody is denied the program; the question is only when it starts.

Sample

About 240 children in 12 classes (6 per arm). With this size, clustering by class and a fall baseline as a covariate, the smallest effect the study can reliably find is a little over a quarter of the spread of scores between children on the primary outcome (0.27 in standardized units). Smaller real effects may go unfound; the report will say so rather than call them absent.

Consent

A school agreement that covers research use, plus a notice to families with an opt-out. A child whose family opts out still uses the program; their data are left out of the analysis.

Measures

• Primary: words correct per minute on grade-level passages, administered and scored by trained assessors who do not work for TaleTykes and do not know the arm.
• Secondary: a standard untaught-word decoding measure, given the same way.
• Fidelity: lesson days per week, lessons taught in order and alerts handled, from the product's own logs (the fidelity index).
• Usage: minutes and opportunities per week.

The product's own numbers (verified skills, learning curves) are reported as practice measures and are not an outcome.

Analysis

A mixed model for the spring score with the fall score as a covariate, arm as the effect of interest and a random effect for class. Missing outcomes are handled by multiple imputation under a stated assumption. Fidelity is reported, and an analysis by fidelity is labeled exploratory.

Success and failure, written in advance

• Success: the primary outcome favors the fall-start arm with an interval that excludes zero.
• Failure: the interval includes zero or favors the comparison arm. Either way the result is posted, and the product changes are listed.

Timeline

Baseline in September, twelve weeks of use, outcome assessment in December, spring start for the comparison classes in January, report by March.

Governance

Pre-registered on a public registry before baseline. Reviewed by an advisory board that is being seated. The de-identified data are available to the board under the export rules on the Evidence page.