Knowledge-component validation report (template)

Purpose

Every skill in TaleTykes is a knowledge component (KC) with a learning curve. When the curve does not fall, the KC, the items or the sequence are wrong, not the child. This template is how a flagged KC is examined and what is decided. Completed reports are kept with the KC and summarized on the Evidence page.

1. The KC

Code, name, strand, unit, prerequisites, the graphemes or words it covers, and the item templates it uses.

2. The flag

Which rule fired (no learning, non-monotone, too easy, prerequisite failing), on what date, over how many children and opportunities. The thresholds: a verdict needs 200 children with a median of 6 opportunities.

3. The curve

Error rate on first attempts by opportunity, with the number of children at each point. Points under 20 children are dropped. Hinted attempts count as errors.

4. The model fit

The Additive Factors Model fit for this KC: initial difficulty, learning rate per opportunity with its interval, convergence. A comparison with Performance Factors Analysis and Bayesian knowledge tracing by student-stratified cross-validation when the question is which model to trust.

5. Item analysis

Error rate per item template and per word. Items that are much harder or easier than the rest, and why: a word the child has not met, a picture that is ambiguous, a distractor that is too close.

6. Prerequisite check

For children who failed this KC repeatedly, the state of its prerequisites. If most had not verified a prerequisite, the sequence or the gate is the problem.

7. Split or merge

Whether the data suggest the KC is really two (a flat curve with a bimodal item pattern) or that two KCs are one (curves that move together). The proposed change and the evidence for it.

8. Retention and transfer

Probe pass rates at 7, 21 and 60 days and the taught-versus-untaught gap on checks, next to the band's typical values.

9. Decision

One of: keep as is, fix the items, change the hint ladder, split the KC, merge with another, move it in the sequence, or add a prerequisite. Who decided and when. What will be watched to confirm the fix.

10. Follow-up

The date the curve is re-read after the change, and the result when it is in. Null results stay in the report.