Ai2 releases TutorMoments, a replay evaluation of whether AI tutors know when to hold back

Introduced 7 August 2026, TutorMoments is a replay-based evaluation built from real one-on-one maths tutoring transcripts: experienced teachers flag the moments where a tutor must choose between easing the problem and pushing the student to do the reasoning, and the transcript up to that point is handed to a language model that takes over as tutor, with another model playing the student. Told only to "tutor well", models tended to over-help and rarely pushed for deeper thinking; spelling the trade-off out in the prompt improved matters without closing the gap to human tutors, and models varied widely in how reliably they made the call. Ai2 released de-identified transcripts, the replay pipeline code and the model replays.

What it means It puts a measurable number on productive struggle — the failure mode where a helpful assistant quietly does the learning for the student — and hands over the data to test your own tutor against.