Study Evaluates AI Tutoring Impact in Tennessee Middle Schools

A two-year cluster randomized trial conducted in 18 Tennessee middle schools examined the impact of the AI tutor Khanmigo, part of Khan Academy, on student math achievement. The study involved randomly assigned students who used Khanmigo during daily remedial mathematics sessions. Results indicated an increase in math achievement by 1.3 national percentile ranks per term, equating to approximately 0.06 to 0.08 standard deviations over a school year. The effect of a full year of active participation was estimated to reach 0.14 standard deviations. These improvements were similar to those observed from Khan Academy practice without AI assistance. However, engagement with the AI tutor was limited; while 96 percent of students tried Khanmigo at least once, the median student interacted with it on only one-third of the days they practiced, and in just 17 percent of sessions where they made mistakes. The findings suggest that increasing student engagement with AI tutoring tools is essential for realizing their potential benefits in education.