Custom AI chatbots are transforming course study and review, according to a Harvard study of nearly 200 students. Researchers developed a specialized AI tutor for Physical Sciences 2 (PS2), a physics course for life sciences majors. Students in this pilot program, conducted in fall 2023, experienced a dramatically higher level of engagement than peers who learned the same material in a traditional classroom setting.
The study compared two groups over two weeks. During the first week, one group engaged with the AI tutor at home, while the other group received a standard active learning lecture in class. In the second week, the groups reversed roles. Both pre- and post-tests showed the AI-supported students almost doubled their learning gains compared to those in conventional sessions.
One reason for this success is that the AI system, built on GPT technology, provides step-by-step support and allows students to progress at their own pace. This flexibility helps advanced learners proceed more quickly and offers extra practice for those who need it. Importantly, the design relies on prompt engineering by the research team to ensure the chatbot behaves like an experienced instructor rather than a generic AI.
Another key finding addressed how AI tutors can display unwarranted confidence, giving incorrect answers or misjudging correct responses. However, it appears “a carefully designed AI tutoring system, using the best current GAI technology and deployed appropriately, can not only overcome these challenges but also address significant known issues with pedagogy in an accessible way that can offer world-class education to any community or learning environment with an internet connection.”
Students reported that they felt more motivated and engaged with the material when working with the AI tutor. Researchers suggest that having the chatbot teach basic concepts outside the classroom frees up class time for higher-level tasks like problem-solving and projects.
At Harvard, other large introductory courses—including multivariable calculus—are already piloting versions of this AI tutor framework. While the results are encouraging, the researchers emphasize that AI should currently only complement, not replace, in-person teaching. With careful oversight and clear learning objectives, these chatbot tutors offer a valuable tool for personalizing education and enhancing student success.
Original report:
https://news.harvard.edu/
Original study:
https://www.researchsquare.
