How We Built AI Tutoring Tools

By Kristen DiCerbo, Ph.D., Chief Learning Officer at Khan Academy

Educational technology should always remember that “education” comes first and “technology” comes second. Whenever there’s any new technology, start with understanding the learning problems that we need to solve and then determine whether technology can help solve them.

We know a lot about how students learn but it is often difficult to scale the teaching and tutoring practices to support that learning in large classrooms. Intelligent tutoring applications have been trying to solve this problem for decades but were somewhat limited by the available technology. We believe generative artificial intelligence, the kind of AI that creates high quality text and images, can help fill in the gaps in existing intelligent tutors and scale good student learning practices.  

Active engagement

We know that students need to be actively engaged in the material they’re learning. This doesn’t mean mindlessly doing something, but instead active cognitive engagement. We can see the impact of cognitive engagement on learning in the ICAP framework developed by Micki Chi and Ruth Wylie that shows moving from passive to active to constructive to interactive experiences. Passive learning, for example watching a video or listening to an explanation, leads to some amount of learning. More learning will occur with active learning, such as highlighting or underlining. In each of these cases, the learner is starting to think about the content in some way (e.g., “is this important enough to highlight?). Even more learning happens when learners are making or constructing responses, for example solving problems or providing explanations in their own words. Finally, research suggests even more learning happens when there is interaction between people around content, for example explaining to someone else or having a debate.

Getting all students in a classroom cognitively engaged in learning is difficult. Certainly we can ask students to solve problems and work in pairs or small groups. However, it is easy for struggling students to not engage. With technology like Khan Academy, every student in a classroom has been able to work constructively, solving problems and answering questions. The addition of Khanmigo allows for interaction that was not possible before. Students can debate Khanmigo, ask questions of simulated historical figures, and co-construct explanations for math problems. 

Providing support for learning

Students learn when they’re working on material that’s at the edge of what they know. If the content is too easy they get bored and if it is too challenging, it is just frustrating and they give up. They need to be in that area where it’s a little bit challenging and maybe they need a little bit of support to be successful. Education technology has historically worked to help learners find the content at the right spot for learning. However, providing support for students working at a number of different levels is challenging at best. All the students who we are trying to actively cognitively engage will not be able to if they are just presented problem after problem without support for solving them. We have all been in classrooms where students are meant to be working independently but students are stuck and sitting with their hands raised to get help while the teachers is moving as quickly as possible to help individual students. If students are working on different skills and pre-requisite skills, it becomes more difficult to solve this with peer help.

Khanmigo can help students get unstuck at these places where students need just a bit of support to be successful on a problem that is at the edge of their abilities. We have written the prompt that tells the model how to respond such that it will ask questions to try to lead the student through what they know about the problem. It will give support and potentially a step to take, but not the answer itself. 

Immediate feedback 

We know that immediate feedback is important for learning, particularly when we are learning something new. Even before Khanmigo, a major benefit of Khan Academy and other tutors was the ability to provide immediate feedback about whether an answer was correct or incorrect and why. This has been a major improvement over requiring a teacher to do this and handing back scored worksheets days later after the student had completely forgotten what they were working on. 

However, this feedback could go further and provide feedback on the steps students are taking so they know where they go off track. This kind of feedback was difficult to design and program with the technology we have had. Now, the generative AI that powers Khanmigo makes this kind of step-level feedback possible. 

Building on research

We should use what we know about learning and human tutoring, based on decades of research to guide us in designing the applications of new technologies in education. There is substantial research on dialogue-based tutors. Not only can we use it to build prompts, but the generative AI can tell you about it if you ask. Drawing a thread from what we know about how people learn to how we design educational technology is the only way those technologies are likely to actually impact learning outcomes.

To learn more about bringing Khanmigo to your students, visit https://districts.khanacademy.org/khanmigo