Khan Academy’s Approach to Prompt Engineering for Khanmigo

posted on October 23, 2023

By Daniel de Angulo, Product Manager, Learner Experience at Khan Academy

Khan Academy’s new AI-powered tutor and teaching assistant, Khanmigo, is a groundbreaking platform that provides high-quality, on-demand tutoring to help kids catch up, review, or accelerate ahead. But how do we teach an AI tool to educate like a human? Enter prompt engineering—the art and science of crafting precise inputs to guide AI responses. In this deep dive, we’ll explore our seven-step approach to crafting prompt engineering.

1. Understanding the ideal tutor-student relationship

One-on-one tutoring has always been the gold standard in education. In 1984, Benjamin Bloom found that personal tutoring could significantly boost a student’s performance, but he called it the “2 Sigma Problem” because it has always been too costly to offer this level of attention to everyone. At Khan Academy, we believe that AI can bridge that gap. With tools like Khanmigo, we aim to bring personalized learning to students everywhere. Our approach to prompt engineering begins with really understanding the tutor-to-student relationship and using it to accelerate learning. We envision the perfect tutor-to-student interaction. What would make it magical? What would make it effective? We’ve all had that one great tutor, teacher, or mentor where everything just clicked. We asked, “How can we generate this experience through Khanmigo?” 

2. Consulting learning science literature

Next, we delve into learning science literature to ensure our prompts align with proven educational methodologies. Every decision is informed by learning science research. 

For example, we know that effective tutors meet students where they are, tie in students’ personal interests with instructional material, provide immediate feedback, and prompt students towards self explanation. So, we built Khanmigo to model these principles of a great tutor. Our goal is for Khanmigo to keep students at their “Goldilocks” learning edge, pushing students at any level to clear up misconceptions and engage in active learning.

3. Integrating information from the Khan Academy library

Personalization is key. One of the benefits of using Khanmigo over other AI-powered chatbots is that it is completely integrated into Khan Academy’s world-class content library and connected to an individual learner’s account. This gives us the unique ability to pull in different key insights, such as the courses a student is taking, their preferred language, or even their personal interests. We are then able to tailor the tutoring experience to fit each individual, much like an effective teacher, tutor, or mentor would do.

4. Crafting the perfect interaction with prompt engineering 

As anyone who has been in a classroom knows, it’s not just about the content covered, it’s how the teacher or tutor delivers it. In our prompts for Khanmigo, we design the tone, personality, and type of questions the tutor asks to make learning engaging. We want Khanmigo to be an encouraging and fun learning partner but have the ability to redirect students who are going off topic or are getting distracted. We have also built in moderation guardrails that will stop inappropriate conversations and immediately inform parents and teachers about these interactions.

5. Testing with diverse user personas

One size doesn’t fit all. We run our prompts through a gamut of user personas to ensure every student, regardless of skill level or subject, feels the magic. For example, before Khanmigo’s launch, a group of pilot users reported that while Khanmigo’s emojis were mostly delightful, they felt inappropriate and even offensive when used in contexts such as nuclear war. We rewrote our prompts to ensure that Khanmigo will not use emojis when the topics under discussion are serious or sad in nature.

6. Prioritizing safety and security

At Khan Academy, our prompt engineering is deeply rooted in a responsible approach to AI development. We’re committed to harnessing AI’s potential in education and balancing its transformative benefits with its inherent risks. To ensure our AI-driven prompts serve learners effectively, we’ve established clear guidelines to ensure each prompt does only what we intend it to do. We’ve adopted best practices from institutions like NIST; we’ve fine-tuned and tailored our prompt engineering with an aim to optimize AI accuracy; and we’ve used a controlled testing environment for user testing and feedback. Ethical considerations are at the forefront of our work, guiding us as we strive to enrich learning experiences and share our journey with the global education community. Read more about our approach to responsible AI development.

7. Iterating based on real user feedback

Once we release updated prompts, we seek out feedback, be it praise or critique. Every piece of feedback is an opportunity to refine and perfect. For example, dozens of users reported feeling that Khanmigo repeated itself too much. This took up valuable time and made learners feel less efficient while they were trying to learn. In response, we rewrote some of the activity prompts to ensure that Khanmigo does not repeat questions while quizzing and that Khanmigo uses economy of language to keep its phrases more to the point.

Try Khanmigo today & see our prompt engineering in action

Not only is our prompt engineering process focused on providing the best tutoring experience, but Khanmigo is also incorporated with Khan Academy’s best-in-class content library of thousands of educational articles, videos, and exercises across math, humanities, social studies, and more. Khanmigo was created by Khan Academy, a trusted resource for millions of families around the world who are looking for academic help for their children. 

If you’re looking for personalized homework help for your kids whenever you need it, and you want an AI tool that features guardrails that give kids a safer introduction to AI, check out Khanmigo, which is available for $9/month or $99/year.