LLM-based educational games will be a big deal

SubscribeSign in

LLM-based educational games will be a big deal<br>For the first time, digital games can make qualitative assessments of learning. Here's what that might look like.

Benjamin Breen<br>May 14, 2024

31

Share

This is a mockup of a non-existent LLM-based educational game called Mexico City Apothecary that I made for this post. But it could be real!<br>For more than a year now, I’ve been experimenting with using ChatGPT (and more recently, Claude and Gemini) to create deeply imperfect — but also, I think, fascinatingly weird and pedagogically useful — “simulations” of historical settings for use in my classes at UC Santa Cruz. The initial simulations were crude proofs of concept, but I have since developed them into a larger set of activities and assignments that I call “HistoryLens.”<br>The basic idea goes like this:<br>The instructor creates a “prompt” which integrates a standard set of rules with a specific setting, event, and/or historical source tailored to the course content.

Working in pairs or alone, students paste this prompt into ChatGPT or an LLM of their choice, then decide how to act within the text-based, interactive scenario it creates based on the specified setting, date, and sources.

Group and class discussions, plus a series of follow-up assignments involving fact checking the simulation and researching questions it raised, allow students to integrate, critique, and reflect on the experience — while also gaining sustained experience with a complex use case of generative AI.

You can read about my early experiments with this setup here:

Simulating History with ChatGPT<br>Benjamin Breen<br>September 12, 2023

Read full story

Recently, I’ve been thinking about what the next steps would look like. Could this sort of thing be the early stage of a whole new category of educational games that will impact how we teach and learn?<br>I think the answer to that question is “yes.”<br>Educational games have been around for a long time, but I believe that the advent of generative AI really does mark something new here. There are two key assumptions I’m making that explain why I’m so excited about the possibilities.<br>Proposal 1: In the near term, education may be the most important field for generative AI<br>Using anything as error-prone as the current generation of consumer-facing LLMs in life-or-death applications like healthcare is downright terrifying. Tools like ChatGPT are simply too prone to hallucinations and prompt injection to be acceptable in situations where truly impactful decisions are being made.<br>But what about in the classroom, where the stakes are lower and confusion can be productive? Discussions of LLMs in education so far have tended to fixate on students using ChatGPT to cheat on assignments. Yet in my own experience, I’ve found them to be amazingly fertile tools for self-directed learning. GPT-4 has helped me learn colloquial expressions in Farsi, write basic Python scripts, game out possible questions for oral history interviews, translate and summarize historical sources, and develop dozens of different personae for my world history students to use for an in-class activity where they role-played as ancient Roman townsfolk.<br>And although all this might sound somewhat niche, keep in mind that there are thousands of classrooms where this sort of thing is happening. Education is a huge and impactful field, one that accounts for 6% of the total GDP of the United States and which represents one of the largest employment sectors globally.<br>Given this, I find it somewhat surprising that media attention so often focuses on the hypothetical uses of AI systems. Sure, future LLMs might develop novel cancer-fighting drugs or automate away millions of jobs. But that’s by no means assured. What is definitely assured is that generative AI has already become interwoven with secondary and postsecondary education. According to one study, over 50% of American university students are using LLMs. That percentage will likely continue growing fast — especially given the rollout, just yesterday, of OpenAI’s free “GPT-4o” model, which is a major advance over the previous free LLMs that were available.<br>In other words, now is the time to find uses for LLMs as tools for learning rather than cheating. That means exploring the new modalities they open up.<br>For example…<br>Proposal 2: Educational games are a major real-world use case for LLMs<br>I already wrote about why I think LLMs are such interesting tools for educational role-playing, so I won’t rehash the argument here. But what would that look like in practice? So far, my classroom experiments with HistoryLens have been mostly about figuring out whether the basic concept of AI-enabled historical simulations even makes sense. Are the AI models capable of pulling it off? Do students like it? Does it augment their understanding of course material, or detract from it?<br>Since these are still largely unanswered questions, I have been...