Making Audio Stories Interactive with AI: The Story of Schmaudio – Dennis Weyland's Blog

Skip to content

Search for:

Recent Posts<br>Recent Comments<br>Archives<br>Categories<br>Meta

I have a somewhat complicated relationship with AI. As a software engineer, I find the technology fascinating. I use it regularly, and it has become a genuinely useful tool in my work. At the same time, I am cautious about some of the enthusiasm surrounding it. I don’t think we should automate everything simply because we can, and I am skeptical of the idea that handing a task to an AI and accepting whatever comes back necessarily represents progress. There are also still many open questions around how this technology will affect creative work, software development, and society more broadly.

But there is another side to AI that I find much more exciting. Sometimes a new technology doesn’t merely make an existing process faster or cheaper. It makes something practical that previously wasn’t. For me, interactive audio stories are an example of that. And somewhat unexpectedly, they connect modern AI with an idea that fascinated me when I was a child.

Choosing What Happens Next

As a child, I was fascinated by pen-and-paper role-playing games. I also loved interactive adventure books. Instead of simply reading from the first page to the last, you would eventually have to make a decision. Perhaps you could enter a mysterious cave or continue along the path through the forest. If you entered the cave, the book might tell you to continue on page 74. If you chose the forest, you might jump to page 112. The story changed depending on what you decided.

I loved the idea that I wasn’t simply waiting to find out what the character would do next. I had to make that decision myself. Of course, those books had an unavoidable limitation: every possible path had to be written beforehand. And branching grows very quickly. If every episode of a story leads to 3 completely different episodes, those 3 lead to 9, then 27, then 81. Continue for long enough and the number becomes absurd. A human author cannot write millions of episodes for a single story.

Traditional interactive stories therefore have to work within that constraint. Branches can eventually reconnect, some decisions can have relatively small consequences, and the overall number of paths has to remain manageable. That doesn’t prevent great interactive stories from being written, but it puts a practical limit on just how far the idea can go.

Generative AI changes that constraint.

From My Childhood to My Children

Many years later, audio stories became part of my own children’s lives. They listen to them regularly, and at some point this brought me back to the interactive stories I had enjoyed as a child. I started wondering what would happen if you combined the two ideas. What if an audio story stopped at an important moment and asked the listener what should happen next? And what if that decision didn’t simply cause a small variation before returning to the same story, but could actually send the adventure in a different direction?

Modern generative AI suddenly made that idea practical, so I started experimenting. There wasn’t a business plan behind it. I simply wanted to create interactive audio adventures for my children and see whether the idea worked. A story would play for several minutes. Eventually, the characters would reach a situation requiring a decision. The listener would choose what they should do, and the next episode would continue from that decision. Then another choice would appear, and another.

The more I experimented with it, the more I liked the idea. Eventually, that experiment became Schmaudio .

AI Isn’t the Interesting Part

It would be easy to describe Schmaudio as a platform for AI-generated audio stories. Technically, that’s true, but I don’t think that’s the interesting part. Ideally, when someone listens to a Schmaudio story, they shouldn’t be thinking about AI at all. They should be thinking about the story.

The interesting part is what AI makes possible. A listener can make a decision, and that decision becomes part of the history of their adventure. Later episodes can build on what happened before. Characters can have experienced different things, relationships can develop differently, and one path might reveal places or characters that another listener never encounters. Over time, two people who started with exactly the same first episode can experience increasingly different stories.

And when someone reaches the end, there is another interesting possibility:

What would have happened if I had chosen differently?

They can go back and find out.

A Story That Grows as You Explore It

There is still an obvious practical problem. Even if AI makes it possible to generate all those episodes, it wouldn’t make much sense to generate an enormous story tree before anyone listens to it. Most of those episodes might never be heard.

Schmaudio therefore...