Narrating your blog with Kokoro, a local and open source text-to-speech model

bartdegoede1 pts0 comments

Narrating your blog with Kokoro, a local and opensource text-to-speech model | Bart de GoedeListen to this article instead

0:00 / 0:00<br>0.75×1×1.25×1.5×2×<br>Your browser does not support the audio element<br>Seven years ago I used Google&rsquo;s text-to-speech API to create audio versions of my posts. There was a Python script that parsed the Markdown file, cleaned it up, chunked it to stay under Google&rsquo;s 5000 byte limit, convert each chunk to an MP3, stitch them together, and finally embed the resulting file with a Hugo shortcode that is essentiall a fancy tag. It worked, it was free1, and I was pretty pleased with it. The little audio player has been sitting at the top of my posts ever since.<br>However, it does sound a little like 2010. Every time I hit play I hear the flat, slightly seasick cadence of a Wavenet voice, and it needs a Google Cloud project, an API key, and a billing account to regenerate2. Now that the blog does fancy semantic search in the browser from a 4 MB lookup table, the narration should not be the most dated thing on the page, and text-to-speech models have come really far recently.<br>The new audio is generated by Kokoro, an 82-million-parameter open weight (Apache 2.0) model that runs comfortably on my M1 MacBook Air. No cloud, no lost keys, no per-character bill should I ever go over a million characters. It also sounds dramatically better.<br>The 2010 sound of 2019#<br>Here is the main argument for doing this at all. This is a paragraph of that 2019 post, read by Google&rsquo;s Wavenet, exactly as it has sounded on the live site since I published it:<br>Listen to this article instead

0:00 / 0:00<br>0.75×1×1.25×1.5×2×<br>Your browser does not support the audio element<br>And here is the same post, re-narrated today by Kokoro running locally on my laptop:<br>Listen to this article instead

0:00 / 0:00<br>0.75×1×1.25×1.5×2×<br>Your browser does not support the audio element<br>I also added a script that samples the audio and creates an actual image of the shape of the soundwave; look at the difference between the nasal Wavenet voice and the Kokoro one. It puts stress in roughly the right places, pauses at clause boundaries instead of on a fixed timer, and doesn&rsquo;t have that metallic edge.<br>It is not going to fool anyone into thinking that it&rsquo;s voiced by a real human, but I do think it crossed the line from &ldquo;accessibility feature I feel slightly guilty about&rdquo; to &ldquo;thing I would actually listen to.&rdquo;<br>What changed: Kokoro and MLX#<br>Kokoro-82M is a small, Apache-2.0-licensed text-to-speech model that punches absurdly above its weight; it spent a good while at the top of the TTS Arena leaderboard for open models, beating things many times its size. The 82 million parameters matter: the weights are a few hundred megabytes, not the tens of gigabytes a large speech model requires. And it was trained mostly on long-form reading, which is exactly the job here. It is a narrator, not a chatbot voice.<br>The thing that makes it practical on a my laptop is MLX-Audio, which runs Kokoro on Apple&rsquo;s MLX framework; natively on the Apple Silicon GPU, no PyTorch, no CUDA, no ONNX runtime. The whole &ldquo;install&rdquo; is:<br>pip install mlx-audio

and then, in Python, the entire synthesis step is:<br>from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/Kokoro-82M-bf16")<br>for result in model.generate(text="Hello there.", voice="af_heart", speed=1.0, lang_code="a"):<br>audio = result.audio # 24 kHz float32 samples

Kokoro ships with 54 voices; I settled on af_heart for narration3.<br>Faster than real time#<br>The internet will tell you Kokoro runs at &ldquo;15x real time and up&rdquo; on Apple Silicon4. When I reprocessed all my posts (14 total), which is about two full hours of audio , it took 922 seconds , or a little over fifteen minutes. That is 7.8x real time on my M1, measured end to end including all the Markdown parsing and MP3 encoding.<br>7.8x is not 15x, but this is more than enough for my use case. I like my little fanless laptop that could, and it&rsquo;s still pretty wild to me that it did it no problem.<br>Dealing with jargon#<br>There _is) one place Kokoro reliably falls on its face, and it is a place a blog like mine steps on constantly. Kokoro converts text to phonemes with espeak-ng, and espeak-ng has never heard of some of the words in my vocabulary. int8 comes out as &ldquo;int eight&rdquo; if you are lucky and something worse if you are not. RRF gets read as a word. Goldmark, the Markdown renderer Hugo uses, becomes something with a hard g that isn&rsquo;t a name anyone would recognize. Kubernetes is also hilarious.<br>The fix is boring and effective: there&rsquo;s a yaml file that maps certain words to more phonetically pleasing spellings:<br># scripts/tts_lexicon.yaml — surface form -> spoken form<br>int8: "int eight"<br>RRF: "R R F"<br>Goldmark: "Gold mark"<br>espeak-ng: "e speak N G"<br>WordPiece: "word piece"

Matching is case-sensitive, whole-token, and longest-match-first, so int8 wins over...

kokoro audio rsquo text model speech

Related Articles