PocketTTS-RAVEN | Faster-than-realtime text-to-voice with voice cloning

POCKET·TTS

Summoning…

JavaScript required<br>This demo runs PocketTTS locally in your browser with WebAssembly, so JavaScript must be enabled.

Browser not supported<br>This demo needs WebAssembly threads. Try a current Chrome, Edge, Brave, Firefox, or Safari with cross-origin isolation enabled.

✕ crypt<br>☠ VOICES CLONED LOCALLY. YOUR AUDIO NEVER LEAVES YOUR MACHINE. NO GPU REQUIRED. ☠ ☠ VOICES CLONED LOCALLY. YOUR AUDIO NEVER LEAVES YOUR MACHINE. NO GPU REQUIRED. ☠ ☠ VOICES CLONED LOCALLY. YOUR AUDIO NEVER LEAVES YOUR MACHINE. NO GPU REQUIRED. ☠ ☠ VOICES CLONED LOCALLY. YOUR AUDIO NEVER LEAVES YOUR MACHINE. NO GPU REQUIRED. ☠

local inference · ▓ no cloud séance

Faster-than-realtime text-to-voice with voice cloning.

Native C++ and browser WASM builds, running locally on CPU.

1 Pick voice<br>2 Write line<br>3 Make audio

Nevermore.exe

☠ VOICE_CRYPT.EXE_□×

“Prophet” said I-“thing of evil. Prophet still if bird or devil.<br>By that Heaven that bends above us, by that God we both adore.<br>Tell this soul with sorrow laden, if within the distant Aiden,<br>it shall clasp a sainted maiden whom the angels name Lenore.

Clasp a rare and radiant maiden, whom the angels name Lenore.”

Quoth the Raven...<br>“Nevermore.”

“Be that word our sign of parting, bird or fiend” I shrieked upstarting-<br>“Get thee back into the tempest and the Night’s Plutonian shore.<br>Leave no black plume as a token of that lie thy soul hath spoken.<br>Leave my loneliness unbroken. Quit the bust above my door.<br>Take thy beak from out my heart, and take thy form from off my door.”

Quoth the Raven...<br>“Nevermore.”

And the Raven, never flitting, still is sitting, still is sitting,<br>on the pallid bust of Pallas, just above my chamber door.<br>And his eyes have all the seeming of a demon’s that is dreaming.<br>And the lamplight over him streaming, throws his shadow on the floor.<br>And my soul from out that shadow, lies floating on the floor.<br>Shall be lifted...

Nevermore...<br>☾ turn off silent mode and up your volume :)

Temperature

0.50<br>higher = more expressive, less predictable

Speak

Loading engine…

Voice<br>Replay<br>WAV<br>MP3<br>Edit in AudioMass

Your audio materializes here as it&rsquo;s generated

═══════✥═══ ☠ ═══✥═══════

README_4U.TXT — NOTEPAD_□×

UNDER THE HOOD - SEE IT ON GITHUB

HOW TO USE

Type your text.

Press SPEAK to listen.

To make a voice:

Upload an MP3 file, ideally around 6-15 seconds of audio with no music, no background noise, just voice, or use the microphone.

Press "Clone this voice".

WHY DOES THIS THING EXIST?

In the past few weeks I have been working on a realtime conversational AI system.<br>The goal is to have a "creature" that can follow complex multi-step instructions,<br>carry a complex internal state, interact with you and the world, and<br>connect. A disembodied robot of sorts.

Such an undertaking requires low-latency speech, ideally response times below<br>400ms. If it waits too long, the illusion falls apart.

You can see (and hear) a very very early WIP test of trying to lower latency in a way<br>that feels conversational (active listening - no press to speak, interruption,<br>turn-repair etc) below...

Your browser does not support embedded video.

Along the way there were a lot of optimizations, detours, and dead ends. Some<br>of the most useful work landed in STT and TTS (it&rsquo;s hard enough to make LLMs<br>run fast enough so any ms we can chop off elsewhere counts). I decided to start open-sourcing<br>pieces of the system before unveiling the companion itself. Today I am<br>releasing one of these pieces :)

Making PocketTTS fast enough

One of those pieces is a PocketTTS fork. The original model comes from<br>Kyutai and was<br>released as a Python/PyTorch implementation. That version is already<br>impressive, but Python is not where I wanted the real-time game loop to wait<br>for audio. VolgaGerm&rsquo;s PocketTTS.cpp<br>port moved the model to ONNX Runtime and C++, and was already fast enough to<br>be exciting, but I noticed there was a lot of leading silence muddying the final experience.

I wanted to see how far I could trim without changing voice quality. On my<br>MacBook M4 Max, this optimized native version reaches roughly 32-33x<br>realtime , with a first usable audio chunk around 30ms<br>on this benchmark text, all on the CPU (3 ar and 2 decode threads).

(own local benchmarks, warm)

StageWhat changedResult<br>Python Pocket TTSOriginal Python/PyTorch model8.7x realtime, 210ms first useful audio<br>PocketTTS.cppClean ONNX Runtime + C++ port22.4x realtime, 120ms first useful audio<br>Optimized native path (ours) Graph rewrites, cache deltas, fused decoder work33.3x realtime, 30ms first useful audio<br>Browser build (ours) WASM SIMD, threads, streaming playback, voice cloningup to ~14x realtime on M4 Max, ~4x on iPhone 16 Pro

Native numbers are warm medians on a MacBook M4 Max using the page text and the Alba voice (3 threads). On different systems performance might vary.

TTS generations often start with a little silence or...