Domain Customization - Moonshine Voice
Skip to content
Initializing search
moonshine-ai/moonshine
Models
Retraining
Quantization
HuggingFace
API Reference
Support
Roadmap
Acknowledgements
License
Retraining
Domain Customization¶
Transcription gets easier when you know something about what is about to be said, and usually you do: an application knows its own jargon, its product names, and the names in the user's contacts. There are two ways to tell the model what to expect. Runtime context is a list of key terms, or a passage of text to find them in, applied with no training step, which is what the rest of this section is about, and it works on words the model can already spell. Teaching it a new accent, dialect or acoustic environment needs retraining instead.
Runtime Context<br>Supply a Context
Supply a Key Terms List
How it works
Tuning the strength
Measuring it on your own data
What a long list costs
What it costs in time
What it can't do
Retraining<br>Worked example: air traffic control
Your own data
Shipping the result
Pitfalls
Runtime Context¶
The most straightforward way to improve a model's accuracy for particular names or phrases for the application to supply hints. You can achieve up to a 40% reduction in errors with no latency cost and only a very small impact on general accuracy.
Supply a Context¶
Often you have context without having a list. The user is dictating into a document, or looking at a ticket, or halfway through a thread, and the words worth listening for are already on screen — you just have not enumerated them. Hand over the text and they will be found for you:
from moonshine_voice import Transcriber, ModelArch
transcriber = Transcriber(<br>model_path,<br>ModelArch.TINY_STREAMING,<br>options={"context": open("migration-plan.md").read()},
# ...or follow the document as the user moves through it:<br>transcriber.set_context(current_page_text)
What gets picked is decided by the model's own tokenizer. That vocabulary is ordered by frequency, so an everyday word has a token to itself while jargon and proper nouns have to be spelled out of several subwords, and needing more than one is the signal used here. Given the passage
Migration notes for the platform team. We will move the remaining services onto Kubernetes this quarter, with Ceph behind the storage classes and etcd holding the cluster state. Ask about the ingress before the meeting.
Tiny Streaming chooses Migration, Kubernetes, Ceph, etcd and ingress, and leaves every function word and every ordinary noun alone. Because the judgment comes from the tokenizer rather than from a word list we ship, it follows whichever language the loaded model was built for at no extra cost.
Terms are ranked by how often the passage says them, with the strangest-looking word winning a tie, and the list is then capped — 200 terms by default, or whatever you pass as context_max_terms at load time and as the second argument to set_context(). Keep the cap modest. As What a long list costs below shows, length is charged against every word you did not ask for, so the terms a passage leans on hardest are worth more than its whole long tail. Passing a book is fine; the cap is what keeps that from being a bad idea.
Everything else behaves like a key terms list, because that is what it becomes: it can be called while audio is streaming, takes effect on the next transcription, and needs a streaming architecture. Capitalization is taken from the passage, so a passage that writes "Kubernetes" is what makes the transcript write it that way too. The one thing to know is that only single words are proposed — a passage cannot tell us that "Anushka Sharma" is one name rather than two — and words containing digits are skipped, since a passage has far more dates and quantities in it than it has names like "IPv6". Name those outright with keyterms alongside the passage, and both sets are used.
The context and context_max_terms load options work anywhere transcriber options do. Replacing the passage on a running transcriber is wrapped in every binding — set_context() in Python, setContext() in Swift, Java and JavaScript, moonshine_transcriber_set_context() in C — and the max-terms argument takes 0 to mean the default of 200.
Supply a Key Terms List¶
When you do know the words — a product catalog, a contact list, the phrases your own interface uses — name them. A list is more precise than a passage: it can carry multi-word terms, it spends no slots on words that happened to be nearby, and nothing is inferred. There's no training step, so the list can be different for every transcriber and can change while audio is streaming:
from moonshine_voice import Transcriber, ModelArch
transcriber = Transcriber(<br>model_path,<br>ModelArch.TINY_STREAMING,<br>options={"keyterms": "Kubernetes,Ceph,etcd"},
# ...or follow whatever the user is looking at, mid-stream:<br>transcriber.set_keyterms(["Anushka Sharma", "Jurgen Klopp"])
Match the...