Parakeet vs Apple's Speech API vs MOSS: Benchmark Round 2

The result, up front

NVIDIA's Parakeet TDT v2, running as the int8 CoreML model a real app would ship, is the most accurate on-device English engine we have measured: 2.01% WER on LibriSpeech test-clean and 3.40% on test-other. But the clean-speech race is now astonishingly tight. Three engines with completely different architectures, a speech transducer from NVIDIA, a 0.9B audio language model from Fudan University's OpenMOSS lab, and Apple's system engine, landed within 0.11 points of each other. Noisy speech is where they separate.

Enginetest-clean WERtest-other WERDownload

Parakeet TDT v2 (CoreML int8) new2.01% 3.40% ~443MB<br>MOSS-Transcribe-Diarize (MLX) new2.07%4.68%~1.7GB<br>Apple SpeechAnalyzer (iOS/macOS 26)2.12%4.56%system<br>Parakeet TDT v3 (CoreML int8) new2.51%4.28%~467MB<br>Whisper Small (WhisperKit CoreML)3.74%7.95%~460MB<br>Whisper Base5.42%12.51%~140MB<br>Whisper Tiny7.88%17.04%~40MB<br>Apple SFSpeechRecognizer (legacy)9.02%16.25%system

Lower is better: WER is word error rate, the percentage of words an engine substitutes, drops, or invents. Same corpus as round 1: LibriSpeech test-clean (2,620 utterances of clean read speech) and test-other (2,939 harder, noisier utterances), all 5,559 run in full on the same Apple M2 Pro (32GB, macOS 26.5.1), scored by the same normalizer and the same scoring code.

Parakeet TDT v22.01%

MOSS-Transcribe-Diarize2.07%

Apple SpeechAnalyzer2.12%

Parakeet TDT v32.51%

Whisper Small3.74%

Why a round 2

When we published the first round of this benchmark, the most common response, on Hacker News and in every developer community that discussed it, was some version of: Whisper is not the state of the art anymore, benchmark Parakeet. Fair. And a Hacker News commenter pointed us at MOSS-Transcribe-Diarize, a model released days earlier that does transcription and speaker diarization jointly in one pass.

So this round adds three engines, measured on the identical 5,559 utterances with the identical scoring pipeline, which makes every number here directly comparable to round 1. As before, every raw transcript is downloadable below, so you can rescore us with your own normalizer if you disagree with ours.

Parakeet: the new accuracy champion, with an asterisk nobody prints

Parakeet TDT 0.6B v2 wins this benchmark. On clean speech its lead over Apple's engine is 0.11 points, a photo finish. On noisy speech the lead is real: 3.40% vs 4.56%, about a 25% reduction in errors. If your only criterion is English accuracy on difficult audio, Parakeet v2 is the strongest thing you can run on-device on a Mac today.

Here is the asterisk. NVIDIA's published numbers for this model are 1.69% and 3.19%, measured on the reference GPU implementation. We measured 2.01% and 3.40%, because we benchmarked what an app can actually ship: the int8-quantized CoreML conversion (via the open-source FluidAudio package, the same one Inscribe already uses for speaker diarization). The difference, roughly +0.3 on clean and +0.2 on noisy, is the cost of quantization plus the CoreML port. We could not find that number published anywhere, and it changes the story: the shippable Parakeet's lead over Apple's built-in engine is about a third of what the published numbers suggest.

Parakeet v3, the multilingual version covering 25 European languages, trades English accuracy for that coverage: 2.51% clean / 4.28% other. Interestingly it still edges out Apple's engine on noisy speech while losing on clean, and it beats Whisper Small, which covers more languages, everywhere.

MOSS: a different kind of contender

MOSS-Transcribe-Diarize is not another speech recognizer. It is a 0.9B-parameter end-to-end audio language model that transcribes and diarizes at the same time: its output is a timestamped transcript with speaker labels like [S01] and [S02] already attached, across 50+ languages, in a single pass over up to 90 minutes of audio.

On pure transcription accuracy it landed at 2.07% clean / 4.68% other. Statistically indistinguishable from Apple's SpeechAnalyzer on clean speech, slightly behind on noisy speech, and ahead of every Whisper model, while also doing a job no other engine in this table attempts.

We probed the diarization with a controlled test: six utterances from three different LibriSpeech speakers stitched into one 62-second file, so the true transcript and the true speaker turns are known exactly. MOSS scored 1.75% WER on the words and attributed all 13 of its output segments to the correct speaker, including correctly re-identifying each speaker when they returned 30+ seconds later. That is a small, easy test, clean speech with no crosstalk, but it is the hard part of diarization done right where plenty of production pipelines get it wrong. A proper multi-speaker meeting benchmark (AMI corpus, real crosstalk, measured DER) is round 3.

The bug we found: MOSS goes silent when speech starts at zero

Benchmarks find bugs. Round 1 found one in...