Apple's New Speech API vs Whisper: The First Real Benchmark

The result, up front

Apple's new SpeechAnalyzer is the most accurate on-device speech engine we tested. It beat every Whisper model we ship, including Whisper Small, on both the clean and the noisy half of LibriSpeech, while running roughly three times faster than Small. And the API it replaces, SFSpeechRecognizer, came last on clean speech: behind even Whisper Tiny, a 40MB model.

Enginetest-clean WERtest-other WERModel size

Apple SpeechAnalyzer (iOS/macOS 26)2.12% 4.56% system<br>Whisper Small (WhisperKit CoreML)3.74%7.95%~460MB<br>Whisper Base5.42%12.51%~140MB<br>Whisper Tiny7.88%17.04%~40MB<br>Apple SFSpeechRecognizer (legacy)9.02%16.25%system

Lower is better: WER is word error rate, the percentage of words an engine substitutes, drops, or invents. LibriSpeech test-clean is 2,620 utterances of clean read speech; test-other is 2,939 harder, noisier utterances. Every engine ran fully on-device on an Apple M2 Pro (32GB, macOS 26.5.1).

Apple SpeechAnalyzer2.12%

Whisper Small3.74%

Whisper Base5.42%

Whisper Tiny7.88%

SFSpeechRecognizer (legacy)9.02%

Why we ran this

With iOS 26 and macOS 26, Apple replaced SFSpeechRecognizer with a new API, SpeechAnalyzer and SpeechTranscriber. It published no accuracy figures for either one. So every developer deciding whether to migrate, and everyone comparing Apple's built-in recognition against Whisper, has been guessing.

We ship both Apple engines and three Whisper models side by side in Inscribe, a private on-device AI workspace, which puts us in an unusual position: we can run all five through identical production code paths on the same machine and the same audio. So we did.

Should you migrate off SFSpeechRecognizer?

Yes. This is the clearest result in the data. The new API cuts word error rate by 3.5 to 4x on the same audio: from 9.02% to 2.12% on clean speech, and from 16.25% to 4.56% on noisy speech. There is no accuracy trade-off to weigh; the new API wins everywhere we measured, and it produces punctuated, cased text where the legacy engine's output is rougher.

Put differently: an hour-long meeting transcribed with the legacy API contains roughly four times as many wrong words as the same meeting through SpeechAnalyzer. If your app still uses SFSpeechRecognizer for anything longer than a voice command, the migration is worth it on accuracy alone.

SpeechAnalyzer vs Whisper

The more surprising result: Apple's new engine also beat Whisper Small, the largest model we ship, by a comfortable margin on both splits, at roughly a third of Whisper Small's compute time per second of audio. For English, on Apple hardware, the built-in engine is now the strongest on-device option we can measure.

Whisper keeps two real advantages. It covers far more languages (SpeechTranscriber supports around 30 locales), and it runs anywhere, not just on Apple platforms with OS 26. But for English transcription on a current iPhone or Mac, the days of Whisper being the automatic accuracy pick are over.

We changed our own product on this result: Inscribe's Auto engine now prefers SpeechAnalyzer for the languages it supports, and Whisper for everything else. Shipping a benchmark and ignoring it in your own defaults would be a strange kind of honesty.

Speed

All five engines ran comfortably faster than real time: between roughly 12x and 40x on the M2 Pro, meaning an hour of audio transcribes in about 1.5 to 5 minutes on-device. SpeechAnalyzer was about 3x faster than Whisper Small per second of audio while beating it on accuracy. We are deliberately not printing a precise per-engine timing table yet: the accuracy runs shared the machine with a development workload, which does not affect WER but does add noise to timing. We will update this page with timings from a dedicated idle run.

Methodology, and why you can check it

A benchmark from a company that sells one of the engines should be treated with suspicion. Ours has two properties designed for that suspicion.

The Whisper column is reproducible against OpenAI's own numbers

We used LibriSpeech precisely because OpenAI published Whisper's WER on it. If our harness measured Whisper correctly, our numbers should land on theirs. They do, on all six measurements:

Engine / splitOursOpenAI publishedDelta

Whisper Tiny, test-clean7.88%7.6%+0.28<br>Whisper Base, test-clean5.42%5.0%+0.42<br>Whisper Small, test-clean3.74%3.4%+0.34<br>Whisper Tiny, test-other17.04%16.9%+0.14<br>Whisper Base, test-other12.51%12.4%+0.11<br>Whisper Small, test-other7.95%7.6%+0.35

The small, consistent positive offset (a slightly stricter text normalizer plus CoreML quantization) is what honest reproduction looks like; random error would scatter in both directions. Since the same corpus, normalizer, and scorer produced the Apple columns, the numbers nobody else can check inherit the validation from the numbers anyone can.

The raw transcripts are public

Every per-utterance hypothesis for both Apple engines is...