Local translation: when small dedicated models beat Goliath - QVAC by Tether

SDK

Fabric

Genesis

Models

QV.AC

Health

Our Vision

Blog

Discord Forum

Keet P2P Room

Feature Requests

Contact Us

Engineering<br>Models<br>Translation

Updated<br>May 12, 2026

Local translation: when small dedicated models beat Goliath

Article by

Olya Sirkin

Imagine you’re translating a medical document for a family member. Or maybe it’s a contract, a private message, a journal entry. You paste it into a cloud translation API and magically, something happens. But… where did your input go? Who read it? For how long is it stored?

Turns out the privacy angle isn’t theoretical. Cloud translation services handle inputs under a wide variety of Terms. Free tiers don’t offer the same declared protections as their paid APIs. Offline translations, on the other hand, eliminate entire categories of compliance burden: no data processing agreements, no audits, no cross-border transfer concerns.

But it turns out that translation that works anywhere, at any time, where your data remains under your control, is not as straightforward an issue as one might think. Still, we decided this was a non-negotiable premise – everything must happen entirely on the device: phones, laptops, even embedded hardware. And this is how we went about it.

"Bigger" is not always the answer

The AI world is obsessed with scale right now. The instinct when you need translation is to reach for a large language model – a 7B or 13B parameter general-purpose model that can translate as one of a hundred things it does. It works. Sometimes well, sometimes less well.

But on edge devices, hardware is the real constraint. Even a small 2B translation LLM like Salamandra takes 1.5GB of storage, 12s to load, and 3.6s per sentence on a laptop CPU. If you upgrade to any modest 7B model you immediately jump to 4-14GB and the results are even slower. On a phone, that UX is not acceptable.

Now compare that with a smaller, dedicated translation model for a specific language pair: it’s in the range 20-35MB (!). It loads in 105ms and translates in 10ms per sentence in batch mode – that’s 344x faster than the LLM . And it doesn’t hallucinate because it’s not generating creative text – it’s doing one thing, and doing it well.

The math is simple: if you need English-to-French translation, you don’t need a model that also writes poetry, summarises articles, and role-plays as a pirate.

The landscape

We’re not the only ones thinking this way:

Mozilla AI built Bergamot with EU funding to bring translation directly into Firefox – no need for extensions or cloud, just WebAssembly running in your browser

Google’s offline translation packs are 35-45MB each and serve billions of users

Apple’s on-device translation works across their native apps without sending text to a server

The trend is clear: translation is moving to the edge. The models are small enough. The hardware is fast enough. The only question is how to get there as a developer without building everything from scratch .

That’s where we want QVAC to help! For translation specifically, QVAC wraps dedicated neural machine translation (NMT) engines (small, fast models built for this job) and exposes them through a clean SDK. You pick a language pair, load the model, and translate. Everything stays on your device.

Here’s a simple example using English to French with the Bergamot engine:

import { loadModel, translate, unloadModel, BERGAMOT_EN_FR } from "@qvac/sdk"

const modelId = await loadModel({<br>modelSrc: BERGAMOT_EN_FR,<br>modelType: "nmt",<br>modelConfig: {<br>engine: "Bergamot",<br>from: "en",<br>to: "fr",<br>beamsize: 1,<br>temperature: 0.2,<br>},<br>onProgress: (progress) => {<br>console.log(progress)<br>},<br>})

const result = translate({<br>modelId,<br>text: "This is a test of the Bergamot translation model.",<br>modelType: "nmt",<br>stream: false,<br>})

const translatedText = await result.text<br>console.log(translatedText)<br>// → "C'est un test du modèle de traduction Bergamot."

await unloadModel({ modelId })

That’s it. The model downloads once, caches locally, and every subsequent call is fully offline. No network request, no third party ever sees your text.

"Yes but I want to translate a batch of sentences at once"

Worry not, just pass an array instead:

const result = translate({<br>modelId,<br>text: [<br>"Hello world",<br>"How are you today?",<br>"The weather is nice",<br>],<br>modelType: "nmt",<br>stream: false,<br>})

const translatedText = await result.text<br>// → "Bonjour le monde"<br>// → "Comment allez-vous aujourd'hui?"<br>// → "Le temps est beau"

What we learned solving this problem

We didn’t start with Bergamot. Our first translation models were Opus-MT (Marian-based), and they worked – but they were larger and slower than what we wanted to deliver on mobile devices. Opus also had a coverage problem: if we needed a language pair that didn’t exist, we’d have to train the model ourselves . This quickly became a huge investment for each new pair. When we found Bergamot, which is...