# Design Mobile On-Device Translation: Offline Models and Privacy Source: https://www.techinterview.org/post/3233475100/design-mobile-on-device-translation/ Updated: 2026-07-26 · techinterview.org Modern mobile translation apps (Google Translate, Apple Translate) increasingly run inference on-device. [The interview](/category/system-design/) tests whether you understand the tradeoffs of running ML models on a phone, the realities of language pack distribution, and the privacy benefits of keeping translations local. ## Functional requirements - Translate text input across many language pairs. Interviewers probe how you scale to hundreds of pairs without shipping every model on the device — clarify that packs are downloaded on demand rather than bundled. Nail down whether you support pivoting through a bridge language (translate xx→en→yy) when a direct pair has no model. - Voice input → translated voice output. This chains speech recognition, translation, and text-to-speech, so latency compounds; call out that each stage adds delay and that you want to stream partial results rather than wait for the full utterance. - Camera-based translation (point at a sign, see translation overlay). The hard part is doing OCR, translation, and rendering fast enough to track a moving frame; mention that you can freeze on a still capture when the user taps, which relaxes the real-time budget. - Conversation mode (two-way real-time translation). Two people speak in turn and each hears the other's language. Interviewers look for how you detect who is speaking and in which language (auto language detection per turn) and how you keep the device's own TTS output from feeding back into the microphone. - Work offline for downloaded language pairs. The offline path is the whole point of on-device; be explicit that a downloaded pair works in airplane mode with no cloud call, and that the app should clearly show which pairs are available offline versus cloud-only. ## On-device vs cloud Tradeoffs: - **On-device:** works offline, no privacy concern, faster latency, works in low-network areas - **Cloud:** better quality (larger models), supports more languages, no storage cost on device Modern apps default to on-device for popular pairs (en-es, en-fr, en-zh) and fall back to cloud for less common pairs or when higher quality is needed. ## Language packs Per-language-pair model files. Sizes: - Text translation: 30–100MB per language pair - Voice: 100–500MB additional - Camera/OCR: depends on script complexity User downloads on first use of a pair, or eagerly via "download for offline" UI. Storage budget: a few GB for serious users. ## Camera translation Pipeline: - Camera frame captured (continuous video). You sample frames from the live preview; you don't need every frame, so downsample to a few per second to save battery and reduce heat. - OCR detects text regions. A detection model finds bounding boxes of text before you try to read them, which lets you skip empty frames cheaply. - Recognize text per region. Each box is passed to a recognizer; script complexity matters here — Latin text is cheap, while dense CJK or Arabic needs heavier models. - Translate. The recognized string goes through the same translation model as text mode, so this step reuses the downloaded language pack. - Render translated text overlaid on the camera frame, replacing the original. Match the original's position, color, and background so the sign looks natural; interviewers like to hear how you handle text that grows or shrinks after translation. The challenge: do this at 30+ fps on a phone. Each step is GPU-accelerated. ML Kit (Android) and Vision framework (iOS) provide pre-built models. ## Voice translation Pipeline: - Audio captured. Buffer microphone input and detect speech boundaries so you know when an utterance ends and translation can start. - Speech recognized to text (in source language). On-device ASR turns the audio into text; accents and background noise are the usual failure points to acknowledge. - Translated. The recognized text runs through the translation model, the same path as typed input. - TTS produces audio (in target language). A text-to-speech voice speaks the result; pick a voice that matches the target language and start playback as soon as the first words are ready. For conversation mode, the device runs the pipeline in both directions concurrently. ## Privacy The strongest argument for on-device translation: - Translated text never leaves the device - Voice input never reaches a server - Sensitive conversations (medical, legal) stay local Some apps offer explicit "Private mode" that disables any cloud fallback. ## Model updates Models improve over time. Background updates: - App periodically checks for newer model versions. A lightweight manifest check tells the app which packs have updates without downloading anything yet. - Downloads on Wi-Fi only. Model files are large, so gate downloads on unmetered connections (and ideally idle or charging state) to avoid burning cellular data and battery. - Replaces old model atomically. Write the new pack to a temp location and swap it in one step so a translation mid-download never reads a half-written file. ## Battery On-device inference is GPU-intensive. Mitigations: - Use platform-optimized inference (CoreML on iOS, TensorFlow Lite or ONNX on Android) - Quantize models to int8 where quality permits - For camera mode, throttle frame rate based on device temperature ## Frequently Asked Questions ### How does on-device translation compare to cloud quality in 2026? For popular language pairs, on-device is now within 5–10% of cloud BLEU scores. For low-resource languages, cloud still wins meaningfully. ### Why does my translation app sometimes use the cloud? If the language pair is not downloaded locally, the app falls back to cloud. Some apps explicitly use cloud for more nuanced text (longer documents, formal language). ### Can on-device models be updated without app updates? Yes. Models are typically distributed separately from the binary, downloaded on demand or in the background.