Mastering Edge AI on Raspberry Pi with LiteRT and Gemma
- Google Developers Blog
Search
Mastering Edge AI on Raspberry Pi with LiteRT and Gemma
AUG. 11, 2026
Lu Wang
Software Engineer
Terry Heo
Software Engineer
Naushir Patuck
Software Engineer
Raspberry Pi Ltd
José María Casanova
Graphics Software Engineer
Igalia
Share
Imagine building a fully autonomous robot that can see, hear, and react to its environment in real time, completely offline on a single compact device like Raspberry Pi. Edge AI unlocks this exact autonomy. It enables developers to build highly secure and self-contained systems like intelligent robotics and local AI agents with zero cloud dependencies, ultra-low latency, and total data privacy.<br>We’ve made running edge AI on the Raspberry Pi a breeze with Google AI Edge’s LiteRT, high-performance production-proven on-device inference runtime. LiteRT allows you to seamlessly deploy everything from classical ML models to state-of-the-art LLMs right out of the box across platforms. By delivering optimized execution and hyper-efficient memory usage on both CPU and GPU, LiteRT maximizes your Raspberry Pi's full computing potential.
Sorry, your browser doesn't support playback for this video
Reachy Mini reacts with responses and movement, powered by Gemma and LiteRT on Raspberry Pi 5.
This hardware-software synergy shines brightest when paired with Gemma, Google’s family of lightweight open models. To show you what’s possible, we'll demonstrate how Gemma and LiteRT on a Raspberry Pi 5 can power the Reachy Mini robot to perceive and react to its environment entirely locally in real time. Read on to get started with your own deployment.<br>Discover the Agentic Capability of Gemma<br>Gemma models are well suited for building autonomous agents, smart cameras, and social robotics that can reason and execute complex, multi-step workflows directly on your Raspberry Pi. To accommodate different hardware constraints, the Gemma family of models provides several highly efficient options:
Gemma 3 270M: A hyper-efficient and compact base model designed for task-specific post fine-tuning, enabling high-speed, low-latency features like sentiment analysis or entity extraction in resource-constrained environments.<br>EmbeddingGemma 300M: A text embedding model that produces high-quality embeddings on-device, great for Retrieval Augmented Generation (RAG), semantic search, and classification.<br>Gemma 3 1B: A lightweight and multilingual text-only model that balances compact size with strong generative capabilities, making it ideal for a wide range of on-device tasks, such as summarization and content creation.<br>Gemma 4 E2B: Tailored specifically for mobile and tight edge environments, it features memory-mapped per-layer embeddings, and is ideal for continuous monitoring, fast text/image/audio inference, and edge-based speech processing where saving RAM is absolutely critical.<br>Gemma 4 E4B: The sweet spot for performance and size. This model delivers noticeably stronger reasoning capabilities and frontier-level edge performance while remaining compact. It is the perfect choice for complex multi-step planning without overwhelming the Pi's resources.
Gemma Performance on Raspberry Pi CPU<br>Through LiteRT-LM, a specialized orchestration layer on top of LiteRT, developers can seamlessly deploy Gemma right out of the box. Under the hood, sophisticated CPU acceleration via LiteRT and XNNPACK ensures the Gemma family of models is highly optimized for resource efficiency and low-latency execution directly on the Raspberry Pi.
On a Raspberry Pi 5, LiteRT-LM delivers a robust performance for Gemma 4 E2B, achieving 99 tokens/sec for prefill and 9 tokens/sec for decode, all while maintaining a remarkably low peak memory footprint of just 1432 MB. This brings Gemma’s highly responsive, general-purpose intelligence to Raspberry Pi .<br>Thanks to Gemma 4 E2B's highly efficient tokenizer, which packs more text into fewer tokens (averaging ~4.2 characters per token), LiteRT-LM achieves an impressive end-to-end generation speed of ~27.3 characters per sec, roughly 300 words per minute (wpm), in the Reachy Mini voice demo. This throughput makes Gemma 4 E2B excellent for real-time speech and translation tasks , delivering text at twice the speed of normal human speech (~150 wpm).<br>Explore more ready-to-use open models to run on Raspberry Pi from the LiteRT Hugging Face Community.<br>Execute on Raspberry Pi GPU with LiteRT<br>On the Raspberry Pi 5, the quad-core ARM Cortex-A76 CPU is a raw computing powerhouse, delivering ~153.6 GFLOPS (FP32) and up to ~2.0 TOPS (INT8). In comparison, the integrated Broadcom VideoCore VII GPU is clocked at 800 MHz and offers a peak of ~76.8 GFLOPS (FP32) and ~0.24 TOPS (INT8).<br>While the CPU possesses a massive capacity advantage, the GPU introduces heterogeneous parallel execution , a paradigm critical for real-time edge applications. Rather than saturating the CPU,...