Stable Diffusion in the Browser with WebNN + ONNX Runtime
Stable Diffusion in the Browser with WebNN + ONNX Runtime
-->
Running stable diffusion image generation models in the browser is a game changing experience for AI. This article explains how to do it using ONNX models, using WebNN and ONNX run time.
Explore Here (No Login)
Blog> Categories:
AI-ML,
Scribbler,
JavaScript
Table of Contents
Hide
A practical guide using Scribbler WebNN + scribbler-webnn/sd.js
🧠 Core Stack Explained
1. WebNN (Hardware-native AI in browser)
2. ONNX Runtime Web
3. Stable Diffusion (ONNX optimized)
4. Scribbler WebNN (sd.js)
⚙️ How It Works (Pipeline)
🧩 An Easy Implementation using scribbler-webnn/sd.js
1. Load ONNX Runtime (WebGPU/WebNN)
2. Import sd.js (THIS is key)
3. Create Pipeline
4. Load Model (async + progress)
5. Generate Image (Correct API)
6. Render to Canvas
🖥️ Sample Page
📓 Scribbler Notebook
⚡ Performance: Why This Is Wild
🟢 Turbo Models (e.g., SD Turbo, LCM)
🔵 Stable Diffusion 1.5
⚙️ Why it’s fast
🔥 GPU Acceleration (Intel + NVIDIA)
🌍 Why This Is a Game Changer
1. Zero Setup AI
2. Infinite Scale (No Servers)
3. Privacy by Default
4. New App Paradigm
5. Shareable AI Apps
6. Democratization of AI
🧠 Architectural Shift: Cloud → Edge → Browser
🧪 What This Enables Next
🧩 Key Takeaway
✨ Final Thought
🚀 Live demos & resources
A practical guide using Scribbler WebNN + scribbler-webnn/sd.js #
For years, running Stable Diffusion meant:
Python environments
CUDA setup
Expensive GPUs
Cloud APIs
That model is breaking.
Today, with WebNN + ONNX Runtime Web + WebGPU , you can:
Run Stable Diffusion directly inside a browser tab — locally, privately, and fast.
This shift is profound. It turns every browser into an AI runtime , not just a UI.
According to recent work in browser ML stacks, technologies like WebGPU, WASM, and ONNX Runtime Web now enable high-performance AI inference locally without servers.
🧠 Core Stack Explained #
1. WebNN (Hardware-native AI in browser) #
WebNN is a new web standard that allows models to run directly on:
GPU (Intel / NVIDIA / AMD)
CPU
NPU (AI accelerators)
It exposes native acceleration without needing CUDA or platform-specific code.
👉 Think of it as:
“DirectML / CoreML — but for the web.”
Note:To run Stable Diffusion efficiently in the browser using WebNN and WebGPU, you must first enable these experimental features in your browser settings. Navigate to chrome://flags (or edge://flags) and enable WebNN and WebGPU, then restart the browser. Without these enabled, the model will fall back to CPU-based execution, resulting in significantly slower performance. With GPU acceleration active, turbo models can generate images in as little as ~100ms, while full Stable Diffusion 1.5 runs in just a few seconds directly inside the browser.
2. ONNX Runtime Web #
ONNX Runtime is the execution engine that:
Loads ONNX models
Optimizes execution graph
Chooses backend (WebNN, WebGPU, WASM)
It supports:
WebGPU acceleration
WebNN execution provider
WASM fallback
It can deliver massive speedups (20x–500x vs CPU in some cases) when GPU is used.
3. Stable Diffusion (ONNX optimized) #
Stable Diffusion runs as a pipeline:
Text encoder
UNet (denoising core)
VAE decoder
The trick is:
✔ Convert to ONNX<br>✔ Quantize (fp16 / int8 / 4-bit)<br>✔ Split into browser-friendly chunks
4. Scribbler WebNN (sd.js) #
The project you shared:
👉 scribbler-webnn
Provides:
A lightweight Stable Diffusion engine in JS
WebNN + ONNX Runtime integration
Notebook-style execution (Scribbler)
This is the missing piece that makes everything usable.
⚙️ How It Works (Pipeline) #
Prompt → Tokenizer → Text Embedding<br>UNet (diffusion steps)<br>Latent image<br>VAE decode<br>Final Image (Canvas)
All of this runs inside the browser .
🧩 An Easy Implementation using scribbler-webnn/sd.js #
1. Load ONNX Runtime (WebGPU/WebNN) #
const ortRes = await fetch("https://data.jsdelivr.com/v1/packages/npm/onnxruntime-web");<br>const ortData = await ortRes.json();<br>const ortVersion = ortData.tags.dev;
await scrib.loadScript(<br>`https://cdn.jsdelivr.net/npm/onnxruntime-web@${ortVersion}/dist/ort.webgpu.min.js`<br>);
2. Import sd.js (THIS is key) #
const SD_BASE = "https://decentralized-intelligence.com/scribbler-webnn/";<br>const { SDPipeline, AVAILABLE_MODELS } = await import(`${SD_BASE}/sd.js`);
3. Create Pipeline #
const pipeline = new SDPipeline({<br>model: "sd-turbo", // or "sdxl-turbo", "sd-1.5"<br>provider: "webnn", // or "webgpu"<br>});
👉 This is the core object — everything runs through it.
4. Load Model (async + progress) #
await pipeline.load({<br>onProgress: (p) => {<br>console.log("Loading:", p, "%");<br>});
5. Generate Image (Correct API) #
const result = await pipeline.generate({<br>prompt: "a futuristic city at sunset, ultra realistic",<br>steps: 4, // turbo models: 1–4 steps<br>});
6. Render to Canvas #
The pipeline does NOT return a DOM image.
It returns raw output that you...