Open-sourcing FastVideo Dreamverse: Real-Time Vibe Directing with LTX-2 on a single NVIDIA B200 GPU | Hao AI Lab @ UCSD
Video generation should keep creators in the loop, not pull them out of it. In our previous Dreamverse post, we introduced vibe directing : a workflow for steering video generation through fast, natural-language iteration. Today, FastVideo is making that workflow open source by releasing the Dreamverse frontend and backend as a reference application for real-time generative video systems. Dreamverse is based on a version of LTX’s open weights diffusion model, LTX-2. Optimized for a single NVIDIA B200 GPU, Dreamverse gives developers the full stack to build their own real-time generative video applications on FastVideo.<br>What Is FastVideo’s Dreamverse?#<br>Dreamverse is a real-time video generation workspace for vibe directing. It is to video what vibe coding is to software: start from a simple idea, watch the result, and keep steering with natural language. Keep the subject, change the camera, continue the scene, or try another direction, all within a quick iteration loop.<br>Edit Video Background<br>Edit Video Character<br>View history generations from Gallery<br>With this release, Dreamverse becomes not only a runnable product prototype, but also a sample architecture for the FastVideo community building real-time video generation and editing applications.<br>What We Are Releasing#<br>FastVideo designed Dreamverse to be a self-hostable application inside the FastVideo ecosystem. You can use your own NVIDIA B200 GPU or rent one from a cloud GPU provider, launch the runtime, and edit directly from your browser. This release includes:<br>a browser workspace for directing and editing generated scenes<br>a FastVideo backend runtime for prompt handling, GPU workers, and streaming<br>an NVIDIA Blackwell-optimized generation path with NVFP4 inference, FA4, and torch compile, built on LTX-2<br>prompt rewriting for edits, continuations, and longer scene control<br>tests, benchmarks, mock backend support, and Docker images for development and deployment<br>How To Run Dreamverse#<br>Running Dreamverse is meant to be simple. Dreamverse is supported on NVIDIA B200 GPUs for the real-time generation path, and each Dreamverse worker occupies one NVIDIA B200 GPU for its workload. We also provide a Docker image for simple deployment with the generation dependencies already installed.<br>Dreamverse deploys on a local GPU, a self-hosted B200 server over SSH, Docker, or serverless Modal — for detailed instructions and scripts, see the Dreamverse README.<br>To start the backend Dreamverse server, simply run:<br>uv pip install "fastvideo[dreamverse]"<br>dreamverse-server --host 0.0.0.0 --port 8009
The backend also exposes liveness and readiness endpoints for checking whether the server is running and ready to generate.<br>curl http://localhost:8009/healthz<br>curl http://localhost:8009/readyz
After the server is ready, start the web app from the Dreamverse frontend package in another terminal:<br>pnpm install --frozen-lockfile<br>BACKEND_HOST=localhost BACKEND_PORT=8009 pnpm run dev
Then open the frontend URL to start your generations!<br>If you want to work on the frontend without a GPU, simply start the mock backend instead. It sends pre-generated video through the same websocket and streaming path as the real backend:<br>dreamverse-mock-server --latency 200 --port 8009
How Dreamverse Works#<br>The browser workspace is where you direct the scene. You type prompts, review generated clips, edit the prompt sequence, and ask Dreamverse to rewrite the rollout. The browser sends those requests to the Dreamverse runtime, then plays each new video segment as it streams back.<br>The Dreamverse runtime is the bridge between the browser workspace and the backend generation stack. It manages the frontend-backend message queue, the current session working memory, prompt memory, prompt enhancer, prompt rewriter, prompt safety, and the lifecycle of GPU workers. When the browser sends a request, the runtime decides which backend component should handle it and what prompt sequence is accepted for generation.<br>User<br>Browser workspace<br>| prompts, rewrites, session controls<br>| video/audio chunks<br>Dreamverse runtime<br>| session state, prompt memory, safety, rewrite<br>GPU worker pool<br>| one worker per visible GPU<br>FastVideo generator<br>| LTX-2 video + audio segments<br>fMP4 streaming layer<br>| fragmented MP4 over websocket<br>Browser playback
After a user prompt reaches the runtime, the prompt pipeline can run safety checks and rewriting before generation. The safety filter uses fastText classifiers for NSFW and hate-speech detection when enabled. The prompt rewriter then expands the user’s instruction into a detailed prompt for the next segment. Dreamverse provides a curated system prompt and a continuation prompt for this job, preserving user intent while adding details such as camera movement, actor movements, and scene context. This makes each continuation smoother and more logical while still letting the user steer at...