AI Video Generation Benchmark: 11 LLMs Compared | MesmerTools<br>All benchmarksAI Video Generation Benchmark<br>11 LLMs each turned the same 4 prompts into Remotion motion-design videos through FrameCall's production pipeline. Every model got the same brief and went through the same renderer. What came out is what you see here, failures included.
Claude Fable 5 in action<br>Tap any clip to watch it full size
YouTube Channel Intro16:9
SaaS Product Launch Teaser1:1
Instagram Story Ad9:16
Three of the four briefs, generated by Claude Fable 5, the newest model in the lineup, with zero hand edits. Keep scrolling to watch all 11 models attempt the same videos side by side.
Methodology<br>Each model received an identical brief for every video: the same instruction set, the same user prompt, and the same canvas specs, with nothing tailored per model. The code each one produced went straight into the same compile and render step, with no hand fixes and no error-correction loop in between. When a provider dropped a connection or returned nothing usable we retried and kept count, which is what the Tries column shows. A failed card is the model's own failure: it either produced no usable code or its code crashed during rendering. The Claude models ran through Claude Code instead of the API, so their generation times are left out and their token counts and costs are estimates based on response sizes priced at OpenRouter's Anthropic rates.
Leaderboard<br>The order comes from watching all four videos from every model and judging the output quality by hand.<br>Swipe the table sideways for all columns<br>#ModelRan via Videos rendered Tries Avg gen time Avg code size Total cost1Gemini 3.1 Pro<br>Google<br>OpenRouter4/413110.0s16.3k$0.8002Claude Fable 5<br>Anthropic<br>Claude Code4/44—23.8k~$2.0553Claude Opus 4.8<br>Anthropic<br>Claude Code4/44—25.9k~$1.1294GPT-5.5<br>OpenAI<br>OpenRouter4/44227.3s28.4k$1.5685Gemini 3.5 Flash<br>Google<br>OpenRouter4/4563.3s30.0k$0.5256Claude Sonnet 4.6<br>Anthropic<br>Claude Code4/44—21.2k~$0.5747Kimi K2.6<br>Moonshot AI<br>OpenRouter4/415378.5s18.1k$0.2928GLM 5.1<br>Z.ai<br>OpenRouter4/48263.8s26.5k$0.3029MiMo V2.5 Pro<br>Xiaomi<br>OpenRouter3/421359.5s25.2k$0.02310DeepSeek V4 Pro<br>DeepSeek<br>OpenRouter3/45142.4s18.0k$0.06211DeepSeek V4 Flash<br>DeepSeek<br>OpenRouter4/45117.1s29.9k$0.019
1. YouTube Channel Intro<br>1920×1080 · 16:9 · 30fps<br>Show the exact prompt every model receivedCreate a polished 12-second YouTube channel intro for a tech review channel called "TECHPULSE".
Include: logo reveal (a "TP" monogram), channel name animation, tagline "honest reviews • weekly uploads", social handles (@techpulse, /techpulse, techpulse.com), and a subscribe CTA with bell icon.
Should look like something a channel with 500k+ subs would use. Professional but approachable vibe. Dark theme with blue accents.
#1Gemini 3.1 ProOpenRouter<br>Google140.1s14.2k chars· 7 tries
#2Claude Fable 5Claude Code<br>Anthropic17.3k chars
#3Claude Opus 4.8Claude Code<br>Anthropic21.4k chars
#4GPT-5.5OpenRouter<br>OpenAI243.9s22.6k chars
#5Gemini 3.5 FlashOpenRouter<br>Google66.9s26.6k chars
#6Claude Sonnet 4.6Claude Code<br>Anthropic18.9k chars
#7Kimi K2.6OpenRouter<br>Moonshot AI339.5s14.5k chars· 4 tries
#8GLM 5.1OpenRouter<br>Z.ai223.6s26.0k chars· 2 tries
#9MiMo V2.5 ProOpenRouter<br>Xiaomi100.6s12.6k chars· 4 tries
#10DeepSeek V4 ProOpenRouter<br>DeepSeek178.4s17.4k chars
#11DeepSeek V4 FlashOpenRouter<br>DeepSeek122.8s22.3k chars
2. SaaS Product Launch Teaser<br>1080×1080 · 1:1 · 30fps<br>Show the exact prompt every model receivedCreate a hype 12-second product launch teaser for a SaaS app called "FlowBoard" - a team productivity tool.
Show off the product name with dramatic typography (gradient text looks good), highlight 3 key features: "Real-time Collaboration", "AI Task Prioritization", "Team Analytics". Include an app mockup, tagline "Built for teams that ship", waitlist CTA, and the URL flowboard.io.
Modern startup aesthetic, square format for social. Make it feel like a Product Hunt launch video.
#1Gemini 3.1 ProOpenRouter<br>Google97.8s19.1k chars· 3 tries
#2Claude Fable 5Claude Code<br>Anthropic21.6k chars
#3Claude Opus 4.8Claude Code<br>Anthropic23.8k chars
#4GPT-5.5OpenRouter<br>OpenAI211.7s30.2k chars
#5Gemini 3.5 FlashOpenRouter<br>Google57.7s24.5k chars· 2 tries
#6Claude Sonnet 4.6Claude Code<br>Anthropic22.6k chars
#7Kimi K2.6OpenRouter<br>Moonshot AI334.0s17.7k chars· 5 tries
#8GLM 5.1OpenRouter<br>Z.ai311.7s19.7k chars· 3 tries
#9MiMo V2.5 ProOpenRouter<br>Xiaomi98.4s28.2k chars· 3 tries
#10DeepSeek V4 ProOpenRouter<br>DeepSeek130.1s11.2k chars
#11DeepSeek V4 FlashOpenRouter<br>DeepSeek131.9s24.5k chars
3. Instagram Story Ad<br>1080×1920 · 9:16 · 30fps<br>Show the exact prompt every model receivedCreate a 15-second Instagram Story ad for wireless earbuds called "SoundPods Pro".
Product showcase with: product visual (earbud silhouette), name, price ($79, crossed out $149 showing discount), features (40hr battery, ANC, wireless charging), social proof (5-star rating, "2,847 reviews", customer quote), and Shop Now CTA with...