How AI Creates Animated Videos: Code-Based Rendering Guide & Open-Source Skill
The article explains how language models like Opus 5.5 generate animated videos by writing deterministic web code that renders each frame, covering HTML/CSS, SVG, Canvas, WebGL shaders, Three.js, and programmatic audio, plus practical tips and an open-source skill for consistent, high-quality results.
Recent viral videos like “Vibe Knowledge Awards” (billions of views) are created by language models such as Opus 5.5. Since LLMs only output text, they cannot draw or generate video directly. Instead, Opus writes a web page with a function that, given a timestamp (e.g., 3.2 s), returns the exact frame for that moment. A headless browser then captures frames at 1/30‑second intervals (450 frames for a 15‑second, 30 fps video), and FFmpeg stitches them into an MP4.
Core Principle: Video as a Pure Function of Time
The rendering function must be stateless: each frame is computed solely from the current time, enabling parallel rendering (multiple browsers starting at different timestamps). This also makes edits trivial — re‑render only the changed seconds — unlike video models where a tweak requires full regeneration.
Rendering Technologies
HTML/CSS & SVG
Web pages are text. Animations are keyframe tables: at 0 s a button is 200×56 px, at 0.5 s it widens to 600 px, at 1 s it becomes a 600×400 px card. Camera moves are just scaling the whole scene around a focal point. SVG paths describe shapes; stroke‑dasharray tricks create “hand‑drawn” line reveals.
Canvas for Particle Systems
When thousands of particles fly, HTML/SVG choke. Canvas provides a blank bitmap; each frame clears and redraws. Example: write text to an off‑screen canvas, read colored pixels as target positions, assign each particle a start position by index, then interpolate toward targets over time.
WebGL Shaders for Retro/Film Looks
Shaders run a tiny rule on all 2 million+ pixels simultaneously. Old‑TV effect = scanlines (darken every few rows), chromatic aberration (RGB offset), barrel distortion (bend edges), film grain (random brightness). Content and texture are separate layers for independent tweaking.
Three.js for 3D
Build a scene: meshes (triangles), materials (matte vs. metal), lights, camera. Animate by updating object transforms per frame. Complex models imported from Blender. Example: domino run with ball rolling down a slope, camera tracking.
p5.brush for Hand‑Drawn Styles
Library simulates pencils, crayons, watercolor bleed. Randomness must be seeded by time to avoid jitter; varying the seed every few frames yields the “living line” effect.
Procedural Audio
Sound is a 44.1 kHz sample array. Sine wave at 440 Hz = A note; kick drum = fast frequency sweep + volume decay; hi‑hat = short noise burst. Drums, bass, piano synthesized and mixed to WAV, then merged with video via FFmpeg. Beat‑accurate sync: frame 96 (3.2 s) = sample 141120, both derived from the same timeline.
Real‑World Use Cases
Product promos — feed repo to Claude, let it extract features and storyboard.
Feature demos — App Store previews, landing pages.
Educational videos — physics concepts, historical maps, 3D teardowns, with TTS narration.
Talking‑head visualizations — audio drives main visuals.
Music videos — lyrics + beat detection drive cuts.
Live‑action enhancement — Opus directs video models (e.g., Seedance) for B‑roll, then composites subtitles/effects via code.
Practical Tips for Better Results
Reference Concrete Styles, Not Adjectives
“High‑end” is ambiguous; “Wes Anderson palette & symmetry” is precise. Specify what to borrow (rhythm, composition, color) and what to ignore (characters, exact shots). Ask model to propose three distinct narrative angles (journey, before/after, morph) and pick one.
Lock Rhythm to Music (BPM)
120 BPM → 0.5 s per beat → 30 beats in 15 s. Place cuts, element drops, text pops on beats. Add a half‑beat pause before the climax for impact.
Seamless Transitions
Grow next scene from previous: circle becomes sun, zoom into button reveals next shot. Keep camera moving (pan/zoom in 2D). One focal element at a time; wait for container to settle before introducing text. Test: hide subtitles — visuals alone should convey the story.
Limit Style to 2‑3 Signature Elements
Synthwave = grid floor + striped sunset + chrome text. Pick only those relevant to theme; keep one unexpected element. Enforce global consistency: one background color, one primary, one accent; unified font and motion feel.
Direct Sound Design
Opus defaults to repetitive A‑minor four‑on‑the‑floor. Specify mood: “chiptune”, “quiet with space”. Reserve heavy impacts for key moments. Duck music/SFX under voice‑over.
Supply Real Assets
Give actual logo, screenshots, brand colors, URLs. Model hallucinates UI otherwise. Add cursor clicks and UI feedback for clarity.
Enable Self‑Review via Frames
Opus sees images, not video. Export 20‑frame contact sheet + consecutive frames at transitions + waveform/loudness plots. Let model critique as a “motion designer”.
Debug with Principles, Not Vague Feedback
Pixel art blurry → render to small canvas, integer upscale, disable smoothing.
Pixel motion too smooth → update sprites at 8‑12 fps, keep camera smooth.
UI blurs on zoom → rebuild UI from code, don’t upscale screenshot.
Audio‑visual drift → share single timeline for frame index and sample index.
Non‑deterministic frames → fix random seed.
Shader tweaks break everything → separate content and texture layers.
Synthetic music sound → swap in real instrument samples.
Fix one or two issues per iteration; wholesale changes cause regressions.
Open‑Source Skill: motion‑showreel
Encapsulates all above practices into a reusable workflow for AI agents (Claude Code, etc.). GitHub:
https://github.com/ConardLi/garden-skills/Key Features
Stable output — mandatory brief (reference, narrative, memory‑point timestamp, ban‑list) before generation.
43 built‑in styles (pixel, CRT, cute 3D, synthwave, ink, Swiss, film titles) with material, palette, font, motion, sound specs and known pitfalls.
Model‑agnostic — detailed process lets weaker models approach Opus 5.5 quality.
Real‑time preview — “watch while making” UI shows frames, audio tracks, playhead; pause and steer anytime.
Forced self‑check — forward/backward frame consistency, per‑shot keyframe audit, contact‑sheet + motion report + loudness analysis; fails hard on black frames, unreadable text, missing breaths.
Workflow
Clarify requirements → Write brief → Render style frames (3 key frames) → Build shots sequentially → Compose & mix audio → Workspace preview → Export & final checksOnly three human gates: requirements, style frames, pre‑export. Rest runs autonomously.
Usage
Paste the GitHub link into your agent and say “Install motion‑showreel skill”, then describe your video need.
Video demo: https://www.bilibili.com/video/BV1f94c6uEr3/
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
