How Karpathy Built a LOTR 3D Scene in Two Hours for $10

Karpathy fed the opening text of The Lord of the Rings to the Opus 5 model, allocating about one million tokens and a cost of roughly US$10, and issued a single requirement: use Three.js to turn the story segment into an interactive 3D scene.

The model spent nearly two hours generating approximately 5,500 lines of code. It first parsed the narrative to understand the setting and plot, then created polygonal assets such as characters, houses, and trees, positioning each with explicit x, y, z coordinates. After asset placement, it planned camera angles, designed character motions, and wrote animation scripts so the scene would follow the story’s sequence.

Although the resulting experience is far from polished—some scenes appear abstract and many assets are misaligned or colliding—Karpathy noted the output is still entertaining. He emphasized that a task which previously required weeks of manual coding by a team can now be produced for a few dollars.

Community members have built enhanced versions using different harnesses, and even a Harry‑Potter variant, demonstrating the experiment’s extensibility.

Karpathy highlighted a key limitation: while the model can write code to create games, it cannot efficiently evaluate its own visual output because current large models lack native video perception. In this experiment Opus 5 could only capture static screenshots at various stages and adjust them manually, a process that is time‑consuming and prone to errors, leaving noticeable misplacements and rough details.

Despite these shortcomings, the ability to generate highly customized 3D content on demand for under $10 suggests a future where users could request AI‑generated virtual worlds—from novel scenes to historical events—instantly, akin to a temporary, on‑demand GTA.