Runway Turns World Model into an OS: Generate Interfaces Without Code

Historically, AI‑assisted software development follows a pipeline where natural language prompts produce code, which is then executed to render webpages or apps. The code layer remains a fixed bridge between a designer’s intent and the final interface.

Solaris, introduced by Runway, proposes a different path: an Interface World Model that generates UI frames directly, much like video generation, removing the intermediate code step. Users can click, drag, or type a prompt, and the model continues generating the next visual frame, making the interface itself a dynamic, executable artifact.

Technically, Solaris builds on Runway’s Gen‑4.5 video generation model and the earlier GWM‑1 world‑model architecture. Runway applied three key modifications: (1) converting generation to a frame‑wise autoregressive process so each frame depends only on previously generated content; (2) distilling the multi‑step denoising process into a few steps; and (3) using the accelerated model to train on its own outputs, reducing drift over long sequences. A language model first interprets user intent, decides whether a local change or a scene transition is needed, and produces a textual prompt for the world model, which then renders the visual update.

This design lets the system treat user actions (clicks, drags, text input) as conditioning variables for the next frame, bypassing the need for explicit code such as “increase X coordinate by N”. Because the model predicts the resulting visual state directly, it can respond in real time, unlike traditional video diffusion models that require dozens of denoising steps and are too slow for interactive use.

Runway evaluated Solaris with two experiment suites. The first measured “translation loss” by asking multimodal models (including Claude Fable 5) to reconstruct 30 interfaces of varying complexity from a single screenshot, using SSIM for structural similarity and DINOv3 features for visual fidelity. Results showed that as interface complexity grew, code‑based reconstruction suffered increasing information loss, especially on natural images. The second suite compared Solaris against code‑generated UIs derived from Claude Opus 5. Starting from the same image, both systems received identical interaction requests; 250 participants performed ~7,500 pairwise judgments across 30 interaction scenarios. Solaris was preferred 61 % of the time for following interaction instructions (vs 24 % for code) and 71 % for naturalness of behavior (vs 21 % for code).

Beyond end‑user UI creation, Solaris offers a novel training environment for computer‑use agents. Because the interface can be regenerated with endless layout variations, agents can learn to generalize across diverse visual configurations rather than memorizing fixed button positions, addressing a key limitation of current agent training on static websites.

Overall, Solaris demonstrates that a video‑generation backbone combined with language‑model reasoning can act as a real‑time, interactive UI simulator, reducing the information loss inherent in code translation and opening new possibilities for AI‑driven software development and agent training.