How Top Tech Companies Turn UI Designs into Ready-to-Run Code with AI

One, Meituan's Exploration

In March 2021 Meituan published an article on automatically generating code from design drafts, discussing two approaches: sketch2json , which parses Sketch files into a DSL to produce code, and imgCook , an AI‑based solution. At that time large‑model AI was immature, so sketch2json’s algorithm was simple and struggled with complex layouts, resulting in low accuracy and limited explainability.

Two, Xianyu's UI2CODE Design Idea

Xianyu divides the engineering system into five parts, with four core components focusing on machine‑vision problems. By linking these components with machine‑learning, they build a UI2CODE pipeline. Because machine‑learning solutions are probabilistic and hard to meet strict production precision, Xianyu adopts a vision‑first, learning‑as‑assistant strategy for the UI2CODE framework.

Three, Microsoft's Sketch2Code

Microsoft launched Sketch2Code, aiming to generate web code from hand‑drawn sketches. While the concept is novel, it still relies on extensive research. Microsoft stresses that generating code from sketches does not mean AI can produce complex logic from arbitrary images; product managers must still translate requirements into designs, and engineers implement the code. Large models like GPT‑4 can assist in the design‑to‑code step but cannot replace human input in requirement analysis and design.

Additional Tools and Methods

Uizard – converts hand‑drawn sketches and design images into digital prototypes and HTML/CSS code.

Adobe XD + Plugins (e.g., Export Kit, Anima) – export designs to HTML, CSS, and JavaScript.

Figma + Plugins (e.g., HTML Generator, Figma to Code) – similar to Adobe XD.

Zeplin – allows developers to extract CSS and design specifications from uploaded designs.

Avocode – supports Sketch, XD, Photoshop, etc., generating HTML and CSS.

Sketch2React – converts Sketch designs into React components.

CodeMyUI – provides UI design and code examples for reference.

Four, Flutter Code Generation Tools

Supernova – exports Flutter code from Sketch, Adobe XD, or Figma.

Flutter Studio – online tool that converts designs to Flutter code.

Parabeac – open‑source tool for converting Sketch/Figma designs to Flutter.

Figma to Flutter – Figma plugin for Flutter code generation.

Adobe XD to Flutter – plugin to export Flutter code from Adobe XD.

Sketch2Flutter – converts Sketch designs to Flutter.

Draftbit – low‑code platform supporting design‑to‑Flutter conversion.

The typical workflow includes: 1) Design phase – create design drafts in a design tool; 2) Export design – use appropriate plugins or tools; 3) Generate code – automatically convert the exported design; 4) Manual adjustment – fine‑tune the generated code as needed.

Five, JD's Ling Platform

JD's Ling platform can generate H5 and Android layout code from design drafts, but it does not yet support Flutter code generation. Its main goal is to streamline development and improve efficiency.

Six, Large‑Model Generated Flutter Exploration and Experience Summary

1) Using historical UI designs

Importing UI design drafts into the Ling platform can produce web front‑end code.

2) Feeding generated web code to a large model to obtain Flutter code

The resulting UI appears complete, with high fidelity in fonts and colors, but positional offsets are noticeable because the original design lacks a structured hierarchy, which large models struggle to interpret accurately.

3) Structuring and componentizing prototype designs

After discussing with UI designers, the team began structuring UI drafts, arranging elements sequentially, and componentizing them. Example component diagrams are shown below.

Component 1: Header Card

Component 2: Waybill Card