Exploring UI Design‑to‑Code Automation: Practices from Meituan, Xianyu, Microsoft and Large‑Model Flutter Generation

With the rapid development of artificial intelligence and large‑model technologies, the automatic generation of code from UI design drafts (UI views) has become increasingly feasible. This article reviews the explorations and practices of several well‑known companies—including Meituan, Xianyu, Microsoft, and JD—as well as common tools and methods, and finally shares experiences of using large models to generate Flutter code in a project.

1. Meituan's Exploration

In March 2021 Meituan published an article describing two technical solutions, sketch2json and imgCook . sketch2json parses Sketch files, extracts layer information, converts it into a DSL and generates code, while imgCook relies on AI techniques. At that time large‑model capabilities were immature; sketch2json used a simple algorithm with low accuracy for complex layouts and limited interpretability, making further optimization difficult.

2. Xianyu's UI2CODE Design Idea

Xianyu divides the entire engineering structure into five parts, with four core sections handling machine‑vision problems. The pipeline links these parts through machine learning, but because machine‑learning solutions are probabilistic and hard to meet strict precision requirements, the system emphasizes machine‑vision understanding as the primary approach, using machine learning as a supplement.

3. Microsoft's Sketch2Code

Microsoft launched a project called Sketch2Code that aims to generate web code from hand‑drawn sketches. Although the concept is novel, it still depends on extensive research. Microsoft stresses that sketch‑to‑code does not mean AI can generate complex logic from arbitrary images; product managers must still translate requirements into designs, and engineers implement the code. Large models such as GPT‑4 can assist in the design‑to‑code step but cannot replace human involvement in requirement analysis and design.

4. Common Design‑to‑Code Tools

Various tools help designers and developers convert designs into code, including Uizard, Adobe XD plugins (Export Kit, Anima), Figma plugins (HTML Generator, Figma to Code), Zeplin, Avocode, Sketch2React, and CodeMyUI.

5. Flutter Code Generation Tools

Generating Flutter code directly from design drafts is challenging, but several tools simplify the workflow: Supernova, Flutter Studio, Parabeac, Figma to Flutter, Adobe XD to Flutter, Sketch2Flutter, and Draftbit. The typical process involves design creation, exporting the design via a plugin or tool, automatic code generation, and manual adjustments as needed.

6. JD's Ling Platform

JD's Ling platform can generate H5 and Android layout code from design drafts, but it does not yet support Flutter code generation. Its main goal is to streamline development and improve efficiency.

7. Large‑Model Generated Flutter Experiments and Summary

Experiments were conducted by feeding historical UI design images into the Ling platform to produce web front‑end code, then feeding that code into a large model to generate Flutter code. The resulting UI displayed all elements with high fidelity in fonts and colors, though positional deviations were noticeable because the original design hierarchy was free‑form and not well‑structured, limiting the model's understanding of layout relationships.

To improve results, the team communicated with UI designers to produce structured and componentized design drafts, arranging elements sequentially and defining reusable components (e.g., header card, waybill card). This structured approach helped the model generate more accurate Flutter code.