Translating UI DSL to Flutter Code: Architecture and Algorithms

Background: After image analysis, UI2CODE produces a DSL layout description in JSON, which is then used by an IntelliJ plugin to generate corresponding Flutter code.

DSL Definition: The DSL is a domain‑specific language that abstracts image and layout recognition results into a structured JSON, describing each layer’s frame, type, style, and value.

Core Idea: Convert the DSL tree into a Flutter widget tree by first establishing relationships among layers (containment, overlap, grouping) and then mapping those relationships to Flutter widgets.

Step 1 – DSL Tree Construction: Layers are grouped based on containment (Shape), block components (Block), repeated layouts (Repeat), and other custom tags. Overlapping and nested relationships are resolved to form a hierarchical DSL tree.

Step 2 – Flutter Tree Construction: The algorithm removes outer padding, then performs horizontal (Row) and vertical (Column) cuts to partition the layout. When a cut is not possible, an absolute layout (Stack) is used, with special handling for text layers via an original frame (oriFrame) to avoid incorrect stacking.

Text Handling: Text widgets are examined for multi‑line or fixed‑width content, padding distribution, and stretchability. A priority system determines which widget receives a Flex property, and alignment (left, center, right) is inferred using an unsupervised distance‑based algorithm.

Alignment Optimization: For Containers, Rows, and Columns, internal padding is analyzed to set appropriate crossAxisAlignment and mainAxisAlignment, improving the aesthetic of the generated code.

Code Generation: Two node types – View and Layout – are defined. Each node provides a code template; for example:

new Column(
    #{alignment},
    children:[
        #{children},
    ]
),

The tree is traversed from the root, assembling the final Flutter widget code.

Data Separation: Generated code and data models are split into separate sections, allowing dynamic data injection and reducing code complexity.

Overall Architecture: The system combines static image analysis, DSL parsing, Flutter tree building, and a factory pattern for post‑processing optimizations.

Business Integration: Deployed internally at Xianyu, UI2CODE automates UI development, reducing effort by about two‑thirds and promoting code consistency across teams.

Future Plans: Continue improving accuracy, explore NLP and AST techniques, and open the tool to the broader Flutter community.