How Frontend Code Was Auto-Generated for Alibaba’s Double‑11 Event

Overview

As one of the four technical directions of Alibaba's Frontend Committee, the Frontend Intelligent Project successfully handled the 2019 Double‑11 peak, with 79.34% of the new modules' code automatically generated. This series shares the technologies and insights behind the project.

Business Module Recognition

Large‑scale promotional pages contain many business modules—code units with specific functions. By extracting module information from visual drafts (DSL), we can reuse code and bind business fields. Unlike backend component or form recognition, this focuses on wireless pages and leverages rich visual information (text, image sizes) from the DSL.

The recognition pipeline returns the module category and its position in the draft.

Overall Function

Sample construction: augment UI layers of visual drafts based on user configuration and custom data‑augmentation rules, then extract and store predefined features.

Algorithm selection: use traditional multi‑class machine‑learning algorithms.

Model implementation: build and deploy models on the corporate ML platform for automated training.

Interface provision: expose prediction and feedback services.

Sample Construction

Training requires abundant real data. Since each Sketch module may have only a few drafts, we solve the quantity issue with data augmentation. Users can configure augmentation rules (e.g., hideability, text length range) to generate diverse DSL samples.

Augmented DSLs are combined to produce a large number of samples.

Configuration interface includes augmentation attributes (size, position, visibility, colors, content) and methods (continuous range, enumerated values).

Feature Extraction

From the augmented DSLs we extract features to form a tabular dataset for traditional ML. Visual features include width, height, layout direction, image count, text count, etc., yielding over 40 dimensions. Additionally, we add business features (e.g., price, popularity) based on custom rules, resulting in a feature vector plus a label, i.e., a sample.

Algorithm and Model

The task is a multi‑class classification problem: identify the business module from a standardized DSL.

Random Forest

We initially chose Random Forest for its speed and low preprocessing requirements. It automatically handles continuous and discrete variables, but suffers from low confidence on some samples, difficulty with out‑of‑distribution (OOD) data, and discrete‑feature limitations.

We tuned parameters (sample count per tree, max depth, tree type ratios) and performed feature‑importance feedback, but the process could not be fully automated.

XGBoost

Switching to XGBoost improved accuracy. However, XGBoost requires preprocessing: label encoding to consecutive integers, storing label‑mapping tables, and data reorganization to avoid missing labels.

With XGBoost, confidence thresholds and logical checks (e.g., size and hierarchy filters) effectively reduce OOD errors.

Out‑of‑Distribution Handling

Since we cannot collect all possible negative samples, OOD data leads to false positives. We mitigate this by:

Setting a confidence threshold on model outputs.

Applying logical rules (e.g., disallowing duplicate components on the same DSL path).

Providing a feedback service where users upload mis‑identified DSLs, which are turned into negative samples for retraining.

Model Deployment

The algorithm platform deploys the model as an online prediction service accessible via the imgcook platform, enabling one‑click invocation and automated training‑deployment pipelines.

Prediction and Feedback

Prediction input is the DSL (JSON) extracted from a design draft; output includes module ID and position. Pre‑prediction filters exclude modules with large size deviations or leaf nodes (pure text/image). Feedback loops allow users to upload incorrect predictions, generating negative samples for further training.

Future Work

Planned improvements include:

Optimizing loss functions in DNNs to enlarge inter‑class distances and compress intra‑class distances for OOD detection.

Automating negative‑sample generation with a pre‑stage binary classifier.

Exploring deep‑learning approaches (CNNs) to extract UI feature vectors directly from images.

Enhancing the sample platform: richer configuration, support for HTML/code sources, and intelligent data‑augmentation to reduce user effort.

These efforts aim to further boost automation and accuracy in frontend code generation.