Alibaba Front‑End Intelligent Technology: PipCook, DataCook, imgcook and Future Directions

The next decade will see AI permeate every domain, reshaping how we work and live. This article shares Alibaba Front‑End’s intelligent technology practices, outlines the overall technical roadmap, and explores future UI‑intelligence trends.

Technical foundation : Data and algorithm engineering are the base for front‑end intelligence. PipCook‑Cloud bridges front‑end development with business data, providing low‑cost data analysis, modeling, and CI/CD for machine‑learning models. DataCook supplies a JavaScript‑based library for data preprocessing, statistical analysis, machine‑learning, and visualization, exposing models such as linear regression, decision trees, and PCA.

End‑device intelligence : To overcome the limitations of WebGL and older browsers, the team adopted two strategies in 2021 – WASM + Rust + SIMD and WebGPU. Rust was chosen for its experimental SIMD support, enabling native‑level performance in the browser.

Example: TensorFlow.js model conversion and inference :

model.save('saved_model/w4model')

After saving, the model produces keras_metadata.pb , saved_model.pb , and the variables and assets directories. The conversion command is:

tensorflowjs_converter --input_format=tf_saved_model \
    --output_node_names="w4model" \
    --saved_model_tags=serve ./saved_model/w4model ./web_model

The resulting model.json and group1-shard1of1.bin are loaded in a web page:

import * as tf from "@tensorflow/tfjs";
import { loadGraphModel } from "@tensorflow/tfjs-converter";
window.onload = async () => {
  const resultElement = document.getElementById("result");
  const MODEL_URL = "model.json";
  console.time("Loading of model");
  const model = await loadGraphModel(MODEL_URL);
  console.timeEnd("Loading of model");
  const test_data = tf.tensor([
    [0.0],[500.0],[1000.0],[1500.0],[2500.0],[6000.0],[8000.0],[10000.0],[12000.0]
  ]);
  tf.print(test_data);
  console.time("execute");
  let outputs = model.execute(test_data);
  console.timeEnd("execute");
  tf.print(outputs);
  resultElement.innerText = outputs.toString();
};

Enabling WebGL backend further reduces inference time from ~257 ms to ~131 ms (and down to ~78 ms after warm‑up).

Application 1 – Sophon Code IntelliSense : A front‑end code‑completion tool that brings AI‑driven suggestions to developers, reducing the need for cloud services and improving security. The team built a data‑asset pipeline (GitHub OpenAPI → OSS → ODPS → SQL tasks) to manage large code datasets for model training.

Application 2 – imgcook (Design‑to‑Code) : An AI platform that converts Sketch, Figma, PSD, or image designs into maintainable front‑end code (HTML, React, Vue, Mini‑Program, Flutter, etc.). The pipeline consists of three stages: Design‑to‑JSON (extracting node types and styles), JSON‑to‑JSON layout analysis (building DOM hierarchy, flex layout, spacing), and JSON‑to‑Code DSL conversion (supporting multiple frameworks, CSS preprocessors, style‑injection methods, and units). Recent improvements focus on handling non‑standard designs, reducing absolute positioning, and increasing flex‑layout accuracy.

Intelligent UI strategy : By decomposing UI into layout and material modules, the system can generate hundreds of UI variants and match them to user personas using recommendation models. This reduces decision time for shoppers and drives higher conversion rates across platforms such as 1688, Taobao, and Tmall.

Future outlook : The team plans to expand intelligent UI to more shopping scenarios, improve model accuracy (text/image semantics, free‑space detection), and strengthen privacy‑preserving on‑device inference (WASM + SIMD, WebGPU). Long‑term goals include a unified “AI User Interface” that eliminates traditional UI layers, enabling voice‑, vision‑, and intent‑driven interactions via a personal assistant app.

Team introduction : The group belongs to Alibaba‑Taobao’s “Big‑Taobao” technology division, focusing on front‑end AI for commerce. Their mission is “Front‑end intelligence makes business innovation more efficient.” The team consists of engineers and researchers (霸天、欣余、灝阳、苏川、继风、闻梦、缺月、妙净、甄子) and is edited by 橙子君.