From Demo to Production: How JD's Firefly AIGC Platform Built Scalable AI Infrastructure

Introduction: From a Single Request to a Production System

Many AIGC products start as a chat window where users input prompts and models return content. But when generation enters actual content production, the challenge shifts from whether one model call succeeds to whether multiple capabilities—knowledge retrieval, image understanding, model inference, copywriting, post-processing, and delivery—can be stably organized and continuously executed. Firefly is JD's AIGC platform for content production, offering content Agents, Workflows, production templates, model capabilities, and browser operations. A user request like "generate a product image-text post" triggers a chain requiring knowledge-base retrieval, image understanding, model inference, copy generation, result processing, and content delivery—each depending on different models, data, and compute resources that must be orchestrated into an executable, observable task pipeline.

Why Content Production Needs AI Infra

Scaling AIGC from single generation to mass content production surfaces four core engineering problems:

How to organize capabilities : A production job often includes knowledge retrieval, product parsing, image understanding, image generation, copy generation, and result processing. If models and algorithms remain as isolated services or ad-hoc scripts, they cannot be uniformly combined, reused, or managed.

How to execute tasks : Online requests prioritize latency; batch production prioritizes throughput; massive material processing demands resource elasticity. CPU preprocessing, GPU inference, and high-initialization-cost models have distinct runtime characteristics—a single execution model cannot cover all scenarios.

How data supports model quality : Output quality depends not only on the model but also on material quality, feature completeness, and retrieval of suitable multimodal assets. Raw multimodal data must be processed, governed, and settled before becoming production-ready inputs.

How to observe long pipelines : A task comprises multiple operators. When failures occur, the platform must know execution progress, which operator failed, and the associated logs and root causes.

Firefly's AI Infra therefore focuses on how models, data, tasks, and compute collaborate to form a stable content production system—unifying capability organization, selecting execution engines per task profile, turning raw multimodal assets into searchable data, and establishing end-to-end observability.

Overall Architecture: Unified Orchestration, Differentiated Execution

The architecture revolves around two coordinated pipelines:

Content production pipeline : Business side (Agents, Workflows, templates) selects and combines AIGC capabilities; platform layer parses them into executable Pipeline/DAGs and schedules to Ray Serve, Ray Data, or Spark/Flink based on task characteristics.

Multimodal data processing pipeline : Raw images/videos enter Lance multimodal tables; elastic compute performs inference tagging, content understanding, feature processing, and cleaning; results, tags, and features are written back to the lakehouse, and qualified materials enter image/video knowledge bases for retrieval during production.

Trace horizontally connects the platform layer and execution engines, using DAG, task IDs, operator IDs, and logs to reconstruct actual execution.

Platform Layer: From Business Workflow to Executable DAG

Business users compose goal-oriented Workflows; execution engines need schedulable operators and dependencies. The platform layer sits between them, converting business capability descriptions into real execution plans.

4.1 Operator Management: Turning Algorithmic Capabilities into Platform Execution Units

An operator is the smallest schedulable unit—image processing logic, a model inference capability, or a standardized external service call. Operator management uniformly constrains inputs/outputs, resource declarations, versions, business parameters, and dependency environments so upper-layer Workflows need not know algorithm internals. Decoupling model weights, business parameters, runtime configs, and core code lets the same capability upgrade independently or enter different runtimes per task type. Operators become platform execution units with contracts, versions, and resource semantics—not one-off scripts.

4.2 Pipeline/DAG: Compiling Capabilities into Execution Plans

The platform parses selected Workflow capabilities into a Pipeline/DAG describing operator dependencies, execution order, and data flow: dependent nodes run sequentially; independent nodes run in parallel. Upstream/downstream data passes via input-output mapping—downstream operators fetch only needed fields, outputs can be filtered, renamed, and mapped, avoiding direct coupling between algorithm implementations and unbounded intermediate data propagation.

4.3 Task Management: From Execution Plan to Runtime

After DAG generation, the platform creates a task linked to the Workflow, operator versions, business config, and resource requirements. Task management provides submission, runtime config, and status viewing; a unified orchestration DSL describes tasks and selects execution backends. The platform unifies task, operator, and data-flow semantics while runtimes stay differentiated: online model serving, offline batch inference, and multimodal data processing go to Ray Serve, Ray Data, and elastic compute respectively.

Execution System: Real-time, Batch, and Elastic

Different AIGC tasks have divergent latency, throughput, model lifecycle, and compute needs. Firefly uses a single task semantics but routes tasks to real-time, batch, or elastic execution paths based on characteristics.

5.1 Why Ray

Ray provides three key abstractions—Task, Actor, and service-model instances (Ray Serve). Stateless compute runs as Tasks; stateful or model-reuse compute uses Actors; online models run as Serve replicas. These match AI tasks' high model initialization cost, long runtimes, and complex resource types. For Python-centric model/algorithm operators, Ray offers unified resource semantics: operators declare CPU, GPU, memory, concurrency, and machine labels; scheduling places image preprocessing, text processing, and GPU inference on appropriate nodes. Online and offline tasks share one distributed execution system but use Ray Serve and Ray Data runtimes respectively. Ray's value is not just single-inference performance but organizing model serving, service replicas, resource scheduling, and distributed data processing in one system, giving upper pipelines a consistent execution interface.

5.2 Ray Serve: Managing Online Model Service Lifecycle

Real-time tasks are scheduled by a custom engine and invoke Ray Serve-deployed operators via HTTP. Serve provides three capabilities:

Model residency and service replicas : Models/operators pre-load in resident instances, avoiding per-request re-initialization. Each service independently configures replica count, CPU/GPU resources, and concurrency; scaling one model doesn't require scaling the whole Workflow.

Rolling deployment and auto-scaling : New versions roll out gradually to reduce online impact; auto-scaling adjusts replicas to request peaks and valleys.

Decoupling platform orchestration from model runtime : Custom scheduler handles task orchestration and request dispatch; Ray Serve handles model loading, replicas, request routing, and runtime resources. They connect via stable HTTP interfaces, leaving room for multi-model collaboration and complex Workflows.

5.3 Ray Data: Offline Batch Processing for Model Inference

Batch content production uses Ray Data for offline inference and distributed data processing. Input data is organized as Datasets, distributed across model instances; read, inference, and write form a continuous data pipeline. This suits large-scale batch inference with Python models better than per-item calls.

Model instance reuse : Operators run as long-lived Actors; models load once at Actor init and process multiple batches. An Actor Pool maintains reusable instances, cutting repeated initialization cost for image/multimodal models.

Batch inference and resource coordination : map_batches sends a batch to model instances, reducing per-item scheduling overhead and aligning with GPU execution. Each operator independently configures batch size, concurrency, CPU/GPU, memory, and machine labels, balancing throughput, VRAM usage, and isolation.

Pipelined parallel execution : Data read, model processing, and result write advance in parallel as data flows; independent branches (e.g., image understanding, attribute extraction) run in parallel and merge results.

Ray Data transforms offline inference from "loop-calling models" to "distributed execution over datasets and model instances."

5.4 Elastic Compute

Elastic compute handles massive multimodal data lakehouse processing—image/video cleaning, inference tagging, content understanding, feature processing. These tasks exhibit phase-specific compute peaks ; isolating them in elastic resources prevents large-scale data processing from impacting online content generation and daily batch production. The core design principle: unified task semantics, differentiated execution —platform layer uniformly describes tasks; execution engines each optimize for online latency, batch throughput, or data scale.

Multimodal Data Lakehouse: From Asset Governance to Data Flywheel

Model effectiveness depends on input accuracy, material relevance, and context completeness. For content production, raw images/videos must undergo understanding, tagging, feature extraction, and quality cleaning before becoming searchable, reusable knowledge-base assets.

6.1 Why Lance

Multimodal data differs from traditional analytics data. One asset record may link image/video, structured metadata, model tags, understanding results, and feature vectors; as algorithms iterate, new tags, features, and scoring columns continuously accrue. Lance, designed for AI data access, unifies images, videos, structured metadata, inference tags, understanding results, features, and vectors in one tabular format. Three factors drove its selection:

Columnar organization and on-demand reads : Metadata, tags, understanding results, features, and vectors stored by column; processing tasks read only needed fields, suiting wide multimodal tables and ever-growing feature columns.

Unified association of assets and inference results : Raw assets, inference results, tags, and vectors link in one data organization. New features or processing results extend around the same asset data, reducing cross-system copying and joining.

Vector search and fast point lookup : Filter by tags/attributes or vector-search for semantically similar images/videos; fast point reads of hit records support visual inspection and knowledge-base construction via a unified access path.

Lance tables thus serve as the unified carrier for multimodal assets and the shared data entry for batch processing, visual inspection, and knowledge-base building.

6.2 Asset Cleaning: From Lake Ingestion to Knowledge Base

The lakehouse centers on a main pipeline: asset ingestion → elastic processing → result settlement → visual inspection → knowledge base. A visualization platform provides a side-channel into Lance tables for data observation and manual operation. Business, algorithm, and operations staff view datasets, inference results, tags, and features; filter by tags, attributes, processing results; vector-search for related data; inspect cleaning outcomes, observe distributions, spot anomalies. Qualified assets enter image/video knowledge bases; production tasks retrieve models, on-body images, and video assets from there. Front-loading high-cost image/video understanding and feature extraction lets production directly use governed multimodal assets, reducing ad-hoc processing pressure in online Workflows.

6.3 Data Flywheel: From Data Settlement to Model/Operator Iteration

Current capabilities cover asset ingestion, elastic processing, result settlement, visual inspection, and knowledge-base construction. The next goal is to gradually settle inference results and problem samples from content production into a complete data flywheel: data settlement → problem analysis → model iteration. This connects production's consumption of model capabilities with accumulation of data for the next optimization round. As this matures, the multimodal lakehouse evolves from an asset governance space into a data asset space serving retrieval, evaluation, training, and Agent context simultaneously.

Trace: Making Workflow Execution Visible

A content production pipeline may contain multiple sequentially dependent or parallel operators. Scattered logs make it hard to quickly determine execution progress and failure location. Trace links DAG structure, tasks, operators, and logs into a workflow-level execution view.

7.1 Task, Operator, and Log Association Mechanism

Trace uses unified identifiers to stitch one task's execution: task_id marks a complete task; DAG provides workflow structure; operator_id maps to specific operators in the flow; trace_id associates operators with actual runtime logs. From task overview → DAG → specific node → drill down to logs, without manual cross-system ID matching.

7.2 What Can Be Observed

Trace shows whether tasks and operators executed, success/failure status, and on failure pinpoints the specific operator, error context, and linked logs. It serves business users tracking progress and algorithm/platform engineers understanding long-pipeline runtime behavior. On this foundation, future work can add automatic task retry and intermediate result recovery.

Summary: Let Models, Data, and Tasks Collaborate in One System

Firefly AI Infra forms a clear technical chain for multimodal content production: platform layer converts business Workflows to executable DAGs; Ray Serve, Ray Data, and elastic compute route tasks to suitable runtimes; Lance multimodal lakehouse turns raw assets into searchable knowledge-base data; Trace connects tasks, operators, and logs into a complete execution view. The key is not stacking components but making unified orchestration, differentiated execution, upfront data governance, and workflow-level observability collaborate. Business continuously combines new AIGC capabilities; platform standardizes them into tasks; execution engines allocate resources per task profile; multimodal data continuously settles and evolves; task execution becomes observable and debuggable. Thus model capabilities move from point calls to continuous, stable content production and delivery.