Design and Implementation of the Comet Workflow Engine at Bilibili

The article introduces the Comet workflow engine, a business‑process platform developed by Bilibili since 2019. It explains the background of rapid technology updates in enterprises and the need for a flexible, generic workflow engine to improve operational efficiency.

Comet supports both mobile H5 and web access and provides rich approval capabilities such as add‑sign, transfer‑sign, parallel‑sign, and optional‑sign. It offers open‑ended forms and plugins for page rendering and external system integration, and uses graph algorithms to validate workflow configurations.

Several practical scenarios are described:

SRE automation – cache resource change requests are turned into self‑service tickets, reducing manual effort.

Permission request – data‑platform table permissions are approved based on context, with dynamic node creation and multi‑group approval.

Business approval – push‑task approvals use a dynamic linear flow that generates nodes on‑the‑fly based on the task’s context.

The design section covers three layers of the system:

Management layer – handles template, plugin, authentication, and graph legality checks.

Engine layer – implements core functions such as start, transition, reject, close, add‑sign, and transfer‑sign.

Asynchronous execution layer – processes events generated by the engine.

Key design concepts include:

Workflow templates – visual, drag‑and‑drop modeling of process nodes, similar to class‑object relationships in programming.

Node logical attributes – AND/OR logic for nodes with multiple incoming edges.

Transition conditions – stored as edge attributes, expressed as MongoDB‑style JSON expressions (e.g., {"$or":[{"m":{">":1}},{"n":{">":2}]} ).

Form templates – both composite (low‑code) and HTML templates that can embed custom HTML/JS/CSS via iframes.

Visualization – Mermaid diagrams can render the workflow graph from node IDs and conditions.

Engine behavior is illustrated with pseudo‑code for node transition, rejection, and status determination. The article also describes the asynchronous execution model, where plugins (binary files or Python scripts) are invoked before or after node processing, and callbacks allow external systems to report success or failure.

Legality detection relies on graph theory. The workflow must be a directed acyclic graph (DAG). The article explains adjacency matrices, the concept of start (in‑degree 0) and end (out‑degree 0) nodes, and presents cycle‑detection algorithms such as Kahn’s algorithm and depth‑first search, with a Go implementation example.

In summary, the Comet engine demonstrates how a low‑code, plugin‑extensible workflow platform can automate diverse business processes, reduce development cost, and ensure reliable execution across Bilibili’s infrastructure, data platform, and push services.