From Simple Chain to Complex Workflow Engine: 15 Design Challenges Solved

Stage 1

A boss asks for a simple workflow engine. The author researches workflows and builds a basic version: a linked list of approvers ending with a terminal node, tracking the current approver and moving to the next after approval, ending when the terminal node is reached.

Add any number of approvers in order to form a linked list, then add a terminal node.

Record the current approver; after approval, move to the next approver.

When the approver reaches the terminal node, the process ends.

Boss: "A bit crude."

Stage 2

The boss now wants support for countersign (parallel) nodes. After researching, the author redesigns the engine using a tree structure.

Adjustments:

Two node types: simple (rectangle) and complex (circle).

The whole process is a tree; leaf nodes are simple nodes.

Each simple node has exactly one approver.

Complex nodes contain multiple child nodes.

Countersign node: activates all child nodes; when all are approved, the node completes.

Serial node: child nodes must be approved left‑to‑right; when the last child is approved, the node completes.

The outermost node of any workflow is a serial node; its completion marks the workflow’s end.

Node states introduced:

Ready : simple node ready for approval.

Complete : node already approved.

Future : node not yet reached.

Waiting : complex node waiting for child approvals.

Example workflow with a countersign node is illustrated with two diagrams.

Stage 3

The boss requests parallel nodes, where any child’s approval completes the node.

Parallel node is a complex node; when any child becomes ready and then complete, the parallel node completes.

New state Skip added: when a parallel node’s child is in a non‑Ready/Waiting state, all sibling nodes and their descendants are set to Skip.

Illustrated with an example diagram.

Stage 4

Support for nested nodes (e.g., a countersign node containing a parallel node, which in turn contains a complex node) is added, enabling unlimited nesting via a tree structure.

Boss: "You’ve got something!"

Stage 5

Conditional nodes are introduced: a workflow carries a form, and the next branch depends on form values.

Conditional node works like a parallel node but only child nodes whose conditions are satisfied become eligible for approval.

Stage 6

Three types of simple approvers are added:

Hard‑coded approver.

Approver read from the form.

Approver derived from the initiator via a mapping function (e.g., get_manager("Qian") → "Li").

Stage 7

Rollback to the initiator is implemented: only Ready nodes can trigger a rollback, effectively restarting the workflow.

Stage 8

Rollback to the previous approver is added, handling complex nested structures to determine the correct prior approvers.

Stage 9

Rollback to any arbitrary node is achieved by repeatedly rolling back one level until a Ready node containing the target node is found.

Stage 10

Time limits are added to ordinary nodes; if a node is not completed within the allotted time, it is marked as timed‑out.

Stage 11

Proxy (delegation) functionality is added: when an approver delegates, a new parallel node is created as the parent, and a sibling node holds the proxy approver, allowing both to approve. Delegation can be nested indefinitely.

Stage 12

Cancel‑proxy is implemented as the inverse of proxy; it cannot be performed if the proxy approver has already acted.

Stage 13

Pre‑ and post‑conditions are added to each node: a node can only be entered when its pre‑condition is satisfied, and it can only be marked complete when its post‑condition is satisfied.

Stage 14

A progress metric is introduced to show the percentage of workflow completion, calculated as the distance from the leftmost node to the rightmost Ready node divided by the total distance to the rightmost node.

Stage 15

Each node can now attach two executable scripts: one runs when the node’s approval starts, and another runs after the node’s approval completes.

Author’s reflection: after implementing all these features, the author’s hair fell out, but the workflow system was eventually sold to major securities firms.