How to Build and Deploy a Scalable Open‑Source Decision Engine with Go and YAML

Decision Engine Overview

Decision engines are core components for scenarios such as financial risk control, content moderation, recommendation, and IoT monitoring, where business decisions are driven by rules or models. Compared with a pure rule engine, a decision engine adds flow orchestration, enabling multi‑step and conditional decision paths.

Implementation Principles

Rule Construction

Rules are defined as a list, optionally with a priority field. Simple engines evaluate each rule sequentially; more advanced engines may respect explicit ordering.

Decision Flow Orchestration

A decision flow arranges rule nodes, decision nodes, and branch nodes into a directed execution graph, allowing complex, conditional processing.

DSL Based on YAML

The engine uses a custom domain‑specific language (DSL) expressed in YAML to describe rules and flows. A rule condition consists of a feature, an operator, and a threshold. Multiple conditions are combined with logical operators (AND/OR) to produce a final boolean outcome (e.g., Reject).

Loan amount > 5000

Number of loans < 3

Monthly salary <= 8000

(Expression1 OR Expression2) AND Expression3

Operator Implementation

Operators must support various data types:

Numeric: >, <, =, !=, >=, <=, range String: equality, fuzzy match

Array: exact match

Logical: and, or Arithmetic: +, -, *, /, % Open‑source libraries such as govaluate (Go) and SpEL (Java) can be leveraged.

Reference implementation (operator package): https://github.com/skyhackvip/risk_engine/tree/master/internal/operator

Rule Combination

Rules can be grouped into rule sets, decision trees, or decision tables. A rule set executes all contained rules independently; parallel execution is achieved with Go goroutines and a sync.WaitGroup. Conflicts are resolved by assigning priorities to rule outcomes.

exec_plan=parallel

Decision Flow Implementation Options

DAG (directed acyclic graph) – preserves dependencies but traverses every node.

Rete network – used by Drools for efficient pattern matching; complex to implement.

Improved singly‑linked list – linear execution for non‑branch flows; branch nodes are flattened via pruning, and a global context controls flow and circuit‑break logic.

The chosen solution uses a global PipelineContext (a “global workbench”) to share features and intermediate results among nodes, enabling decoupled orchestration and circuit‑break handling.

Key interfaces and structures: INode – each node implements Parse() for DSL parsing. Run() – traverses the linked list, returning the next node and a break flag. PipelineContext – stores feature dependencies, intermediate results, execution metadata, and supports circuit‑break decisions.

Reference flow implementation: https://github.com/skyhackvip/risk_engine/blob/master/core/flow.go

Open‑Source Engine Practice

The “TianWang” decision engine is a free, Go‑based project that provides:

A YAML‑based DSL for rule and flow definition.

HTTP APIs for easy integration.

Full decision trace output for downstream consumption.

Production‑grade considerations include:

Building a visual management console for non‑technical users to edit rules and draw flows.

Integrating upstream feature services (e.g., risk data platforms, third‑party credit APIs) in push or pull mode.

Downstream consumption such as risk dashboards or real‑time alerts.

Support for circuit‑break to avoid unnecessary external calls when a rule already determines the outcome.

Repository: https://github.com/skyhackvip/risk_engine