StarRocks Vectorized Execution & CBO Optimizer Deep Dive: Pipeline, CodeGen & Cost Model

1. Problem Background and Technical Evolution

The value of real-time lakehouse ultimately depends on query performance: users expect sub-second dashboard responses. Traditional MPP databases face two key issues: (1) row-by-row execution overhead from frequent function calls, virtual dispatch, and branch mispredictions; (2) rigid binding between threads, execution tasks, and CPU resources, limiting concurrency control flexibility.

StarRocks addresses these through three pillars: vectorized execution (batch columnar processing), Pipeline scheduling (decomposing plans into schedulable Pipeline/Driver units with pipeline_dop and query queues for parallelism control), and CBO optimizer (using statistics and cost models for physical plan selection). These mechanisms work together, not as isolated parameters.

From a source-code perspective, the stack comprises three layers: FE for plan generation/optimization, BE/CN for execution, Pipeline Engine for converting physical plans into Drivers, and vectorized operators for Chunk processing. Production tuning adds a fourth layer: parameters ( vector_chunk_size, pipeline_dop, memory limits, query queues, resource groups) and Query Profile for validation.

2. Key Takeaways

Why vectorized execution beats row-by-row for OLAP: columnar Chunks, batch processing, cache/SIMD friendliness, and the vector_chunk_size parameter.

Pipeline scheduling model: Fragment, Pipeline, PipelineDriver, Operator, and the roles of pipeline_dop and max_pipeline_dop.

CodeGen/JIT: which expression evaluation overheads it eliminates at runtime, and why not all SQL benefits.

CBO internals: how statistics (row count, NDV, histograms) and CPU/Memory/Network/I/O cost models drive join reordering and physical plan selection.

End-to-end SQL lifecycle: logical plan → physical plan → BE/CN execution.

Tuning logic behind vector_chunk_size, pipeline_dop, max_pipeline_dop, memory limits, Query Queue, Resource Group.

Troubleshooting methodology for slow queries, CBO plan errors, BE OOM, and external lake table slowness from source-code and parameter perspectives.

Interview-ready explanation: StarRocks speed = storage pruning + vectorized execution + Pipeline parallelism + CBO optimization, not just "columnar storage".

3. Core Concepts and Architecture

FE handles metadata, SQL parsing, optimization, plan generation, and scheduling. Compute execution runs on traditional BE nodes (storage-compute coupled) or CN nodes (Shared-Data architecture). A SQL lifecycle: FE parses SQL → logical plan → rule-based + cost-based optimization → physical plan → Fragment/FragmentInstance distribution → BE/CN Pipeline Engine builds PipelineDrivers → vectorized operators process Chunks → results aggregated via Exchange.

Execution side breaks into four layers: Plan layer (FE produces Fragments/Instances), Pipeline layer (BE/CN converts to Pipelines/Driver instances), Operator layer (Scan, Filter, Project, HashJoin, Aggregate, Sort, Exchange), Chunk layer (columnar batch data flow between operators). Query Profile mirrors this hierarchy; troubleshooting must drill down to specific Fragment, Pipeline, Driver, or Operator consuming CPU/memory/I/O.

For lakehouse queries, StarRocks reads internal tables and external lake tables (Paimon/Iceberg) via External Catalog. External scans incur extra costs: object storage/HDFS access, file enumeration, partition pruning, predicate pushdown, type conversion. "Lake tables can be fast" requires co-optimization of scan and execution layers, not treating external tables as local OLAP tables.

4. Deep Dive into Core Principles

4.1 Vectorized Execution: Why Batch Columns Beat Single Rows

Classic Volcano model uses next() per-row pulls — simple and flexible but incurs high per-row function call, virtual dispatch, and branch overhead. Vectorized execution passes batches of columnar Chunks (typically thousands of rows) with contiguous per-column memory. Benefits: (1) amortized function call/loop overhead; (2) CPU cache locality and SIMD parallelism; (3) columnar storage reads only needed columns, reducing I/O and memory bandwidth.

Key parameter: vector_chunk_size default 4096 rows . It controls batch size — too small increases scheduling overhead; too large raises per-batch memory and small-query latency. Not "larger is faster"; keep default unless Profile confirms batch size bottleneck.

Source-code focus: Chunk/Column, vectorized operators, Expr/ExpressionContext. Vectorization is not a single class but an execution style from Scan through Filter, Project, Join, Aggregate, Sort built around Chunk batching.

Design rationale: analytical workloads = "scan many rows, compute few columns, aggregate/filter" — batch amortization and cache/SIMD gains outweigh row-by-row. Trade-off: vectorized operators must handle diverse types, nullability, complex types, expression combos — higher code complexity.

4.2 Pipeline Scheduling: From One-Thread-Per-Query to Operator-Chain-Driven

Traditional MPP isn't strictly "one thread per query" but often binds tasks, threads, and query lifecycle tightly, limiting concurrency control, thread reuse, and operator parallelism adjustment. Pipeline scheduling splits the execution graph into operator chains (Pipelines) . A Pipeline is a continuously executable operator sequence within a Fragment; data redistribution boundaries (Exchange) connect stages. Each Pipeline instantiates multiple PipelineDrivers — the actual concurrent units scheduled by a thread pool. pipeline_dop controls Pipeline parallelism (default 0 = auto-tune). max_pipeline_dop caps the upper bound. Crucially, pipeline_dop is not thread count . Setting

pipeline_dop=8</strong> means the Pipeline can spawn up to 8 Drivers, scheduled by the thread pool — not 8 dedicated threads. Resource relationship: Drivers multiplex onto limited threads.</p><p>Increasing <code>pipeline_dop

from 4 to 32 doesn't guarantee 8x speedup. Join/Aggregate/Sort may create more state, shuffle volume grows, bottleneck shifts from CPU to memory/network. StarRocks 3.0+ supports dynamic pipeline_dop adjustment based on concurrency; Query Queue adds admission control. Prefer auto-tuning; manual override only when Profile proves parallelism insufficiency.

Core distinction: traditional model binds threads directly to queries; Pipeline model decomposes work into schedulable Drivers run by a shared thread pool. Similar to Flink operator chaining but solves different problems: Flink for continuous streaming, StarRocks for high-throughput parallel OLAP with resource reuse.

4.3 CodeGen: Eliminating Runtime Overhead via JIT

Even with vectorization, per-operator loops still evaluate expressions (e.g., a + b * c). Generic expression evaluation incurs type dispatch, function calls, tree traversal. CodeGen/JIT specializes hot paths using runtime-known information (types, constants) into optimized machine code, reducing interpreter/dispatch overhead.

CodeGen complements vectorization: vectorization batches data; CodeGen speeds up per-element computation within the batch. However, CodeGen is not "regenerate entire native program per SQL" nor "remove all boundary checks". JIT compilation has startup cost; benefits vary by expression complexity, data types, query scale. StarRocks 3.3 enabled JIT by default with ongoing optimizations. Production analysis must consider JIT compile time, expression types, Query Profile, and version-specific Release Notes.

Engineering conclusion: CodeGen is another "mechanism exists ≠ every query should be manually tuned" case.

4.4 CBO: Letting the Optimizer Choose Plans Based on Facts

Execution speed is irrelevant if the plan is wrong. CBO core: collect statistics → estimate costs of alternative physical plans. StarRocks CBO (Cascades framework) parses SQL → logical plan → rule-based rewrites (predicate pushdown, column pruning) → cost-based exploration (join reordering, physical operator selection) → estimates CPU/Memory/Network/I/O per operator → picks lowest estimated cost.

Production focus isn't "will CBO optimize?" but: (1) statistics freshness (row count, NDV, histograms, null ratios); (2) histogram support since 2.4, external table stats (Hive/Iceberg/Hudi) since 3.2; (3) ANALYZE TABLE alone insufficient — need comprehensive stats collection; (4) compare Estimated Rows vs Actual Rows in Profile — large gaps indicate statistics/distribution skew.

Critical: Join reordering — multi-join order drastically impacts intermediate result size and downstream CPU/memory/network. CBO uses cardinality, selectivity, cost model. But statistics staleness is only one cause of plan errors; data skew, column correlation, complex expression estimation errors, incomplete external stats, cost model approximations also contribute.

Physical plan choices: Broadcast Join, Shuffle Join, Bucket Shuffle, Colocate — not fixed "small table broadcast, large table shuffle" rules. Decision combines table size, distribution, join keys, network cost, memory, statistics. CBO selects lowest estimated cost plan under current stats/model constraints, not "theoretically optimal".

5. Core Pipeline and Key Flows

SQL core flow: FE parse → logical operator tree → rule optimization (predicate pushdown, column pruning, expression rewrite) → cost optimization (join order, physical operator costing) → physical plan → Fragment/Exchange distribution → FragmentInstance dispatch to BE/CN. BE/CN builds Pipelines, instantiates PipelineDrivers, scheduled by thread pool, operators consume/produce Chunks.

Source-code reading mainline: FE planner → optimizer → physical plan → Fragment → Pipeline → Driver → Operator → Chunk. Production troubleshooting adds: slow query → Profile → identify bottleneck Fragment/Pipeline/Driver/Operator → trace to statistics, parameters, resource contention.

External lake table flow: External Catalog → partition pruning, predicate pushdown, file filtering, column pruning per connector capability → convert to StarRocks Chunks → unified vectorized+Pipeline execution. Caveat: "lake query" ≠ automatic zero-data-scan. Performance depends on partition design, file count, manifest/metadata reads, object storage network, predicate pushdown effectiveness, connector I/O concurrency, cache hits, CBO stats. Optimization essence: reduce scanned data, improve I/O efficiency, lower post-scan CPU/memory cost .

6. Real-World Scenarios and Engineering Practices

E-commerce real-time dashboard: operators filter time/category/channel, need 1s GMV/order/refund rates. Data partly in StarRocks internal tables, partly via External Catalog on Paimon real-time tables.

Key engineering actions:

Statistics : keep core table stats fresh (auto/scheduled), especially join keys and filter columns (cardinality, null ratio, distribution) — foundation for CBO join cardinality estimation. Use ANALYZE TABLE, verify with SHOW STATS META and SHOW HISTOGRAM META. For huge tables, use async/sampling stats to avoid collection overhead.

Vectorized params : vector_chunk_size default 4096. Don't blindly increase for throughput. Tuning order: Profile first, confirm Operator/Chunk bottleneck, then adjust — not "query slow → increase chunk size".

Pipeline parallelism : pipeline_dop default 0 (auto), max_pipeline_dop caps. Distinguish: pipeline_dop = intra-query parallelism; Query Queue = concurrency control; Resource Group = resource isolation. If "single query fast, 20 concurrent all slow", problem likely not pipeline_dop but Queue/Resource Group/CPU/memory/I/O saturation.

Predicate pushdown & partitioning/bucketing : queries should carry partition filters (time) for scan pruning. Bucket-aware execution can reduce shuffle for lake tables, but bucket filtering isn't universally auto-enabled.

Runtime Filter : for fact-dimension joins, verify Runtime Filter generation and pushdown to Scan. Ideal chain: dimension filter → Runtime Filter → fact table Scan pruning. High-selectivity Runtime Filter often beats increasing Pipeline DOP because it reduces downstream data volume.

Join shapes : small dimension tables may use Broadcast Join if memory/cost permits; large-large joins need Shuffle/Bucket Shuffle/Colocate evaluation. Don't treat "small=Broadcast, large=Shuffle" as rigid rules — verify via EXPLAIN and Profile.

Memory & concurrency : high-concurrency interactive queries can't rely solely on Pipeline DOP. exec_mem_limit caps per-query memory; Query Queue and Resource Group handle higher-level concurrency/isolation. No fixed numbers — must benchmark with actual data volume, concurrency, query complexity.

7. Performance, Cost, and Technology Selection

StarRocks query cost breakdown:

Scan cost : columnar reads + partition/index pruning; volume depends on predicates, data layout, connector pushdown capability.

Compute cost : vectorization + CodeGen improve per-CPU efficiency, but complex joins, aggregations, sorts, large intermediates remain CPU/memory bottlenecks.

Network/Shuffle cost : join/aggregation data exchange; large shuffles expensive — mitigated by join distribution strategies, pre-aggregation, Bucket Shuffle/Colocate.

CBO quality cost : stale stats → cardinality errors → suboptimal join order/algorithm.

Parameter tuning in four layers:

Vectorization layer : vector_chunk_size default 4096 — balance throughput, CPU overhead, memory. Not "bigger is better".

Pipeline parallelism layer : pipeline_dop default 0 (auto), max_pipeline_dop upper bound. Tune: CPU headroom + insufficient parallelism → consider increase; CPU saturated + memory pressure → don't increase. Note enable_runtime_adaptive_dop (runtime adaptive) distinct from pipeline_dop=0 auto-adjust; check version docs.

Resource control layer : exec_mem_limit caps per-query memory — not a performance knob. If Profile shows high HashJoin/Aggregate/Sort memory, first reduce data volume, optimize join, add filters, adjust execution strategy — not just raise limit. Above that: Query Queue (concurrency), Resource Group (isolation).

Lake I/O layer : external lake table bottlenecks often in remote file access, file count, metadata handling. Key params: scan_lake_partition_num_limit (scan guardrail), allow_lake_without_partition_filter (control partition-less scans), connector I/O adaptive, Lake DataCache, enable_bucket_aware_execution_on_lake (uses bucket info to reduce shuffle). Bucket-aware ≠ auto-enabled for all lake tables; effectiveness depends on table format + Catalog + bucket definition + join key + actual plan. Don't disable partition filters for "convenience" — one bad BI SQL can trigger massive object storage scans.

True production optimization: scan less, compute faster per unit, parallelize reasonably, shuffle less, CBO plan accuracy, stable resource isolation .

Internal table vs direct lake query: high-frequency, low-latency, repeated dashboard queries → StarRocks internal (or acceleration) tables; exploratory, low-frequency, need freshest lake data → External Catalog. Bridge via External Catalog, INSERT INTO, async materialized views — capabilities depend on Catalog, format, version. Selection must match team stack and query patterns, not single-table benchmarks. All QPS/scan bytes/latency are environment-dependent; rely on own TPC-like benchmarks.

8. Production Issues and Troubleshooting

Issue 1: Slow Query, CBO Wrong Join Order

Symptom: multi-join report suddenly slow. Method: EXPLAIN physical plan + Query Profile compare Estimated Rows vs Actual Rows per join operator. Root causes: stale stats, data skew, correlation, complex expression estimation error, incomplete external stats, cost model approximation. Steps: check SHOW STATS META → SHOW HISTOGRAM META → if stats stale, ANALYZE TABLE. For reproducible bad plans, use Join Hint, Broadcast/Shuffle Hint, Query Tuning Guide. Prevention: schedule core table stats collection, monitor estimation vs actual deviation.

Practical heuristic: if Estimated ≈ Actual but query slow → problem shifted to Scan/CPU/Network/Memory/I/O. If Estimated ≫ Actual or Estimated ≪ Actual → prioritize stats/distribution/cardinality investigation.

Issue 2: BE OOM, Query Killed

Symptom: BE memory spike, query failure, node instability. Common causes: oversized HashJoin intermediate, large Aggregate state, large Sort buffer, big shuffle, missing partition filter, high concurrency, excessive Pipeline DOP. Diagnosis: Query Profile → QueryPeakMemoryUsage + per-operator (HashJoin/Aggregate/Sort/Exchange) memory breakdown. Fix priority: (1) reduce scanned data, (2) reduce intermediate results, (3) lower unreasonable Pipeline DOP, (4) set sensible exec_mem_limit, (5) control concurrency via Query Queue/Resource Group. Pipeline DOP is not an OOM silver bullet — increasing DOP may increase Driver/Operator instances and intermediate state, worsening memory pressure.

Issue 3: External Lake Table Slow

Symptom: direct Paimon/Iceberg query slower than expected. Causes beyond missing partition/predicate pushdown: excessive partitions, excessive files, manifest/metadata reads, object storage latency, connector I/O concurrency, cache misses, data type conversions, inaccurate CBO stats. Fix: ensure partition filters, predicate type matching, healthy Catalog metadata access; for frequent queries consider internal tables, caching, async materialized views instead of repeated large lake scans.

Issue 4: High Concurrency, Single Query Fast but Overall Slow

Classic misdiagnosis: single query 500ms, 10 concurrent 1s, 50 concurrent 5s. Don't just raise pipeline_dop. Possible: CPU saturation, memory contention, I/O saturation, Queue queuing, Resource Group limits. Investigation order: Query Profile → Query Queue → Resource Group → CPU/Memory/I/O → Pipeline DOP. Only adjust DOP after confirming intra-query parallelism insufficiency.

Production tuning matrix (image referenced) summarizes parameter interactions.

9. Advanced Interview and Architecture Thinking

Q1: Why vectorized faster than Volcano? Batch columnar Chunks amortize call/scheduling overhead, cache/SIMD friendly; execution model change, not just columnar storage. vector_chunk_size default 4096 is engineering balance, not universal optimum.

Q2: What problem does Pipeline solve vs traditional MPP? Not "one thread per query" but decomposes plan into Pipelines → PipelineDrivers scheduled by thread pool. pipeline_dop controls intra-pipeline parallelism, max_pipeline_dop caps it. Distinguish: Pipeline DOP ≠ query concurrency; Query Queue/Resource Group handle higher-level resource control.

Q3: Why increasing pipeline_dop can slow queries? DOP↑ → Drivers↑ → Operator instances↑ → CPU/Memory/Network pressure↑ → resource contention. DOP is parallelism parameter, not performance multiplier. If Profile shows CPU saturated, raising DOP usually pointless.

Q4: Why CBO sometimes picks wrong plan? Cost estimation relies on stats and cost model. Stale stats or distribution change → cardinality error → join cardinality error → join order/method error → intermediate result explosion. Troubleshoot: Statistics → EXPLAIN → Estimated Rows → Actual Rows → Query Profile. Not just "CBO broken".

Q5: When does CodeGen benefit most? Complex expressions, large data volumes amortize JIT compile cost; tiny queries may lose due to compilation overhead. Core: "specialized code reduces generic expression overhead, amortizing compile cost in suitable workloads" — not "JIT always faster".

Q6 (parameter trio): vector_chunk_size → batch size (vectorization batch param); pipeline_dop → how many Drivers per Pipeline (intra-query parallelism param); exec_mem_limit → per-query execution memory cap (resource protection param). Distinct: Chunk Size → batching; DOP → parallelism; Memory Limit → resource boundary.

Q7 (end-to-end): Why External Catalog lake queries can be fast? External Catalog provides direct lake access; scan phase does partition pruning, predicate pushdown, column pruning, file filtering per connector; data converted to StarRocks Chunks → vectorized+Pipeline execution. But if: huge file count, no partition filter, object storage I/O bottleneck → even strong Pipeline/vectorization can't compensate. Lake query performance = data pruning + I/O + cache + vectorized exec + Pipeline + CBO.

10. Summary, Knowledge Distillation, Next Episode

This article decomposes "fast query" into three layers: Vectorization (row→Chunk batch, cache/SIMD), Pipeline (plan→Pipeline/Driver, thread pool + DOP for flexible parallelism), CodeGen/JIT (reduce expression overhead in suitable cases), CBO (stats + CPU/Memory/Network/I/O cost model for join order/physical plan), Parameter system (exposes mechanisms as production-controllable resource knobs).

Source-code reading mainline: FE Planner → Optimizer → Physical Plan → Fragment → Pipeline → Driver → Operator → Chunk. Combined with troubleshooting flow: Symptom → Profile → Bottleneck Operator → Root Cause (stats/params/resources) → Fix → Prevention. These two lines form the complete source-to-production loop.

Key parameter cheat sheet: vector_chunk_size = 4096: vectorization batch size pipeline_dop = 0: default auto-adjust pipeline parallelism max_pipeline_dop: pipeline DOP ceiling enable_runtime_adaptive_dop: runtime adaptive DOP, distinct from pipeline_dop=0 auto exec_mem_limit: per-query execution memory guard

Query Queue : concurrency control

Resource Group : resource isolation

Statistics + Histogram : CBO's "data cognition"

Runtime Filter : join-scene early data pruning

Lake Scan / Partition / Connector I/O : external lake performance control plane

Unified performance formula:

Query Performance ≈ Data Scan Volume × Unit Compute Cost × Parallel Execution Efficiency + Network/Shuffle Cost + I/O Cost

Where:

Data Scan Volume: partition pruning, predicate pushdown, Runtime Filter, data layout

Unit Compute Cost: vectorization, SIMD, CodeGen/JIT

Parallel Execution Efficiency: Pipeline, Driver, DOP

Network/Shuffle Cost: join/aggregation strategy, Bucket/Colocate

I/O Cost: local storage, lake files, object storage, connector, cache

Plan Quality: CBO, Statistics, Histogram, Cost Model

Bottom line: StarRocks speed isn't "uses columnar storage" — it's storage pruning, columnar batching, Pipeline parallelism, CodeGen/JIT, CBO cost optimization, and resource control working together. Execution engine organizes CPU/memory/network/I/O efficiently; optimizer strives to compute less, earlier, correctly; parameter system keeps mechanisms within production boundaries.

Next episode (Part 13): Real-Time Lakehouse Production Incident Retrospectives — chain Kafka lag, Paimon compaction backlog, Flink checkpoint failures, StarRocks OOM, Exactly-Once breaks into full incident postmortems using "symptom → root cause → remediation → prevention" to see how these kernel mechanisms interact in real failures.