How Inceptor 5.1 Boosts Batch and Interactive Analytics with Windrunner and Shiva

Transwarp Inceptor is a database designed for batch processing and analytics, widely used to build data warehouses and data marts. It is built on Hadoop and Spark and incorporates self‑developed innovative components to solve enterprise‑level big‑data processing challenges.

In Inceptor 5.1 two major modules changed: the execution layer added the vectorized engine Windrunner, and the distributed columnar storage was rebuilt on the new Shiva framework.

Performance improvements include an average 20% boost for batch processing and a 40% boost for interactive analysis.

Vectorized Compute Engine

Windrunner is a high‑performance distributed engine for data marts and real‑time warehouses. It reads the columnar store Holodesk and performs vectorized calculations, replacing traditional scalar execution.

By processing one column at a time, it fully utilizes CPU SIMD instructions, reduces parsing and transmission overhead, improves parallelism, and enables an SQL compilation model.

Reduces overhead of parsing and transmission through batch operations.

Accelerates computation using CPU SIMD instructions.

Improves system parallelism with vectorized algorithms.

Realizes an SQL compilation model via vectorized operators.

Windrunner analyzes hotspot computations, generates efficient code, dynamically parses SQL structures, and selects high‑efficiency runtime row/column object models, thereby improving performance while saving memory.

New Generation Distributed Storage Framework

Shiva is a self‑developed storage framework that can integrate various storage engines (Holodesk, ES, LevelDB) and provides unified storage and transaction interfaces, enhancing stability, operability, and high availability.

It relies only on the local file system, not HDFS, offering better performance. High availability is ensured by a Raft‑based group, with efficient RPC communication and merge strategies for unified distributed storage management.

Faster and More Reliable Columnar Storage

Holodesk, built on Shiva, inherits high availability and multi‑replica features. Optimizations reduce index column overhead (up to 40% space saving), improve read performance, add support for VARCHAR2 and CHAR types, and achieve near‑SSD performance on multiple HDDs.

Interactive Analysis Performance Gains

Windrunner provides vectorized execution for Holodesk, supporting existing SQL, operators, and UDFs, dramatically improving OLAP interactive analysis.

Shiva offers a unified high‑availability distributed storage framework, enhancing stability and operability.

Holodesk optimizations increase data availability, reduce storage overhead, and boost read performance.

Experiments show that the Windrunner + Holodesk + Shiva combination delivers multiple‑fold performance and stability advantages over competing products.

In TPC‑DS 1TB tests, Windrunner improves performance by an average of 34% compared with Inceptor 5.0, reaching up to 86% improvement for certain OLAP scenarios.

Offline (Batch) Analysis Performance Gains

Compiler architecture adjustments increase SQL support for complex sub‑queries, and optimizer enhancements improve execution performance across scenarios.

Key optimizer improvements:

Materialized‑Based Optimizer (MBO)

Supports materialized optimization for INSERT statements.

Enables incremental Cube building, improving Cube instantiation efficiency.

Applies to more sub‑query optimizations.

Cost‑Based Optimizer (CBO)

New statistics collection algorithm boosts task performance by 2‑3× and improves accuracy over 80%.

New cost model allows optimization even when statistics are missing.

Adds support for Outer Join optimization.

SQL Inter‑Operator Optimizer (ISO)

Better handles WITH AS clauses in global SQL.

These enhancements lead to a 20% batch performance increase in TPC‑DS 1TB tests, significantly outpacing other platforms.

Conclusion

The main changes in Inceptor 5.1 are:

Addition of the vectorized execution engine Windrunner.

Introduction of the Shiva storage framework for a new storage architecture.

Optimizations to the columnar storage engine Holodesk.

Accelerated interactive analysis enabled by the above three improvements.

Batch processing performance enhancements.

These upgrades give Inceptor a competitive edge in building data warehouses, data marts, and performing data analysis.