Databases 10 min read

Apache Doris 1.0 Release: New Vectorized Engine, Hive External Tables, Z‑Order Indexing and More

Apache Doris (incubating) announced its 1.0 release on April 18, 2022, featuring a new vectorized execution engine, Hive external tables, Lateral View syntax, Z‑Order indexing, SeaTunnel integration, numerous performance optimizations, new bitmap functions, security enhancements, and detailed upgrade instructions for users.

DataFunTalk
DataFunTalk
DataFunTalk
Apache Doris 1.0 Release: New Vectorized Engine, Hive External Tables, Z‑Order Indexing and More

Apache Doris (incubating) officially released version 1.0 on April 18, 2022, marking the first major release since entering the Apache Incubator. The release involved over 114 contributors who submitted more than 660 optimizations and fixes.

Key new features in the 1.0 version include:

Vectorized execution engine that replaces the row‑based Volcano model, improving CPU cache affinity, enabling batch‑level type checking, and allowing compiler inlining and SIMD optimizations.

Hive external table support, allowing direct querying of Hive tables without data import.

Lateral View syntax with Table Functions (e.g., explod_bitmap , explode_split , explode_json_array ) to expand bitmap, string, or JSON array columns into multiple rows.

Z‑Order data indexing to accelerate non‑prefix column queries.

Integration with Apache SeaTunnel (incubating) for high‑performance data synchronization and ETL across multiple sources.

New bitmap functions such as bitmap_max , bitmap_and_not , bitmap_and_not_count , bitmap_has_all , bitmap_and_count , bitmap_or_count , bitmap_xor_count , bitmap_subset_limit , and sub_bitmap .

Support for Chinese national cryptographic algorithms SM3/SM4.

Performance improvements include setting set batch_size = 4096 and enabling the vectorized engine with set enable_vectorized_engine = true , which yields up to 3× speedup on SSB and 2.6× on OnTime benchmark suites. Additional optimizations reduce segment file count during bulk loads, lower BRPC serialization overhead, improve memory usage for INSERT INTO , and enhance fault tolerance of imports.

Significant bug fixes address query errors, Broker Load scheduling issues, STREAM keyword metadata loading, decommission failures, schema change errors, and BE crashes caused by certain string types.

Upgrade instructions: users can upgrade directly from Apache Doris 0.15.0 or any 0.15.x release to 1.0 by following the documentation at http://doris.apache.org/zh-CN/installing/upgrade.html . Detailed release notes are available at https://github.com/apache/incubator-doris/issues/8549 .

The community thanks over 300 contributors from companies such as Baidu, Meituan, Xiaomi, JD, ByteDance, Tencent, NetEase, Alibaba, PingCAP, and Nebula Graph for their extensive work on the release.

For feedback or issues, users are encouraged to join the GitHub Discussions forum ( https://github.com/apache/incubator-doris/discussions ) or contact the developer mailing list at [email protected] .

PerformanceDatabaseApache DorisVectorized EngineHive External TableRelease 1.0Z-Order Indexing
DataFunTalk
Written by

DataFunTalk

Dedicated to sharing and discussing big data and AI technology applications, aiming to empower a million data scientists. Regularly hosts live tech talks and curates articles on big data, recommendation/search algorithms, advertising algorithms, NLP, intelligent risk control, autonomous driving, and machine learning/deep learning.

0 followers
Reader feedback

How this landed with the community

login Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.