Apache Fluss Graduates to Top‑Level Project, Launching Agentic Lake’s Full Real‑Time Era

Graduation and community growth

In July 2024 the Apache Fluss graduation proposal received a unanimous IPMC vote and was approved by the Apache Software Foundation board, graduating Fluss to an Apache Top‑Level Project (TLP). The project moved from internal development to open‑source incubation and then to top‑level status.

Origin and motivation

Fluss was created by the Alibaba Cloud Flink team to address long‑standing challenges in streaming storage for analytical workloads, including the need for unified stream‑batch storage, high Flink compute costs, and complex data pipelines.

Timeline

November 2024 – Open‑sourced at Flink Forward Asia 2024 in Shanghai.

June 2025 – Entered the Apache Incubator.

July 2024 – Graduated to an Apache TLP after a unanimous IPMC vote.

Community metrics

157 contributors.

2,000+ GitHub stars.

1,700+ merged pull requests.

Production deployments

Fluss is deployed in production at Alibaba, Xiaohongshu, JD.com, Ant Group, Fresha, and iQIYI. Typical use cases include log collection, real‑time data warehousing, search recommendation, indexing pipelines, and real‑time feature services. Deployments reduce cross‑system data replication, lower real‑time processing and storage costs, and improve lakehouse data freshness and analytical efficiency.

Lakehouse vs. Lakestream in the Agentic era

AI agents in the Agentic era require sub‑second fresh context that combines complete historical data with the latest business state. Traditional lakehouses excel at storing massive historical data but struggle to provide millisecond‑level freshness. Apache Fluss adds a native real‑time streaming storage layer on top of lakehouse formats, unifying continuously updated data with long‑term history via a Union Read, thereby delivering fresh, complete, low‑latency context for AI agents.

Core technical capabilities

Lakestream integration : Long‑term data resides in open lake formats such as Paimon, Iceberg, Hudi, and Lance; Fluss handles the latest data. Union Read enables a unified table view spanning history and real‑time updates.

Efficient columnar streaming storage : Built on Apache Arrow columnar format, supporting server‑side column pruning, predicate push‑down, and partition pruning to minimize data reads and transmission.

Real‑time updates and low‑latency queries : Primary‑key tables natively support streaming updates, partial updates, changelog, KV queries, and Delta Join, allowing the same data to serve both stream processing and point queries.

Agent‑centric real‑time context : Continuous state and historical data are stored together, providing AI agents with fresh, complete, low‑latency, and trustworthy context for real‑time decision‑making.

Open ecosystem and multi‑language access : Integrates with Apache Flink, Apache Spark, and StarRocks; provides Java, Rust, Python, and C++ clients for easy adoption in real‑time analytics and AI applications.

Future direction

Following graduation, Fluss will deepen lakehouse‑native streaming storage to make lakehouses truly real‑time and to serve AI applications that depend on up‑to‑date features and dynamic state.

References

[1] Full‑vote IPMC thread: https://lists.apache.org/thread/kltvfrklyoqm9dj6dgwdzf82sm097427

[2] Apache announcement: https://news.apache.org/foundation/entry/the-apache-software-foundation-announces-new-top-level-projects-5

Project website: https://fluss.apache.org/

GitHub repository: https://github.com/apache/fluss

Contribution guide: https://fluss.apache.org/community/how-to-contribute/overview/