Understanding Stateful Functions: API, Runtime, and Stream Processing with Apache Flink

Recently, the open‑source project Stateful Functions (statefun.io) was announced, promising to dramatically reduce the complexity of building and orchestrating distributed stateful applications by integrating Apache Flink with Function‑as‑a‑Service (FaaS) stream‑processing benefits, offering a powerful abstraction for next‑generation event‑driven architectures.

Problem: Stateful applications remain difficult

Although technologies such as Kubernetes and FaaS have advanced the orchestration of stateless computation, most products focus on compute rather than state, leaving distributed stateful applications poorly supported and making function interaction a challenge for usability and data consistency.

Stateful Functions was created to overcome these limitations, allowing developers to define loosely coupled, tiny functions that occupy minimal space and can interact consistently and reliably within a shared resource pool. The framework consists of an abstract API that implements Stateful Functions and a runtime built on Apache Flink for distributed coordination, communication, and state management.

Stateful Functions API

The API encapsulates small business‑logic fragments, similar to actors. These functions exist as virtual instances—typically one per entity such as a user—and are sharded across the system, providing out‑of‑the‑box horizontal scalability. Each function maintains persistent, user‑defined state in local variables and can send messages to any other function (including itself) with exactly‑once delivery guarantees.

Runtime

The runtime supporting Stateful Functions is built on Apache Flink’s stream‑processing engine. State is stored within the stream engine alongside computation, enabling fast and consistent state access. Persistence and fault tolerance rely on Flink’s robust snapshot model, ensuring state and computation coexist on the same network without external round‑trips.

Compute state, not compute based on state

The framework is not intended to replace FaaS or Serverless; rather, Stateful Functions provides Serverless‑like capabilities tailored to state‑centric problems.

State‑centric

Stateful Functions focuses on measuring state and the interactions between different states and events, making the logic of these interactions the primary computational focus. Event‑driven applications that need to manage interactive state machines and retain context fit naturally into this paradigm.

Compute‑centric

Conversely, FaaS and Serverless frameworks excel at elastically scaling dedicated compute resources but integrate poorly with state and inter‑function communication, which is not their core strength. A classic example is scaling AWS Lambda for image processing.

To achieve this, the runtime under the Stateful Functions API relies on Flink’s stream processing and extends its powerful state management and fault‑tolerance model. The main advantages are that state and computation share the same network, eliminating the need for round‑trip logging and allowing direct state retrieval from external stores (e.g., Cassandra, DynamoDB) without imposing specific consistency patterns such as event sourcing or CQRS. Additional benefits include:

No need to manage dynamic messaging or maintain complex replication/repartition strategies, as persistence and snapshot storage are handled automatically.

High throughput for both real‑time stream and batch processing lets developers blur the line between event‑driven applications and general data processing.

Stateful Functions separates computation and storage differently from the classic two‑layer architecture: it maintains a short‑lived state/computation layer (Apache Flink) and a simple persistent blob storage layer. Programmatically, persistence is based on value‑based concepts, allowing each function instance to independently maintain and track fault‑tolerant state.

Extending the scope of stream processing

Although the Stateful Functions API is independent of Flink, the runtime is built on Flink’s DataStream API and uses lightweight process functions (low‑level functions with state access) to implement the underlying abstraction. Compared with vanilla Flink, the core advantage is that functions can arbitrarily send events to any other function, not just downstream DAG nodes.

Stateful Functions applications are typically modular, consisting of multiple function packages that can interact consistently and reliably, and can be reused within a single Flink application. This enables many small tasks to share the same resource pool and be utilized on demand without reserving peak resources. At any time, the vast majority of virtual instances remain idle and consume no compute resources.