How OpenFunction Enables Serverless Data Archiving for Scalable Cloud‑Native Systems

OpenFunction, an open‑source Function‑as‑a‑Service (FaaS) project from the KubeSphere community, provides a serverless application framework that lets developers focus on business logic without worrying about the underlying runtime and infrastructure.

Using OpenFunction’s cloud‑native capabilities, we built a serverless data‑archiving (and distribution) system to meet three core requirements: scalability for business growth, elastic resource scaling for operations, and efficient development.

Business Scenario

Data archiving is a common requirement in online services, including autonomous‑driving platforms. In addition to large AI‑training datasets, these platforms generate massive, time‑sensitive vehicle data whose value decays over time. The data is gradually migrated to lower‑cost, high‑capacity cold storage.

Challenges

Multiple storage types : relational, key‑value, document, time‑series, object storage, each with its own APIs.

Inconsistent data scale : different workloads (e.g., high‑frequency time‑series data vs. document data) across business domains.

Mixed sync/async operations : native read/write modes of storage lead to a blend of synchronous and asynchronous accesses.

Observability : fine‑grained status feedback is needed for each batch operation.

Why Serverless?

Micro‑service architecture can encapsulate operations for each storage type, but it introduces complexity in task orchestration and scaling. Serverless naturally provides on‑demand scaling, making it a suitable solution for the above challenges, although task chaining and mixed sync/async still require careful handling.

What OpenFunction Brings

OpenFunction addresses the challenges by:

Packaging Go and Node.js functions into container images via OpenFunction Build.

Deploying functions with the Function CRD in asynchronous serving mode.

Wrapping each function as a Dapr Component, allowing optional Bindings or Pub/Sub brokers for input/output.

Leveraging KEDA ScaledObject to monitor traffic and perform elastic scaling to zero when idle.

OpenFunction and Dapr

Dapr serves as the core asynchronous runtime for OpenFunction. OpenFunction simplifies Dapr usage by consolidating component and binding configurations and exposing a single‑line call such as ctx.Send(data, "dapr_component_name") instead of interacting with the Dapr sidecar directly.

Future Work

We plan to introduce synchronous function support in the upcoming 0.5.0 release, adding a Domain capability that binds functions to an Ingress entry, enabling synchronous access while preserving asynchronous message‑driven composition.

References

OpenFunction – https://github.com/OpenFunction/OpenFunction

KubeSphere – https://kubesphere.com.cn/

Data Archiving – https://baike.baidu.com/item/数据归档/10033756

KEDA – https://keda.sh/

Knative – https://knative.dev/docs/

Dapr – https://dapr.io/

OpenFunction CRD diagram – https://github.com/OpenFunction/OpenFunction#customresourcedefinitions

OpenFuncAsync – https://github.com/OpenFunction/OpenFunction#openfuncasync

Function CRD – https://openfunction.dev/docs/reference/function-spec/

Dapr Component – https://docs.dapr.io/concepts/components-concept/

Bindings – https://docs.dapr.io/reference/components-reference/supported-bindings/

Pub/Sub Brokers – https://docs.dapr.io/concepts/components-concept/

KEDA ScaledObject – https://keda.sh/docs/2.4/concepts/scaling-deployments/

Dapr Sidecar – https://docs.dapr.io/concepts/overview/#sidecar-architecture

0.5.0 release – https://github.com/OpenFunction/OpenFunction/releases/tag/v0.5.0

Domain – https://github.com/OpenFunction/OpenFunction#customresourcedefinitions