KubeTEE: An Open‑Source Cloud‑Native Confidential Computing Framework for Large‑Scale TEE Clusters

On September 25, Ant Group announced the open‑source release of KubeTEE, a cloud‑native large‑scale cluster confidential computing framework that addresses the end‑to‑end challenges of Trusted Execution Environment (TEE) technology in cloud‑native environments, from development and deployment to operations.

Background: Since 2018 Ant Group has been migrating to cloud‑native architecture and identified security as a critical gap. In 2020 they introduced the “Trust‑Native” concept, embedding trust throughout the stack, and built the SOFAEnclave confidential computing stack, which includes three main components: Occlum LibOS (simplifies TEE application development), HyperEnclave (solves deployment consistency), and KubeTEE (handles cluster‑level TEE issues).

KubeTEE combines Kubernetes and TEE to provide a holistic solution for trusted applications, offering Serverless‑style confidential computing services such as Trusted FaaS, allowing developers to focus on business logic while the platform manages the complex TEE workflow.

Open‑source components currently available include: sgx-device-plugin – SGX container plugin for enabling SGX features in containers. trusted-function-framework – TFF, a framework that abstracts SGX details and simplifies trusted function implementation. enclave-configuration-service – AECS, a remote‑attestation based configuration service for secure key management. protobuf-sgx – Modified protobuf library supporting enclave communication.

The article then outlines how to build a cluster‑wide trusted application using KubeTEE components, describing the additional SGX‑specific steps required in development (choosing a development mode, handling signing keys, building container images) and in deployment/operation (provisioning SGX‑enabled machines, network configuration, resource allocation, scaling, and maintenance).

KubeTEE aims to streamline these processes through cloud‑native mechanisms, providing three layers of support: trusted application development (via Occlum LibOS and TFF), infrastructure support (Kubernetes‑based SGX resource pooling and the sgx‑device‑plugin), and micro‑service assistance (e.g., AECS for secure key sharing, logging, monitoring, auto‑healing, and scaling).

Future outlook: KubeTEE will continue to deepen the integration of cloud‑native practices with confidential computing, adding more components and services to make TEE usage more efficient, simple, and cloud‑native.