How KubeSphere’s Cloud‑Native Architecture Simplifies Kubernetes Management

KubeSphere is a cloud‑native container hybrid‑cloud management system built on top of Kubernetes. It supports multi‑cloud and multi‑cluster management and provides a full‑stack automated operations capability, simplifying DevOps workflows with a user‑friendly, wizard‑style interface.

KubeSphere offers many enterprise‑grade features such as multi‑cloud/multi‑cluster management, Kubernetes resource management, DevOps pipelines, application lifecycle management, service mesh, log collection, multi‑tenant management, monitoring, alerting, event and audit queries, storage management, access control, GPU support, network policies, image registry, and security management.

Benefiting from Kubernetes' robust architecture, KubeSphere adopts a lightweight architecture that flexibly integrates resources and enriches the Kubernetes ecosystem.

KubeSphere Core Architecture

The core architecture is illustrated below:

The three main components are:

ks-console (frontend service component)

ks-apiserver (backend service component)

ks-controller-manager (resource state maintenance component)

KubeSphere’s backend follows Kubernetes’ declarative API style, abstracting operable resources as CustomResource objects. Declarative APIs are concise and focus on the desired end state rather than procedural steps.

Key characteristics of declarative APIs include:

Few, small objects (resources)

Objects define configuration for applications or infrastructure

Low update frequency

Typically read or write by humans

CRUD‑style operations

No need for cross‑object transactions

In contrast, imperative APIs exhibit signs such as explicit “do this” commands, operation IDs, RPC‑like behavior, bulk data storage, non‑CRUD actions, and difficulty modeling with objects.

Data synchronization and persistence are achieved via kube‑apiserver and etcd, while ks‑controller‑manager maintains resource states to ensure eventual consistency.

ks-apiserver Core Architecture

ks‑apiserver handles data exchange between frontend and backend, request proxying, authentication, and authorization. Its architecture is shown below:

ks‑apiserver is built with go‑restful , allowing multiple filters for dynamic request/response interception, enabling authentication, authorization, audit forwarding, and reverse proxy. It adopts Kubernetes’ RBAC model for permission control.

By using CRD + controller, ks‑apiserver decouples integrations, simplifying third‑party tool integration.

API Aggregation and Permission Control

Extending ks‑apiserver enables API aggregation for feature expansion and aggregated queries. Development must follow conventions to integrate with KubeSphere’s tenant and resource models.

API aggregation

Permission control

CRD + controller

API Specification

# Through api group for grouping
/apis/{api-group}/{version}/{resources}
# Example
/apis/apps/v1/deployments
/kapis/iam.kubesphere.io/v1alpha2/users
# Core API
/api/v1/namespaces
# Path distinguishes actions
/api/{version}/watch/{resources}
/api/{version}/proxy/{resources}/{resource}
# Path distinguishes resource hierarchy
/kapis/{api-group}/{version}/workspaces/{workspace}/{resources}/{resource}
/api/{version}/namespaces/{namespace}/{resource}

The purpose of API standardization is to:

Better abstract resources as objects suitable for declarative APIs.

Facilitate API management through versioning, grouping, and layering.

Integrate seamlessly with permission control, extracting metadata such as apigroup, scope, version, and verb.

Permission Control

KubeSphere’s permission control core is RBAC (Role‑Based Access Control). Key objects are Role, User, and RoleBinding. Roles define accessible resources at different scopes (cluster, workspace, namespace).

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: role-grantor
rules:
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["rolebindings"]
  verbs: ["create"]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["clusterroles"]
  verbs: ["bind"]
  # Omit resourceNames to allow binding any ClusterRole
  resourceNames: ["admin","edit","view"]
- nonResourceURLs: ["/healthz","/healthz/*"]
  verbs: ["get","post"]

RoleBinding binds a role to a subject (group, user, or service account).

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: role-grantor-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: role-grantor
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: user-1

CRD + Controller

Custom Resources extend the Kubernetes API, allowing users to create and manage them via kubectl just like built‑in resources.

Example CRD for a User resource:

apiVersion: iam.kubesphere.io/v1alpha2
kind: User
metadata:
  annotations:
    iam.kubesphere.io/last-password-change-time: "2021-05-12T05:50:07Z"
  name: admin
  resourceVersion: "478503717"
  selfLink: /apis/iam.kubesphere.io/v1alpha2/users/admin
  uid: 9e438fcc-f179-4254-b534-e913dfd7a727
spec:
  email: [email protected]
  lang: zh
  description: 'description'
  password: $2a$10$w312tzLTvXObnfEYiIrk9u5Nu/reJpwQeI66vrM1XJETWtpjd1/q2
status:
  lastLoginTime: "2021-06-08T06:37:36Z"
  state: Active

Corresponding API endpoints:

# Create
POST /apis/iam.kubesphere.io/v1alpha2/users
# Delete
DELETE /apis/iam.kubesphere.io/v1alpha2/users/{username}
# Update
PUT /apis/iam.kubesphere.io/v1alpha2/users/{username}
PATCH /apis/iam.kubesphere.io/v1alpha2/users/{username}
# List / Get
GET /apis/iam.kubesphere.io/v1alpha2/users
GET /apis/iam.kubesphere.io/v1alpha2/users/{username}

ks‑controller‑manager watches these resources and reconciles their state. Example reconcile function for a user controller:

func (c *userController) reconcile(key string) error {
    // Get the user with this name
    user, err := c.userLister.Get(key)
    if err != nil {
        if errors.IsNotFound(err) {
            utilruntime.HandleError(fmt.Errorf("user '%s' in work queue no longer exists", key))
            return nil
        }
        klog.Error(err)
        return err
    }
    if user, err = c.encryptPassword(user); err != nil {
        klog.Error(err)
        return err
    }
    if user, err = c.syncUserStatus(user); err != nil {
        klog.Error(err)
        return err
    }
    if c.multiClusterEnabled {
        if err = c.multiClusterSync(user); err != nil {
            c.recorder.Event(user, corev1.EventTypeWarning, controller.FailedSynced, fmt.Sprintf(syncFailMessage, err))
            return err
        }
    }
    c.recorder.Event(user, corev1.EventTypeNormal, successSynced, messageResourceSynced)
    return nil
}

Declarative APIs delegate complex logic to controllers, enabling easy integration with other systems, such as pipelines, credentials, applications, and notification configs.

/apis/devops.kubesphere.io/v1alpha2/namespaces/{namespace}/pipelines
/apis/devops.kubesphere.io/v1alpha2/namespaces/{namespace}/credentials
/apis/openpitrix.io/v1alpha2/namespaces/{namespace}/applications
/apis/notification.kubesphere.io/v1alpha2/configs

Example RBAC role granting full CRUD permissions on user resources:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: user-manager
rules:
- apiGroups: ["iam.kubesphere.io"]
  resources: ["users"]
  verbs: ["create","delete","patch","update","get","list"]

Example role for creating pipeline resources:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: devops-manager
rules:
- apiGroups: ["devops.kubesphere.io"]
  resources: ["pipelines"]
  verbs: ["create","delete","patch","update","get","list"]