Stop Hand‑Crafting ClusterRoles: Build a Production‑Grade Kubernetes RBAC Governance System with rbac‑manager

Many teams assume the problem with Kubernetes RBAC is the sheer amount of YAML, but the real issue is that permission rules, bindings, identity sources, change entry points and audit loops are scattered across multiple repositories, making it impossible to answer "who has what permissions in which environment".

Why manual ClusterRole management fails

When a large number of services are deployed, teams end up maintaining hundreds of RoleBinding and ClusterRoleBinding objects. This creates two extremes: platform teams drown in a sea of bindings, while business teams resort to granting cluster-admin directly. The article illustrates this with a real incident where a temporary view binding persisted for weeks, and where duplicated RBAC across dev, staging and prod namespaces drifted over time.

Four governance gaps exposed by native RBAC

Permission rules are pure additive – there is no deny mechanism. Role objects are namespace‑scoped, while ClusterRole can be reused across namespaces, but both are low‑level primitives.

Binding objects are mutable only at creation; roleRef cannot be edited, forcing delete‑recreate cycles.

Without a unified distribution layer, teams cannot answer audit questions about current permissions.

rbac‑manager’s three core capabilities

Declarative bindings : you define an RBACDefinition that references stable ClusterRole templates and lets the controller continuously converge the underlying RoleBinding / ClusterRoleBinding objects.

Automatic recycling : the controller tracks which bindings it created and removes them when the RBACDefinition is deleted or changed.

Label‑based distribution : using namespaceSelector, permissions are granted to all namespaces matching a label (e.g., team=order), eliminating manual copy‑paste.

Step‑by‑step migration guide

1️⃣ Define stable permission templates

Create minimal ClusterRole objects that capture the exact capabilities a team needs. Example read‑only template:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant-readonly
rules:
- apiGroups: [""]
  resources: ["pods","pods/log","services","configmaps","events"]
  verbs: ["get","list","watch"]
- apiGroups: ["apps"]
  resources: ["deployments","replicasets","statefulsets"]
  verbs: ["get","list","watch"]
- apiGroups: ["batch"]
  resources: ["jobs","cronjobs"]
  verbs: ["get","list","watch"]

And a limited deployer template that only allows patching deployments and reading configmaps/secrets:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tenant-deployer
rules:
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get","list","watch","patch","update"]
- apiGroups: [""]
  resources: ["configmaps","secrets"]
  verbs: ["get","list","watch"]

2️⃣ Distribute bindings with RBACDefinition

Assuming all order‑service namespaces carry the label team=order, the following definition grants the read‑only template to the order‑devs group:

apiVersion: rbacmanager.reactiveops.io/v1beta1
kind: RBACDefinition
metadata:
  name: order-developers-readonly
rbacBindings:
- name: order-developers
  subjects:
  - kind: Group
    name: order-devs
  roleBindings:
  - clusterRole: tenant-readonly
  namespaceSelector:
    matchLabels:
      team: order

This single object automatically creates the necessary RoleBinding s for every matching namespace, and removal of the label instantly revokes the permission.

3️⃣ Manage robot accounts

Declare a service account and its binding together, ensuring the robot only receives the permissions it needs:

apiVersion: rbacmanager.reactiveops.io/v1beta1
kind: RBACDefinition
metadata:
  name: order-release-bot
rbacBindings:
- name: order-release-bot
  subjects:
  - kind: ServiceAccount
    name: order-release-bot
    namespace: cicd
  roleBindings:
  - clusterRole: tenant-deployer
  namespaceSelector:
    matchLabels:
      team: order
      env: staging
  serviceAccounts:
  - name: order-release-bot
    namespace: cicd

4️⃣ Auditing and break‑glass

For full‑cluster read‑only auditors, use a ClusterRoleBinding but keep it minimal and isolated:

apiVersion: rbacmanager.reactiveops.io/v1beta1
kind: RBACDefinition
metadata:
  name: audit-readers
rbacBindings:
- name: audit-readers
  subjects:
  - kind: Group
    name: security-auditors
  clusterRoleBindings:
  - clusterRole: audit-readonly

Never let normal teams manage ClusterRoleBinding s; reserve cluster-admin for emergency break‑glass accounts only.

Common pitfalls

rbac‑manager does not design least‑privilege policies – you must audit your ClusterRole templates.

It cannot fix chaotic identity‑to‑subject mappings; clean up IAM/OIDC mappings first.

It is not an admission controller – dangerous direct kubectl create clusterrolebinding commands must be blocked by OPA/Gatekeeper/Kyverno.

Beware of aggregate‑to‑edit labels that unintentionally grant extra rights.

Repository layout recommendation

platform-rbac/
├── clusterroles/
│   ├── tenant-readonly.yaml
│   ├── tenant-deployer.yaml
│   ├── audit-readonly.yaml
│   └── ops-breakglass.yaml
├── rbac-definitions/
│   ├── teams/
│   │   ├── order-developers.yaml
│   │   └── payment-developers.yaml
│   ├── bots/
│   │   └── order-release-bot.yaml
│   └── audit/
│       └── security-auditors.yaml
└── policies/
    ├── deny-cluster-admin-for-normal-groups.rego
    ├── deny-wildcard-verbs.rego
    └── deny-aggregate-labels.rego

Pre‑deployment validation checklist

Semantic checks: no verbs: ["*"], no resources: ["*"], and no wildcard access to secrets, roles, clusterroles.

Distribution checks: subjects reference correct groups/service accounts, namespaceSelector matches only intended namespaces, and no accidental use of clusterRoleBindings for namespaced permissions.

Effectiveness checks: run kubectl auth can-i queries for representative users and bots to confirm expected allow/deny outcomes.

Evolution roadmap

Phase 1 : Consolidate all ClusterRole templates into a small, reviewed set.

Phase 2 : Hand over high‑frequency bindings (team roles, bots, batch label‑based grants) to rbac‑manager.

Phase 3 : Automate recycling and integrate continuous audit (can‑i checks, usage reports) into CI pipelines.

Final takeaways

rbac‑manager does not replace proper permission design, identity hygiene, or policy enforcement, but it transforms the *distribution* and *recycling* of RBAC into a declarative, observable process that scales with the organization. When combined with native ClusterRole design, GitOps versioning, OPA/Gatekeeper safeguards, and regular audit, you obtain a production‑grade, cloud‑native access‑control system.