How Kubebuilder Simplifies Building Custom Kubernetes Controllers and CRDs

Controller Pattern and Declarative API

Kubernetes orchestrates workloads using controllers that continuously reconcile the actual state of objects with the desired state defined in YAML manifests. The core loop compares the current cluster state to the expected state and, if they differ, executes a Reconcile operation to bring the system into alignment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
spec:
  selector:
    matchLabels:
      app: test
  replicas: 2
  template:
    metadata:
      labels:
        app: test
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

The controller for a Deployment ensures that the number of Pods with label app=test always matches the replica count. Internally, the kube-controller-manager runs a reconcile loop similar to the following pseudocode:

for {
    actualState := GetResourceActualState(rsvc)
    expectState := GetResourceExpectState(rsvc)
    if actualState == expectState {
        // do nothing
    } else {
        Reconcile(rsvc)
    }
}

In a declarative API, users specify the desired final state (e.g., via kubectl apply), and the controller performs the necessary create, update, or delete actions to achieve it.

Custom Resource Definitions (CRDs)

CRDs extend the Kubernetes API, allowing users to define new resource types. Implementing a CRD involves two steps:

Write the CRD manifest and apply it to the cluster so the API server recognizes the new type.

Write a controller that contains the reconciliation logic for the custom resource.

Kubebuilder Overview

Kubebuilder is a scaffolding tool that automates the creation of CRDs, controllers, and admission webhooks. It generates boilerplate code, sets up a Go module, and provides utilities for interacting with the Kubernetes API.

Core Concepts

GVK & GVR : GroupVersionKind identifies a resource type (e.g., apps/v1/Deployment), while GroupVersionResource identifies the REST endpoint for that type.

Scheme : Maps GVKs to Go structs, enabling the client to serialize and deserialize objects.

Manager : Initializes the shared cache, client, and runs all registered controllers. Its responsibilities include:

Running all controllers.

Initializing shared caches with list‑and‑watch capabilities.

Creating a client for direct API‑server communication.

Cache : Creates an informer for each GVK in the scheme. Each informer runs a reflector that watches the API server, populates a local store, and generates delta events.

Controller : Registers watch handlers on informers, enqueues reconcile.Request objects, and processes them concurrently. The controller’s main loop looks like:

for i := 0; i < c.MaxConcurrentReconciles; i++ {
    go wait.Until(func() {
        for c.processNextWorkItem() {}
    }, c.JitterPeriod, stop)
}

Clients : Read operations use the cache; write operations use the underlying Kubernetes Go client.

Index : Provides indexed lookups on cached objects to improve query performance.

Finalizer : Allows cleanup logic to run before an object is fully deleted by keeping the object alive until the finalizer is removed.

OwnerReference : Enables cascading deletion and triggers reconciliation of the owning resource when a dependent changes.

Using Kubebuilder

Initialize a project: kubebuilder init --domain edas.io Create an API:

kubebuilder create api --group apps --version v1alpha1 --kind Application

Define the CRD spec and status in the generated files.

Implement the Reconcile method in the controller.

Build the Docker image with the provided Makefile and deploy the CRD and controller to the cluster.

Source Code Walkthrough

main.go sets up the scheme, creates a manager, registers the controller, and starts the manager:

var (
    scheme   = runtime.NewScheme()
    setupLog = ctrl.Log.WithName("setup")
)
func init() {
    appsv1alpha1.AddToScheme(scheme)
    // +kubebuilder:scaffold:scheme
}
func main() {
    mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{Scheme: scheme, MetricsBindAddress: metricsAddr})
    if err != nil { setupLog.Error(err, "unable to start manager"); os.Exit(1) }
    err = (&controllers.ApplicationReconciler{Client: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("Application"), Scheme: mgr.GetScheme()}).SetupWithManager(mgr)
    if err != nil { setupLog.Error(err, "unable to create controller", "controller", "EDASApplication"); os.Exit(1) }
    setupLog.Info("starting manager")
    if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil { setupLog.Error(err, "problem running manager"); os.Exit(1) }
}

The manager creates a cache and client, registers informers for each GVK, and starts them. The cache’s Start method launches a reflector for each informer, which watches the API server and pushes delta events into a queue.

The controller’s Watch method registers handlers such as EnqueueRequestForObject and EnqueueRequestForOwner, which convert resource events into reconcile.Request objects placed on the work queue.

When the manager runs, it starts the cache, then each controller runs its processing goroutine, pulling requests from the queue and invoking the user‑implemented Reconcile method.

Key Questions Answered

Synchronizing custom and built‑in resources: Register both GVKs in the scheme; the cache automatically watches and caches them.

How Reconcile is triggered: Informer events are handled by watch handlers, enqueued, and processed by the controller’s work loop.

Cache mechanics: GVK → Informer mapping; each informer runs a reflector (list & watch) and an indexer for local storage.

Best Practices

Use OwnerReference for cascading deletion and to ensure dependent changes trigger the owner’s reconciliation.

Implement Finalizer logic to perform cleanup before an object is fully removed.

Make Reconcile idempotent to handle retries safely.

Leverage IndexFunc to speed up cache queries.