GitOps on Kubernetes: Secure, Scalable Continuous Deployment Best Practices

What Is GitOps?

GitOps is a cloud‑native method for implementing continuous deployment of applications by treating a Git repository as the single source of truth for the desired state of infrastructure and workloads.

Core Principles

Maintain a Git repo that always contains a declarative description of the required production infrastructure and an automated process that reconciles the live cluster with that description.

The desired state is stored immutably and version‑controlled, preserving a complete history.

Deployments are triggered simply by updating the repository; a software agent automatically extracts the required state and applies it.

The agent continuously observes the actual system and attempts to enforce the declared state.

What GitOps Does Not Manage

Persistent application data such as user uploads.

Schema‑based deployments, e.g., database schemas.

Even when using GitOps, backup and recovery of such data remain essential.

Declarative Configuration and CRDs

Kubernetes has supported declarative configuration since day 1, making it a natural fit for GitOps. Custom Resource Definitions (CRDs) extend declarative configs to custom resources, allowing seamless GitOps adoption.

Kubernetes‑Based GitOps Best Practices

Git becomes the sole source of truth for the system’s desired state, enabling repeatable automated deployments, cluster management, and monitoring. Enterprise Git workflows (compile, test, scan) feed into the main branch, after which a GitOps toolchain validates, deploys, observes, alerts, and repairs to achieve the target state.

GitOps‑Based Continuous Deployment

Developers commit application code and configuration to a source repository.

The CI server builds, tests, scans, pushes a new container image to a registry, and updates K8s manifests in the Git repo.

A GitOps agent (e.g., Argo CD or Flux) detects the repository change and automatically synchronizes the K8s cluster.

Multi‑Cluster Management

When a Pull Request merges into the platform repository, GitOps agents in each cluster watch the repo and apply updates to the kube-system or a platform namespace.

Each cluster has its own configuration repository for access control, DNS, etc., which synchronizes with the upstream platform repo.

Application Development Experience

Business teams handle image build, test, and scan; configuration is committed to a team‑specific repo.

A GitOps agent runs in the tenant’s namespace, syncing the application state from the team repo to that namespace.

Multi‑Tenant Cluster Support

Different business or application teams own separate configuration repositories. Within the cluster, each team’s workloads run in isolated namespaces, and GitOps agents continuously deploy to the appropriate namespaces.

Using Helm for Application Packaging

Helm is the package manager for the Kubernetes ecosystem, analogous to apt on Debian. Seamless Helm support lets you leverage the mature K8s application packaging ecosystem for third‑party components and internal services.

Multi‑Environment Deployment Management

Enterprises often run multiple environments, clouds, or hybrid setups. Deployments are simplified by combining a base manifest with environment‑specific overlays.

Kustomize was created to solve exactly this problem and is now a native part of the K8s toolchain; supporting Kustomize in GitOps satisfies multi‑environment needs.

Cloud‑Native GitOps Tools

Because Kubernetes is a CNCF flagship project, the CNCF landscape is the primary source for GitOps tooling. The 2020 CNCF Continuous Delivery Radar lists Flux and Helm as “Adopt”, Kustomize as “Trial”, and Argo CD as “Assess”. The 2021 Multicluster Management Radar also rates Flux, Kustomize, Argo, and Helm as adoptable.

Among these, Argo CD and Flux are the most widely used in the community.

Push vs. Pull Deployment Styles

Push style advantages:

Simple and easy to understand; already used by many CI/CD tools such as Jenkins and AWS Code services.

Flexible; integrates easily with other scripts or tools.

Pull style advantages:

More secure because the GitOps agent runs inside the cluster with minimal permissions.

Ensures consistent management across many clusters.

Provides isolation; deployments do not depend on external CI/CD pipelines.

Scales easily to hundreds or thousands of clusters.

Therefore, a pull‑based deployment model is generally preferred for security, scalability, isolation, and consistency.

Comparison of Main Cloud‑Native GitOps Projects

Both Argo CD and Flux satisfy core GitOps principles and enterprise requirements such as multi‑tenant RBAC, multi‑cluster management, secret handling, alerting, Helm, and Kustomize support.

Argo CD offers a rich abstraction layer (Cluster, RBAC, Application, Hook, etc.), providing extensive functionality at the cost of higher complexity and a steeper learning curve.

Flux adopts a simpler architecture with five core components (Source, Kustomize, Helm, Notification, Image automation), reusing native Kubernetes primitives like ServiceAccount for RBAC, resulting in lower complexity and better compatibility with other cloud‑native projects.

Notes:

Git repositories may be publicly readable; secrets should be encrypted before committing.

Flux defaults to polling the Git repo but also supports webhook‑triggered updates.

Reference

https://kane.mx/posts/gitops/the-best-practise-of-gitops-in-kubernetes/