Mastering Argo Workflows: Core Concepts, Templates, and CI/CD Practices

Component Overview

Argo Workflows is an open‑source container‑native workflow engine that runs on Kubernetes, using Custom Resource Definitions (CRDs) to define complex business processes. Typical use cases include CI/CD, data processing, machine learning, and batch jobs.

Core Concepts

Workflow

A Workflow resource defines the desired state, metadata, spec and status of a workflow. The spec contains templates, entrypoint, arguments, etc.

type Workflow struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata" protobuf:"bytes,1,opt,name=metadata"`
    Spec   WorkflowSpec   `json:"spec" protobuf:"bytes,2,opt,name=spec"`
    Status WorkflowStatus `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"`
}

Spec

Spec holds the list of templates, the entrypoint, arguments, service account, volumes, affinity, tolerations, image pull secrets, onExit hook, TTL strategy, and other execution details.

type WorkflowSpec struct {
    Templates []Template `json:"templates,omitempty" patchStrategy:"merge" patchMergeKey:"name"`
    Entrypoint string    `json:"entrypoint,omitempty"`
    Arguments  Arguments `json:"arguments,omitempty"`
    ServiceAccountName string `json:"serviceAccountName,omitempty"`
    Volumes []apiv1.Volume `json:"volumes,omitempty"`
    // ... other fields omitted for brevity
}

Template Types

Container : defines the main container that runs the business logic.

ContainerSet : runs multiple containers in the same pod, allowing parallel execution and shared emptyDir.

Script : wraps a container and adds a source field for inline scripts.

Resource : performs CRUD operations on Kubernetes resources (get, create, apply, delete, replace, patch).

Suspend : pauses the workflow for a specified duration.

Steps : defines a series of serial or parallel steps, each referencing a template.

DAG : defines a directed‑acyclic‑graph of tasks with explicit dependencies.

WorkflowTemplate

WorkflowTemplate is a reusable definition of a set of templates that can be referenced by multiple Workflows. Its structure mirrors the Workflow resource but without a status field.

type WorkflowTemplate struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata" protobuf:"bytes,1,opt,name=metadata"`
    Spec WorkflowSpec `json:"spec" protobuf:"bytes,2,opt,name=spec"`
}

Architecture

Argo consists of two main components deployed in the argo namespace: the Argo Server (provides a Web UI and API) and the Workflow Controller (the reconciler that drives workflow execution). Each step or DAG task creates its own pod with three containers: init, main, and wait.

Installation & Monitoring

Installation follows the official documentation. A Service exposing port 9090 is created for Prometheus metrics, and a Prometheus scrape configuration is added to collect those metrics. Grafana can be used for visualization.

Application Practice

A typical CI/CD pipeline uses Argo to compile code, build Docker images, push them to a registry, update image tags in the source repository, and run a global exit script that cleans up temporary artifacts.

Reference Documentation

Official docs: https://argo-workflows.readthedocs.io/en/latest/

Source code: https://github.com/argoproj/argo-workflows