Mastering Argo Workflows: From Installation to Advanced CI/CD Pipelines

What is Argo Workflows?

Argo Workflows is an open‑source, container‑native workflow engine for Kubernetes that implements its functionality through Kubernetes CRDs.

Key features:

Each step runs in its own container.

Workflows can be modeled as a series of tasks or expressed as a directed acyclic graph (DAG) to describe dependencies.

Suitable for short‑running, compute‑intensive jobs such as machine‑learning or data‑processing tasks.

Can run CI/CD pipelines on Kubernetes without complex software configuration.

Installation

Install the controller

Install Argo Workflows with a single command:

kubectl create ns argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo-workflows/stable/manifests/quick-start-postgres.yaml

Four pods are created:

# kubectl get po -n argo
NAME                                 READY   STATUS    RESTARTS   AGE
argo-server-574ddc66b-62rjc          1/1     Running   4          4h25m
minio                                1/1     Running   0          4h25m
postgres-56fd897cf4-k8fwd            1/1     Running   0          4h25m
workflow-controller-77658c77cc-p25ll 1/1     Running   4          4h25m

Components:

argo-server – the Argo API server.

minio – artifact storage.

postgres – the workflow database.

workflow-controller – the controller that executes the workflow.

Expose the UI with an Ingress (example uses Traefik):

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: argo-ui
  namespace: argo
spec:
  entryPoints:
  - web
  routes:
  - match: Host(`argowork-test.coolops.cn`)
    kind: Rule
    services:
    - name: argo-server
      port: 2746

Configure Minio Ingress similarly:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: minio
  namespace: argo
spec:
  entryPoints:
  - web
  routes:
  - match: Host(`minio-test.coolops.cn`)
    kind: Rule
    services:
    - name: minio
      port: 9000

Default credentials are admin:password.

Install the client

Argo CLI installation on Linux:

# Download the binary
curl -sLO https://github.com/argoproj/argo/releases/download/v3.0.0-rc4/argo-linux-amd64.gz

# Unzip
gunzip argo-linux-amd64.gz

# Make executable
chmod +x argo-linux-amd64

# Move to PATH
mv ./argo-linux-amd64 /usr/local/bin/argo

Verify installation:

argo version
argo: v3.0.0-rc4
  BuildDate: 2021-03-02T21:42:55Z
  GitCommit: ae5587e97dad0e4806f7a230672b998fe140a767
  GitTreeState: clean
  GitTag: v3.0.0-rc4
  GoVersion: go1.13
  Compiler: gc
  Platform: linux/amd64

Key commands:

list      List workflows
logs      View workflow logs
submit    Create a workflow
watch     Watch workflow progress
get       Show detailed information
delete    Delete a workflow
stop      Stop a workflow

More commands are available via argo --help.

Core concepts

Workflow

A Workflow defines what to execute and stores its state. The specification lives in Workflow.spec and mainly contains templates and an entrypoint.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    container:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["hello world"]

Templates

Templates are listed under templates and come in two categories: concrete task definitions and parallel‑control templates.

Concrete task definitions

Container – runs a container (most common).

Script – wraps a container with an inline script.

Resource – performs Kubernetes resource actions (create, apply, delete, etc.).

Suspend – pauses execution; can be resumed with argo resume.

Example Container template:

- name: whalesay
  container:
    image: docker/whalesay
    command: [cowsay]
    args: ["hello world"]

Example Script template:

- name: gen-random-int
  script:
    image: python:alpine3.6
    command: [python]
    source: |
      import random
      i = random.randint(1, 100)
      print(i)

Example Resource template (creates a ConfigMap):

- name: k8s-owner-reference
  resource:
    action: create
    manifest: |
      apiVersion: v1
      kind: ConfigMap
      metadata:
        generateName: owned-eg-
      data:
        some: value

Example Suspend template:

- name: delay
  suspend:
    duration: "20s"

Parallel‑control templates

Two types: Steps and DAG.

Steps

Defines a list of lists; outer list executes sequentially, inner list runs in parallel.

- name: hello-hello-hello
  steps:
  - - name: step1
      template: prepare-data
    - name: step2a
      template: run-data-first-half
    - name: step2b
      template: run-data-second-half

Conditional execution uses when expressions.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: coinflip-
spec:
  entrypoint: coinflip
  templates:
  - name: coinflip
    steps:
    - - name: flip-coin
        template: flip-coin
    - - name: heads
        template: heads
        when: "{{steps.flip-coin.outputs.result}} == heads"
    - - name: tails
        template: tails
        when: "{{steps.flip-coin.outputs.result}} == tails"
  - name: flip-coin
    script:
      image: python:alpine3.6
      command: [python]
      source: |
        import random
        result = "heads" if random.randint(0,1) == 0 else "tails"
        print(result)
  - name: heads
    container:
      image: alpine:3.6
      command: [sh, -c]
      args: ["echo \"it was heads\""]
  - name: tails
    container:
      image: alpine:3.6
      command: [sh, -c]
      args: ["echo \"it was tails\""]

DAG

Defines explicit dependencies between tasks.

- name: diamond
  dag:
    tasks:
    - name: A
      template: echo
    - name: B
      dependencies: [A]
      template: echo
    - name: C
      dependencies: [A]
      template: echo
    - name: D
      dependencies: [B, C]
      template: echo

Variables

Variables can be defined in arguments.parameters and referenced with {{}}. Functions such as filter, asInt, asFloat, string, and toJson are supported.

filter([1, 2], {# > 1})
asInt(inputs.parameters["my-int-param"])
asFloat(inputs.parameters["my-float-param"])
string(1)
toJson([1, 2])

WorkflowTemplate

A reusable workflow definition that can be referenced from other Workflows or WorkflowTemplates.

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: workflow-template-1
spec:
  entrypoint: whalesay-template
  arguments:
    parameters:
    - name: message
      value: hello world
  templates:
  - name: whalesay-template
    inputs:
      parameters:
      - name: message
    container:
      image: docker/whalesay
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]

Create with: argo template create workflowtemplate.yaml Reference in a Workflow:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: workflow-template-hello-world-
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    steps:
    - - name: call-whalesay-template
        templateRef:
          name: workflow-template-1
          template: whalesay-template
        arguments:
          parameters:
          - name: message
            value: "hello world"

ClusterWorkflowTemplate

A cluster‑wide reusable template.

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: cluster-workflow-template-whalesay-template
spec:
  templates:
  - name: whalesay-template
    inputs:
      parameters:
      - name: message
    container:
      image: docker/whalesay
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]

Reference with templateRef and clusterScope: true in a Workflow.

Practice – CI/CD pipeline

The article demonstrates a simple CI/CD pipeline that checks out code, builds a Java project, creates a Docker image with Kaniko, and deploys via Kustomize.

Key steps are defined in a WorkflowTemplate named devops-java with parameters for repository, branch, image, etc. The template uses scripts for Git operations, Maven build, Kaniko image build, and Kustomize deployment.

Creating a secret for Docker credentials:

kubectl create secret generic docker-config --from-file=.docker/config.json -n argo

After applying the template, a Workflow can be launched either via the UI or with:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: workflow-template-devops-java-
spec:
  workflowTemplateRef:
    name: devops-java

The UI shows each step, logs, and final status.

Reference documentation

https://github.com/argoproj/argo-workflows/releases

https://argoproj.github.io/argo-workflows

https://github.com/antonmedv/expr/blob/master/docs/Language-Definition.md

https://github.com/argoproj/argo-workflows/tree/master/examples