Step‑by‑Step Guide to Deploy Spinnaker with Halyard on Kubernetes Using External Redis and MySQL

Architecture Overview

Spinnaker is deployed with Halyard, Kubernetes, an external Redis cluster, MySQL 5.7, and MinIO (S3) storage. Services such as Gate, Orca, Clouddriver, Rosco, Igor, Fiat, and Kayenta originally use Redis and S3; this guide replaces them with SQL databases and an external Redis cluster.

Preparation

Six‑node Redis cluster (3 master + 3 slave)

MySQL 5.7 instance

MinIO for S3‑compatible object storage

Pull the Halyard container image

Download required Spinnaker images from Alibaba Cloud registry

Obtain the custom BOM files

2.1 Start Halyard Container

docker pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
mkdir /root/.hal
docker run -itd --name halyard \
  -v /root/.hal:/home/spinnaker/.hal \
  -v /root/.kube:/home/spinnaker/.kube \
  registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0

# Enter container as root to edit configuration
docker exec -it halyard bash
vi /opt/halyard/config/halyard.yml   # modify gcs.enabled = false, etc.
hal shutdown
docker start halyard

2.2 Download Required Images

All images are mirrored to the Alibaba Cloud registry registry.cn-beijing.aliyuncs.com/spinnaker-cd/. Use the provided script to pull them to each node.

# Upload script package to the node running Halyard
scp 1.22.1-Image-Script.zip [email protected]:/root
unzip 1.22.1-Image-Script.zip
cd 1.22.1
sh -x GetImages.sh
chmod 777 -R .hal/

2.3 Prepare BOM Files

# Move BOM files into the Halyard directory
mv .boms/ ~/.hal/
cd ~/.hal/.boms
# Directory layout shows version‑specific YAML files for each Spinnaker service

3. Halyard Configuration Management

Initialize Halyard configuration (set Spinnaker version, timezone, storage type)

Add Docker registry (Harbor) and Kubernetes cluster account

Enable feature flags (pipeline‑templates, artifacts, managed‑pipeline‑templates‑v2‑ui)

Configure Jenkins CI integration

Configure GitHub/GitLab artifact integration

3.1 Initialize Halyard

# Set Spinnaker version
hal config version edit --version local:1.22.1
# Set timezone
hal config edit --timezone Asia/Shanghai
# Configure storage (required placeholder)
hal config storage edit --type s3 --no-validate
# Override UI and API base URLs
hal config security ui edit --override-base-url http://spinnaker.idevops.site
hal config security api edit --override-base-url http://spin-gate.idevops.site

3.2 Add Registry and Kubernetes Account

hal config provider docker-registry enable --no-validate
hal config provider docker-registry account add my-harbor-registry \
  --address http://192.168.1.200:8088 \
  --username admin \
  --password Harbor12345

hal config provider kubernetes enable
hal config provider kubernetes account add default \
  --docker-registries my-harbor-registry \
  --context $(kubectl config current-context) \
  --service-account true \
  --omit-namespaces=kube-system,kube-public \
  --provider-version v2 \
  --no-validate

hal config deploy edit \
  --account-name default \
  --type distributed \
  --location spinnaker

3.3 Enable Feature Flags

hal config features edit --pipeline-templates true
hal config features edit --artifacts true
hal config features edit --managed-pipeline-templates-v2-ui true

3.4 Jenkins CI Integration

hal config ci jenkins enable
hal config ci jenkins master add my-jenkins-master-01 \
  --address http://jenkins.idevops.site \
  --username admin \
  --password admin
hal config ci jenkins master edit my-jenkins-master-01 --csrf true

3.5 GitHub / GitLab Integration

# GitHub
hal config artifact github enable
hal config artifact github account add my-github-account \
  --token 02eb8aa1c2cd67af305d1f606 \
  --username zey
# GitLab
hal config artifact gitlab enable
hal config artifact gitlab account add my-gitlab-account \
  --token qqHX8T4VTpozbnX

4. Use External Redis Cluster

# Create service‑settings directory and configure Redis endpoint
mkdir -p ~/.hal/default/service-settings
cat > ~/.hal/default/service-settings/redis.yml <<EOF
overrideBaseUrl: redis://192.168.1.200:6379
skipLifeCycleManagement: true
EOF

# Adjust gate profile to use the external Redis
cat > ~/.hal/default/profiles/gate-local.yml <<EOF
redis:
  configuration:
    secure: true
EOF

5. Use SQL Databases for Services

5.1 Clouddriver

Create the database and grant privileges.

CREATE DATABASE `clouddriver` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,EXECUTE,SHOW VIEW ON `clouddriver`.* TO 'clouddriver_service'@'%' IDENTIFIED BY '[email protected]';
GRANT ... TO 'clouddriver_migrate'@'%' IDENTIFIED BY '[email protected]';

Enable SQL in the Clouddriver profile.

cat > ~/.hal/default/profiles/clouddriver-local.yml <<EOF
sql:
  enabled: true
  connectionPools:
    default:
      jdbcUrl: jdbc:mysql://192.168.1.200:3306/clouddriver
      user: clouddriver_service
      password: [email protected]
    tasks:
      jdbcUrl: jdbc:mysql://192.168.1.200:3306/clouddriver
      user: clouddriver_service
      password: [email protected]
  migration:
    jdbcUrl: jdbc:mysql://192.168.1.200:3306/clouddriver
    user: clouddriver_migrate
    password: [email protected]
redis:
  enabled: false
EOF

5.2 Front50

CREATE DATABASE `front50` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,EXECUTE,SHOW VIEW ON `front50`.* TO 'front50_service'@'%' IDENTIFIED BY "[email protected]";
GRANT ... TO 'front50_migrate'@'%' IDENTIFIED BY "[email protected]";

cat > ~/.hal/default/profiles/front50-local.yml <<EOF
spinnaker:
  s3:
    enabled: false
sql:
  enabled: true
  connectionPools:
    default:
      jdbcUrl: jdbc:mysql://192.168.1.200:3306/front50
      user: front50_service
      password: [email protected]
  migration:
    jdbcUrl: jdbc:mysql://192.168.1.200:3306/front50
    user: front50_migrate
    password: [email protected]
EOF

5.3 Orca

CREATE SCHEMA `orca` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,EXECUTE,SHOW VIEW ON `orca`.* TO 'orca_service'@'%' IDENTIFIED BY "[email protected]";
GRANT ... TO 'orca_migrate'@'%' IDENTIFIED BY "[email protected]";

cat > ~/.hal/default/profiles/orca-local.yml <<EOF
sql:
  enabled: true
  connectionPool:
    jdbcUrl: jdbc:mysql://192.168.1.200:3306/orca
    user: orca_service
    password: [email protected]
    connectionTimeout: 5000
    maxLifetime: 30000
    maxPoolSize: 50
  migration:
    jdbcUrl: jdbc:mysql://192.168.1.200:3306/orca
    user: orca_migrate
    password: [email protected]
executionRepository:
  sql:
    enabled: true
redis:
  enabled: false
keiko:
  queue:
    sql:
      enabled: true
    redis:
      enabled: false
EOF

6. Deploy Spinnaker

hal deploy apply --no-validate

Create Ingress resources for UI and API access.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: spinnaker-service
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: spinnaker.idevops.site
    http:
      paths:
      - path: /
        backend:
          serviceName: spin-deck
          servicePort: 9000
  - host: spin-gate.idevops.site
    http:
      paths:
      - path: /
        backend:
          serviceName: spin-gate
          servicePort: 8084
  - host: spin-front50.idevops.site
    http:
      paths:
      - path: /
        backend:
          serviceName: spin-front50
          servicePort: 8080
  - host: spin-fiat.idevops.site
    http:
      paths:
      - path: /
        backend:
          serviceName: spin-fiat
          servicePort: 7003
EOF
kubectl create -f ingress.yml

7. Additional Settings

7.1 Authentication & Authorization

Enable LDAP or OAuth2 authentication and configure role‑based access.

# LDAP authentication example
hal config security authn ldap edit \
  --user-search-base 'ou=devops,dc=zy,dc=com' \
  --url 'ldap://192.168.1.200:389' \
  --user-search-filter 'cn={0}' \
  --manager-dn 'cn=admin,dc=zy,dc=com' \
  --manager-password '12345678'
hal config security authn ldap enable

# OAuth2 with GitHub example
hal config security authn oauth2 edit --provider github \
  --client-id 66826xxxxxxxxe0ecdbd7 \
  --client-secret d834851134e80a9xxxxxxe371613f05bc26
hal config security authn oauth2 enable

Authorization can be driven by LDAP groups or a static YAML file.

# File‑based role mapping (userrole.yaml)
users:
  - username: devops
    roles:
      - yunweizu
  - username: user2
    roles:
      - demo
hal config security authz file edit --file-path $HOME/.hal/userrole.yaml
hal config security authz edit --type file
hal config security authz enable

7.2 Email Notifications

cat > ~/.hal/default/profiles/echo-local.yml <<EOF
mail:
  enabled: true
  from: [email protected]
spring:
  mail:
    host: smtp.qq.com
    username: [email protected]
    password: ubxijwaah
    protocol: smtp
    default-encoding: utf-8
    properties:
      mail:
        display:
          sendname: SpinnakerAdmin
        smtp:
          port: 465
          auth: true
          starttls:
            enable: true
            required: true
          ssl:
            enable: true
        transport:
          protocol: smtp
        debug: true
EOF

hal config security authz edit --type file
hal deploy apply --no-validate

7.3 Canary Analysis

# Enable canary and configure AWS S3 (or MinIO) bucket
hal config canary enable
hal config canary aws enable
hal config canary aws account add my-canary \
  --bucket spinnaker-canary \
  --endpoint http://minio.idevops.site \
  --access-key-id AKIAIOSFODNN7EXAMPLE \
  --secret-access-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
hal config canary edit --default-storage-account my-canary
hal config canary aws edit --s3-enabled true

# Prometheus as metrics store
hal config canary prometheus enable
hal config canary prometheus account add my-prometheus \
  --base-url http://prometheus.idevops.site \
  --username admin \
  --password admin
hal config canary edit --default-metrics-account my-prometheus
hal config canary edit --default-metrics-store prometheus
hal deploy apply --no-validate

7.4 Monitoring Spinnaker

# Enable Prometheus metric store and redeploy
hal config metric-stores prometheus enable
hal deploy apply --no-validate

# Example Prometheus scrape config for Spinnaker sidecar "monitoring-daemon"
- job_name: 'spinnaker-services'
  kubernetes_sd_configs:
  - role: pod
  metrics_path: "/prometheus_metrics"
  relabel_configs:
  - source_labels: [__meta_kubernetes_pod_label_app]
    action: keep
    regex: 'spin'
  - source_labels: [__meta_kubernetes_pod_container_name]
    action: keep
    regex: 'monitoring-daemon'

# ServiceMonitor for prometheus‑operator
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: spinnaker-all-metrics
  labels:
    app: spin
    release: prometheus-operator
spec:
  selector:
    matchLabels:
      app: spin
  namespaceSelector:
    any: true
  endpoints:
  - targetPort: 8008
    interval: 10s
    path: "/prometheus_metrics"
EOF

After applying the configuration, Spinnaker metrics become visible in Prometheus and can be visualised in Grafana using the official Spinnaker monitoring dashboards.