Build a Two‑Region, Three‑Center Disaster‑Recovery System with ACK One Multi‑Cluster Deployment

Overview

This guide demonstrates how to build a two‑region, three‑center disaster‑recovery (DR) architecture by distributing a sample web application across three Kubernetes clusters using ACK One multi‑cluster management and Global Traffic Manager (GTM) for automatic failover.

Architecture

Two cities host three processing centers: two clusters in the same city (production and intra‑city DR) and one cluster in a different city (inter‑city DR). High‑speed links keep data synchronized so any center can take over when another fails.

Prerequisites

Three Kubernetes clusters named cluster1-beijing, cluster2-beijing, and cluster1-hangzhou.

ACK One instance with multi‑cluster management enabled.

GTM instance for global DNS routing.

Procedure

1. Enable the multi‑cluster control plane

Create an ACK One control‑plane instance according to the official documentation.

2. Associate the three clusters

Use the console or kubectl amc CLI to add the three clusters to the control plane, forming the two‑region, three‑center topology.

3. Create a GTM instance

Provision a GTM instance and define a global DNS name (e.g., web-demo.example.com) that will resolve to the address pools of the clusters.

4. Deploy the sample application

The sample consists of a Deployment, Service, Ingress, and ConfigMap. First create a namespace: kubectl create namespace demo Then create the manifest file app-meta.yaml with the following resources:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-demo
  namespace: demo
  labels:
    app: web-demo
spec:
  replicas: 5
  selector:
    matchLabels:
      app: web-demo
  template:
    metadata:
      labels:
        app: web-demo
    spec:
      containers:
      - name: web-demo
        image: acr-multiple-clusters-registry.cn-hangzhou.aliyuncs.com/ack-multiple-clusters/web-demo:0.4.0
        env:
        - name: ENV_NAME
          value: cluster1-beijing
        volumeMounts:
        - name: config-file
          mountPath: "/config-file"
          readOnly: true
      volumes:
      - name: config-file
        configMap:
          name: web-demo
---
apiVersion: v1
kind: Service
metadata:
  name: web-demo
  namespace: demo
spec:
  selector:
    app: web-demo
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-demo
  namespace: demo
spec:
  rules:
  - host: web-demo.example.com
    http:
      paths:
      - path: "/"
        pathType: Prefix
        backend:
          service:
            name: web-demo
            port:
              number: 80
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: web-demo
  namespace: demo
data:
  config.json: |
    {
      "database-host": "beijing-db.pg.aliyun.com"
    }

Apply the manifest on the control plane:

kubectl apply -f app-meta.yaml

5. Define application distribution rules

Create app.yaml to describe the target clusters, override policies, workflow, and the Application resource. Replace the placeholder managedcluster-xxx with the actual managed‑cluster names returned by kubectl amc get managedcluster.

apiVersion: core.oam.dev/v1alpha1
kind: Policy
metadata:
  name: cluster1-beijing
  namespace: demo
type: topology
properties:
  clusters: ["managedcluster-xxx"]
---
apiVersion: core.oam.dev/v1alpha1
kind: Policy
metadata:
  name: cluster2-beijing
  namespace: demo
type: topology
properties:
  clusters: ["managedcluster-xxx"]
---
apiVersion: core.oam.dev/v1alpha1
kind: Policy
metadata:
  name: cluster1-hangzhou
  namespace: demo
type: topology
properties:
  clusters: ["managedcluster-xxx"]
---
apiVersion: core.oam.dev/v1alpha1
kind: Policy
metadata:
  name: override-env-cluster2-beijing
  namespace: demo
type: override
properties:
  components:
  - name: "deployment"
    traits:
    - type: env
      properties:
        containerName: web-demo
        env:
          ENV_NAME: cluster2-beijing
---
apiVersion: core.oam.dev/v1alpha1
kind: Policy
metadata:
  name: override-env-cluster1-hangzhou
  namespace: demo
type: override
properties:
  components:
  - name: "deployment"
    traits:
    - type: env
      properties:
        containerName: web-demo
        env:
          ENV_NAME: cluster1-hangzhou
---
apiVersion: core.oam.dev/v1alpha1
kind: Policy
metadata:
  name: override-replic-cluster1-hangzhou
  namespace: demo
type: override
properties:
  components:
  - name: "deployment"
    traits:
    - type: scaler
      properties:
        replicas: 1
---
apiVersion: core.oam.dev/v1alpha1
kind: Policy
metadata:
  name: override-configmap-cluster1-hangzhou
  namespace: demo
type: override
properties:
  components:
  - name: "configmap"
    traits:
    - type: json-merge-patch
      properties:
        data:
          config.json: |
            {
              "database-address": "hangzhou-db.pg.aliyun.com"
            }
---
apiVersion: core.oam.dev/v1alpha1
kind: Workflow
metadata:
  name: deploy-demo
  namespace: demo
steps:
- type: deploy
  name: deploy-cluster1-beijing
  properties:
    policies: ["cluster1-beijing"]
- type: deploy
  name: deploy-cluster2-beijing
  properties:
    auto: false
    policies: ["override-env-cluster2-beijing", "cluster2-beijing"]
- type: deploy
  name: deploy-cluster1-hangzhou
  properties:
    policies: ["override-env-cluster1-hangzhou", "override-replic-cluster1-hangzhou", "override-configmap-cluster1-hangzhou", "cluster1-hangzhou"]
---
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: web-demo
  namespace: demo
spec:
  components:
  - name: deployment
    type: ref-objects
    properties:
      objects:
      - apiVersion: apps/v1
        kind: Deployment
        name: web-demo
  - name: configmap
    type: ref-objects
    properties:
      objects:
      - apiVersion: v1
        kind: ConfigMap
        name: web-demo
  - name: same-resource
    type: ref-objects
    properties:
      objects:
      - apiVersion: v1
        kind: Service
        name: web-demo
      - apiVersion: networking.k8s.io/v1
        kind: Ingress
        name: web-demo
  workflow:
    ref: deploy-demo

Apply the distribution rules:

kubectl apply -f app.yaml

6. Verify deployment status

Check the Application resource on the control plane: kubectl get app web-demo -n demo Typical output shows workflowSuspending before manual approval and running after the workflow is resumed.

NAME       COMPONENT    TYPE          PHASE                HEALTHY   STATUS   AGE
web-demo   deployment   ref-objects   running   true      -       47h

Inspect per‑cluster Deployments:

kubectl amc get deployment web-demo -n demo -m all

7. Configure GTM

Create two address pools in GTM:

pool-beijing : contains the Ingress IPs of the two Beijing clusters; load‑balancing policy returns all addresses so traffic is split evenly.

pool-hangzhou : contains the Ingress IP of the Hangzhou cluster.

Enable health checks for each pool. When a health check fails, the corresponding address is removed from the pool.

Define a traffic policy that sets pool-beijing as the primary pool and pool-hangzhou as the backup. GTM will route all traffic to Beijing while the pools are healthy; if both Beijing clusters become unhealthy, traffic automatically switches to Hangzhou.

8. Deployment validation

Generate traffic with a looped curl command and observe the responses.

for i in {1..50}; do curl web-demo.example.com; sleep 3; done

Expected behavior:

When all clusters are healthy, requests are served alternately by the two Beijing clusters (each returns This is env cluster1-beijing ! or This is env cluster2-beijing ! with the Beijing database host).

If the Deployment in cluster1-beijing fails, GTM routes all traffic to cluster2-beijing.

If both Beijing clusters fail, GTM routes traffic to cluster1-hangzhou, which returns the Hangzhou‑specific environment string and database address.

9. Clean‑up (optional)

When the DR test is complete, you can delete the Application, Workflow, and Policy resources with kubectl delete -f app.yaml and remove the GTM address pools.