Tagged articles
canary deployment
73 articles · Page 1 of 1
This article presents a systematic, hypothesis-driven methodology for tuning Linux kernel parameters to improve Nginx concurrency, covering file descriptor limits, listen queue configuration, network buffer tuning, and safe canary deployment with rollback procedures, all validated through metrics and load testing.
ConcurrencyLinux kernelNGINX
0 likes · 25 min read

The article compares full AI development and AI-assisted development using a bricklaying metaphor and a notification retry case study, analyzing how human intervention points affect speed, risk, and acceptance across investigation, design, implementation, review, testing, deployment, operations, and handover.
AI-assisted developmentIdempotencyacceptance criteria
0 likes · 30 min read

A comprehensive guide to configuring Nginx upstream for reliable traffic distribution, covering environment validation, weighted round-robin, least_conn, consistent hashing, passive health checks, failover strategies, observability with structured logs, canary deployments, and safe rollback procedures.
FailoverNGINXObservability
0 likes · 38 min read

This article analyzes three real-world gray release failures in Spring Cloud, revealing that successful gray release requires full-chain context propagation, traffic-proportion accuracy, and bidirectional data compatibility—not just gateway routing. It covers implementation details, LoadBalancer filter chains, fallback strategies, database migration patterns, and troubleshooting techniques.
Context PropagationDatabase MigrationGray Release
0 likes · 53 min read

This article provides a step-by-step guide to implementing SaltStack for operations automation, covering master/minion deployment, key-based authentication, state tree and pillar configuration, grain-based targeting, canary deployments with dry-run verification, batched rollouts, health checks, troubleshooting techniques, and Git-based rollback procedures on Ubuntu 22.04.
Configuration ManagementInfrastructure as CodeOperations Automation
0 likes · 21 min read

This article provides a comprehensive decision framework for choosing between Layer 4 and Layer 7 load balancing, covering protocol analysis, TLS termination, health checks, session persistence, canary deployments, observability, and rollback procedures using HAProxy as a reference implementation.
HAProxyLayer 4 Load BalancingLayer 7 Load Balancing
0 likes · 27 min read

This article details how to build a proactive fault prevention system for 10M QPS architectures by modeling risk, automating change verification, observing leading signals, continuously validating defenses, and layering independent safeguards—moving beyond reactive incident response to prevent failures before they occur.
SLOcanary deploymentcapacity planning
0 likes · 36 min read

This article details a zero-downtime model update strategy using immutable artifacts, a model registry, canary deployments with readiness probes, traffic shifting via service mesh, and optional in-process generational hot loading, demonstrated with a GPU ranking service handling 600 QPS and 7 GiB models.
GPU InferenceIstioKubernetes
0 likes · 32 min read

This article dissects why static DNS-based traffic scheduling fails at ten-million-QPS scale and how layered dynamic scheduling — across access, gateway, service, and data layers — combined with unitized routing, automated decision loops, and rigorous drill practices enables precise, reversible, and safe failover.
AnycastFailoverGSLB
0 likes · 33 min read

This article details how to build reliable automated rollback systems for high-throughput architectures by establishing change identity, causal evidence through experiment/control groups, tiered rollback actions, state-machine execution, data compatibility matrices, verification budgets, and guardrails—progressing from manual processes to closed-loop autonomy.
Verificationautomated rollbackcanary deployment
0 likes · 37 min read

This article presents a comprehensive methodology for implementing real-time lakehouse architectures, covering requirements quantification, architecture selection, capacity planning, canary deployment, rollback strategies, and SLA decomposition with a six-stage pipeline, e-commerce case study, troubleshooting guides, and interview frameworks.
Apache FlinkApache PaimonReal-time Lakehouse
0 likes · 31 min read

The article analyzes the engineering challenges of moving configuration management from single-machine files to distributed cluster control planes, covering immutable versioning, schema validation layers, canary deployment by failure domains, two-phase commit for atomic switches, rollback with side-effect mitigation, observability across control and data planes, and maturity stages of config platforms.
Configuration ManagementObservabilityVersioning
0 likes · 47 min read

This article dissects why manual rollback breaks at 10M QPS and details a five-phase automatic rollback loop—context collection, health evaluation, traffic freeze, phased execution, and layered verification—emphasizing multi-signal decision rules, rollback-object dependency graphs, blast-radius control via canary, idempotent execution, and circuit-breaker governance to prevent automation-induced failures.
Circuit BreakerIdempotencyOperations Automation
0 likes · 26 min read

This article explains why anytime deployment fails at 10M QPS scale and details a comprehensive change window mechanism that controls change quantity, risk exposure rhythm, personnel availability, and recovery resources, including layered windows, admission criteria, controlled rollout phases, freeze periods with emergency lanes, automated gates, and effectiveness metrics.
Production OperationsRisk Controlautomation gates
0 likes · 30 min read

The article explains why independent deployment doesn't guarantee independent service evolution, detailing five required capabilities—identifiable dependencies, compatible interfaces, controllable traffic routing, verifiable releases, and recoverable failures—using a refund service case study to illustrate dependency topology, interface evolution patterns, phased release strategies, and evidence-based rollback decisions.
Dependency ManagementRollback Strategycanary deployment
0 likes · 21 min read

Cloudflare migrated its blog to EmDash CMS as a production testbed, load-tested with k6, built a multi-layer caching architecture on Cloudflare Workers, rolled out via canary deployment, achieved flatter latency and fewer errors, and exposed blog content via MCP for AI agents.
AI agentsCMS migrationCloudflare
0 likes · 11 min read

This article details a production-ready implementation of full-chain gray release using Spring Cloud Gateway, covering context propagation via Reactor Context and headers, dynamic rule engine with Caffeine and Nacos, deterministic traffic splitting, downstream interception, circuit breaking, rollback strategies, and lessons learned from reactive blocking, cache consistency, distributed transactions, and header injection pitfalls.
Caffeine CacheDistributed TracingGray Release
0 likes · 21 min read

This article explains how to build a full‑link gray (canary) release for a Spring Cloud microservice system by leveraging Nacos as the service registry, Spring Cloud Gateway filters, a custom Ribbon load‑balancer, and OpenFeign interceptors, complete with demo project structure, configuration files, and three test scenarios.
Gray ReleaseNacosOpenFeign
0 likes · 22 min read

Zero‑downtime releases require coordinated control‑plane and data‑plane actions—proper RollingUpdate settings, pod lifecycle handling, service‑mesh draining, pre‑warm of dependencies, capacity safeguards, and automated monitoring/rollback—otherwise brief spikes of 502/503 errors and duplicate consumption will appear.
Argo RolloutsHPAKubernetes
0 likes · 41 min read

This guide walks through deploying a demo application on Kubernetes, then demonstrates how to implement canary releases and blue‑green deployments using Nginx Ingress annotations for header‑based, IP‑based, and weight‑based traffic splitting, complete with YAML manifests and verification commands.
IngressKubernetesNGINX
0 likes · 14 min read

This guide explains why gray (canary) releases are essential for production stability in internet companies, and provides step‑by‑step configurations using Istio’s VirtualService, Gateway, and DestinationRule resources to route traffic by percentage or request headers in a Kubernetes cluster.
IstioKubernetesTraffic Routing
0 likes · 6 min read

The article explains how Canary Deployment gradually rolls out new software to a small user subset, monitors performance, and rolls back quickly, offering lower risk, faster feedback, and higher release confidence for large‑scale, mission‑critical systems.
CI/CDGradual ReleaseKubernetes
0 likes · 11 min read

This guide explains how to implement canary releases on Kubernetes using Argo Rollout, covering manual setup of production and canary environments, traffic splitting via Ingress annotations, and fully automated rollout steps with weight adjustments, pauses, and promotion to production.
Argo RolloutDevOpsIngress
0 likes · 16 min read

This article explains how to implement safe, incremental gray releases in a microservice architecture using Envoy Proxy, covering weight‑based traffic splitting, header‑based canary routing, and cookie‑based routing with full configuration examples and testing commands.
EnvoyGray ReleaseRouting
0 likes · 10 min read

This article explains how to implement gray (canary) release in a Spring Cloud project using Nacos for service discovery, custom request headers, ThreadLocal gray flags, Spring Cloud Gateway filters, custom Ribbon load balancer rules, and provides full code snippets, configuration steps, and deployment instructions.
Gray ReleaseNacosRibbon
0 likes · 20 min read

This guide explains how to implement full‑link gray release using Alibaba Service Mesh (ASM) swimlane isolation together with the open‑source Kruise Rollout framework, providing step‑by‑step configurations, Kubernetes manifests, and command‑line examples for traffic routing, canary deployments, and rollback in a cloud‑native environment.
ASMGray ReleaseIstio
0 likes · 19 min read

Learn how to implement safe, incremental releases using Canary deployment with the Ingress Nginx Controller, covering traffic-splitting strategies based on headers, cookies, and service weight, detailed annotation configurations, and practical YAML and curl examples for verification.
IngressKubernetesNGINX
0 likes · 10 min read

This article explains gray (canary) release concepts and demonstrates two practical Nginx-based methods—weight distribution and client‑request routing—complete with configuration snippets, enabling safe incremental rollouts and A/B testing for production systems.
DevOpsGray Releasecanary deployment
0 likes · 5 min read

This article explains how to implement gray (canary) release in a Spring Cloud micro‑service system using Nacos for service discovery, Spring Cloud Gateway filters, ThreadLocal gray flags, custom Ribbon load‑balancing rules, and configuration files to switch between production and gray versions.
Gray ReleaseJavaNacos
0 likes · 21 min read

This article provides a comprehensive DevOps‑focused guide on gray (canary) release, covering the problems it solves, four typical deployment scenarios, integration into development workflows, practical K8s ingress demos, and detailed Q&A on implementation nuances.
Gray Releasecanary deploymentcloud-native
0 likes · 20 min read

This article explains how to implement a gray (canary) release strategy in a Spring Cloud microservice ecosystem using Nacos for service discovery, Spring Cloud Gateway for request routing, Ribbon for load balancing, and custom filters and interceptors to control traffic based on request headers, IP, city or user ID.
JavaNacosRibbon
0 likes · 20 min read

This article explains the concept of gray (canary) release, why it’s essential for large‑scale architectures, outlines the step‑by‑step workflow, describes common traffic‑splitting strategies, and offers practical tips for monitoring and gradually scaling deployments to ensure system stability.
Gray ReleaseOperationscanary deployment
0 likes · 4 min read

This article explains how to implement gray (canary) releases using Nginx, Lua, and Redis, covering common gray‑release strategies, detailed configuration examples, Lua scripts for routing by request parameters, IP or cookies, and suggestions for extending the solution with other data sources or scripting languages.
Gray ReleaseLuacanary deployment
0 likes · 34 min read

The article explains how to safely upgrade services by using gray (canary) release strategies with Spring Cloud Gateway, compares rolling and blue‑green deployments, and demonstrates weight‑based routing, hint‑based routing, and custom ServiceInstanceListSupplier implementations to control traffic between old and new service versions.
Gray ReleaseLoadBalancerSpring Cloud Gateway
0 likes · 17 min read

This article explains the concept, architecture, strategy configuration, and execution mechanisms of a gray (canary) release system for microservices, covering simple designs, Nginx and gateway implementations, and complex multi‑service and database scenarios with practical code snippets.
Gray ReleaseNGINXbackend architecture
0 likes · 7 min read

This article explains how to perform data‑driven A/B testing in progressive delivery using Argo Rollouts Experiments, covering the concepts of progressive delivery, A/B testing fundamentals, the Argo Rollouts architecture, required Kubernetes resources, and step‑by‑step commands and YAML manifests for a weather‑app example.
A/B testingArgo RolloutsKubernetes
0 likes · 19 min read

This article explains how to perform data‑driven A/B testing in progressive delivery using Argo Rollouts Experiments on Kubernetes, covering the concepts, required resources, YAML manifests, command‑line steps, and the benefits of combining A/B tests with canary deployments.
A/B testingArgo RolloutsKubernetes
0 likes · 17 min read

This article introduces Kruise Rollout 0.3.0, explains its new batch deployment, header‑&‑cookie traffic shading, and Lua‑based Ingress extensions, compares it with Flagger and Argo Rollout, and provides detailed installation and usage instructions with code examples for Kubernetes users.
GitHubIngressKruise Rollout
0 likes · 19 min read

This article explains gray (canary) release concepts, compares canary, A/B testing, and blue‑green strategies, and provides a complete Spring Cloud Gateway implementation—including custom load balancer, global filter, and configuration—so you can smoothly roll out new features while preserving system stability.
AB TestingGray Releasecanary deployment
0 likes · 13 min read

This article explains how to achieve end‑to‑end gray (canary) traffic routing for microservice chains on Kubernetes by using MSE's lane features and configuring Ingress controllers, covering the required concepts, configuration steps, and verification methods.
IngressKubernetesMSE
0 likes · 8 min read

This article describes how to design and implement a gray (canary) messaging capability for RabbitMQ and Kafka, covering background, gray scenarios, two consumption strategies, and detailed production and consumption flows with code snippets for header tagging, requeue handling, and consumer group management.
Gray ReleaseKafkaMessage Queue
0 likes · 8 min read

This guide explains how to use Ingress-Nginx's canary annotations for blue‑green, weight‑based, header‑based, and cookie‑based traffic splitting in Kubernetes, providing step‑by‑step YAML examples, command‑line verification, and a comparison of deployment strategies.
A/B testingBlue-GreenIngress NGINX
0 likes · 16 min read

This article explains how Industrial and Commercial Bank of China (ICBC) adopts gray‑release strategies—including canary, rolling, and blue‑green deployments—combined with standardized specifications, management processes, and automation tools to control version risk, ensure financial stability, and accelerate digital innovation.
DevOpsGray ReleaseRolling Deployment
0 likes · 13 min read

This article explains how the open‑source Kruise Rollout framework enables progressive delivery on Kubernetes, covering Gateway API‑based canary releases, batch rollouts for StatefulSet and Advanced StatefulSet, pod batch labeling, and integration with KubeVela for seamless canary deployments.
Gateway APIKruise RolloutKubernetes
0 likes · 7 min read

Kruise Rollout, an open‑source progressive delivery framework from the OpenKruise community, extends native Kubernetes workloads with incremental, traffic‑aware rollouts, automated batch control via Prometheus metrics, and non‑intrusive integration, offering a robust alternative to Argo Rollout and Flagger for large‑scale deployments.
GitOpsKruise RolloutKubernetes
0 likes · 9 min read

The article details vivo Game Center’s end‑to‑end intelligent gray‑release data analysis system, which combines experimental design, statistical significance testing, multi‑dimensional anomaly root‑cause analysis (via the Adtributor algorithm), automated reporting, and sample‑size estimation to enable scientific version evaluation, rapid issue detection, and a fast closed‑loop verification process for game updates.
Game PlatformGray Releaseab-testing
0 likes · 14 min read

This guide explains the concept of serverless computing and gray (canary) releases, compares manual console methods with the FC-Canary plugin, and provides step‑by‑step instructions, code snippets, and best‑practice tips for configuring and testing automated canary deployments on Alibaba Cloud Function Compute.
FC-CanaryFunction Computecanary deployment
0 likes · 13 min read

This article explains how OpenKruise Rollout enables non‑intrusive, extensible progressive delivery on Kubernetes, covering its architecture, design goals, traffic scheduling, metric‑driven automation, and a complete canary release walkthrough with code examples.
KubernetesOpenKruiseRollout
0 likes · 11 min read

This article explains how to perform graceful service shutdowns and various deployment techniques—including /shutdown, /pause, and /service‑registry endpoints, EurekaAutoServiceRegistration, blue‑green, rolling, and canary deployments—using Spring Cloud to ensure zero‑downtime upgrades and smooth traffic migration.
Blue-GreenRolling UpdateSpring Cloud
0 likes · 10 min read

This article explains how to achieve a full‑link gray (canary) release in a Spring Cloud Alibaba microservice architecture by marking traffic at the gateway, propagating the gray tag through request headers, customizing Ribbon load‑balancing, and configuring Nacos metadata for selective service routing.
Gray ReleaseNacosOpenFeign
0 likes · 14 min read

This article explains how to perform rolling, blue‑green, and canary (gray) releases in Kubernetes, focusing on implementing canary deployments with the ingress-nginx controller using specific annotations, demonstrating weight‑based and header‑based scenarios, providing full YAML manifests, CI/CD pipeline designs, and recommendations for automated rollouts.
CI/CDDevOpsIngress
0 likes · 18 min read

This article explains the concept of gray release, outlines a simple architecture with essential components, describes common traffic-splitting strategies, shows how to implement control in Nginx and service layers, and discusses complex scenarios such as multi‑service and data‑centric deployments.
A/B testingGray ReleaseOperations
0 likes · 7 min read

This guide explains the concept of gray (canary) releases and provides detailed Istio configuration files—including Deployments, Service, Gateway, VirtualService, and DestinationRule—to gradually shift traffic between version 1 and version 2 of a service.
DestinationRuleIstioKubernetes
0 likes · 6 min read

This comprehensive guide introduces Traefik as an open‑source edge router, explains its core concepts and components, walks through Helm deployment, demonstrates HTTP and HTTPS routing with native Ingress, CRD IngressRoute, TLS (self‑signed and Let’s Encrypt), middleware usage, IP whitelisting, TCP services, canary releases, traffic mirroring, and the new Kubernetes Gateway API, providing step‑by‑step YAML examples and screenshots for each feature.
Gateway APITLSTraefik
0 likes · 29 min read

This guide shows how to use Ingress‑Nginx annotations to perform canary releases in Kubernetes, covering header‑based, cookie‑based, and weight‑based traffic splitting with complete YAML manifests and curl commands for verification, providing step‑by‑step examples and demonstrating practical testing.
Cookie RoutingHeader RoutingIngress NGINX
0 likes · 11 min read

This article explains the concept of canary releases, outlines their advantages and drawbacks, and provides a detailed, command‑line tutorial for performing a canary deployment, monitoring, pausing, resuming, and rolling back a Kubernetes application using kubectl.
Blue‑GreenDevOpsKubernetes
0 likes · 7 min read


DevOpsOct 12, 2021 · Operations
This article explains what gray release (canary deployment) is, why it is needed to reduce risk and improve product quality, and provides a step‑by‑step guide covering strategy, user targeting, data feedback, rollback, deployment architectures, and version management for modern software operations.
Gray ReleaseOperationscanary deployment
0 likes · 13 min read

Argo Rollouts is a Kubernetes operator that adds blue‑green, canary, analysis, experiment, and progressive delivery capabilities, offering automated GitOps‑driven rollouts, detailed implementation principles, architecture components, installation steps, and practical usage examples with code snippets for cloud‑native applications.
Argo RolloutsKubernetescanary deployment
0 likes · 19 min read

This article explains the concept of gray (canary) release, outlines a simple system architecture with essential components, details common traffic‑splitting strategies, and demonstrates both basic and complex implementation scenarios using Nginx, gateway services, and database migration techniques.
Database MigrationGray ReleaseTraffic Routing
0 likes · 6 min read

This article explains the concept of gray (canary) release, outlines a basic architecture with essential components, describes common gray release strategies such as header, cookie, and parameter based routing, and provides practical guidance for implementing gray releases using Nginx, gateway services, and handling complex scenarios like multi‑service and database migrations.
A/B testingGray ReleaseNGINX
0 likes · 7 min read

This article explains gray release (canary deployment) as an A/B testing‑style rollout, outlines a simple architecture with strategy configuration, execution, and service registry components, describes common traffic‑splitting strategies, and discusses implementation details for Nginx, gateway layers, and complex multi‑service and data‑centric scenarios.
A/B testingGray Releasecanary deployment
0 likes · 6 min read

This guide explains how to use Ingress‑Nginx 0.21+ Canary annotations to perform blue‑green, canary, and A/B testing deployments on Kubernetes, covering annotation syntax, priority rules, and step‑by‑step examples with weight‑based, header‑based, and cookie‑based traffic splitting.
A/B testingBlue-GreenIngress NGINX
0 likes · 17 min read

This article explains how OpenKruise SidecarSet enables automatic sidecar injection, in‑place upgrades without restarting the main container, and advanced rollout strategies such as rolling updates, canary releases, and scatter deployments, illustrated with a logtail sidecar example and detailed YAML configurations.
KubernetesOpenKruiseRolling Update
0 likes · 16 min read

This tutorial walks you through preparing two servers, installing Git, Nginx, and Jenkins, building two Angular projects, configuring Nginx for canary traffic routing, defining a cookie‑based gray‑release strategy, and creating Jenkins jobs to automate the entire deployment pipeline.
Gray ReleaseJenkinsNGINX
0 likes · 16 min read

This guide shows how to combine Jenkins with Argo CD and Argo Rollouts to create a fast, automated CI/CD pipeline that speeds up Argo CD triggers, performs canary releases, and automatically tags GitLab repositories for version control.
Argo CDArgo RolloutsCI/CD
0 likes · 16 min read

This guide explains what Argo Rollouts is, how to install it, and demonstrates step‑by‑step canary, blue‑green, and traffic‑shifting deployments on Kubernetes using NGINX Ingress, including commands for updating, promoting, aborting, and rolling back releases.
Argo RolloutsIngressKubernetes
0 likes · 20 min read

This article explains three key deployment strategies—Blue‑Green, Canary (gray release), and Dark Launch (feature toggles)—detailing their concepts, step‑by‑step traffic switching processes, rollback mechanisms, database considerations, and practical usage scenarios for reliable production releases.
CI/CDDark LaunchDeployment Strategies
0 likes · 10 min read

This step‑by‑step guide shows how to set up a gray (canary) release environment from server preparation and tool installation to Nginx traffic routing and Jenkins automation, enabling developers to safely roll out new features to a subset of users before full deployment.
Gray ReleaseJenkinsNGINX
0 likes · 16 min read

Traefik 2.0 finally arrives with major upgrades such as native TCP and SNI routing, unified HTTP/TCP entrypoints, a powerful middleware system, a revamped dashboard, built‑in canary releases via weighted load‑balancing, traffic mirroring, and comprehensive migration tools for moving from 1.x to 2.0.
KubernetesTCP RoutingTraefik
0 likes · 9 min read

At KubeCon EU 2019, Alibaba Cloud expert Deng Hongchao dissected key application‑management advances—including automated ConfigMap‑driven upgrades, server‑side apply, GitOps workflows, and automated canary rollouts—offering practical analysis and guidance for modern Kubernetes deployments.
Config ManagementKubernetesServer-side Apply
0 likes · 12 min read

Google Cloud’s engineers explain how microservice architectures benefit from small, fast-deployable containers, using Docker, Docker‑Compose, and Kubernetes features such as Pods, Replication Controllers, Services, Labels, and canary releases to achieve automated, high‑availability deployments at Google’s massive scale.
DockerKubernetescanary deployment
0 likes · 10 min read

The article explains gray release as a smooth, risk‑mitigating deployment method, outlines why it is needed, describes its limitations, and compares four practical gray‑release solutions—including code‑level flags, pre‑release machines, SET isolation, and dynamic routing—before recommending a combined approach.
Gray ReleaseOperationscanary deployment
0 likes · 11 min read

This article details the design, architecture, implementation steps, and early results of a graded release system for Baidu's mobile Tieba product, showing how staged rollouts, traffic splitting, and automated validation improve release quality and user experience.
NGINXRelease AutomationTIP platform
0 likes · 13 min read
