Skip to content
Tagged articles

production incidents

12 articles · Page 1 of 1
Raymond Ops
Raymond Ops
Oct 15, 2026 · Operations

10 Linux Commands to Diagnose 80% of Production Incidents

This guide presents a systematic 10-command Linux troubleshooting workflow—from uptime to dmesg—that helps engineers quickly isolate CPU, memory, disk, and network bottlenecks, avoid common misdiagnoses, and apply evidence-based fixes with proper verification and post-mortem practices.

Linuxcommand lineoperations
0 likes · 35 min read
10 Linux Commands to Diagnose 80% of Production Incidents
TonyBai
TonyBai
Oct 13, 2026 · R&D Management

Debugging AI-Generated Code: The 'Facts First' Method That Catches Agent Hallucinations

The article presents a debugging methodology for AI-generated code based on three production incidents, advocating a constraint-satisfaction approach where agents collect immutable facts before forming hypotheses, use control comparisons to isolate strong signals, and verify against all evidence to avoid hallucinated fixes.

AI-assisted developmentDebuggingDistributed Systems
0 likes · 16 min read
Debugging AI-Generated Code: The 'Facts First' Method That Catches Agent Hallucinations
Internet Backend Architecture Stack
Internet Backend Architecture Stack
Oct 7, 2026 · Cloud Native

Kubernetes Self-Healing Backfires: Probe Misconfiguration Causes Cascading Failures

The article analyzes four production incidents where Kubernetes liveness and readiness probe misconfigurations turned transient dependency glitches into cluster-wide outages, explains the architectural boundary between liveness (process health) and readiness (traffic readiness), provides parameter tuning guidelines, warns against dynamic probe configuration, and presents a multi-layer fault isolation model with a practical review checklist.

Fault IsolationKubernetesLiveness Probe
0 likes · 17 min read
Kubernetes Self-Healing Backfires: Probe Misconfiguration Causes Cascading Failures
liandk
liandk
Sep 29, 2026 · Backend Development

Java Fault Analysis Masterclass: 7-Step Review, 6 Root Causes, 8 HA Principles

This final chapter of a 20-part Java series presents a 7-step fault review SOP, categorizes 99% of production faults into six root causes, defines eight high-availability architecture principles, and outlines a four-layer risk prevention system to shift from reactive firefighting to proactive stability.

Architecture PrinciplesFault AnalysisHigh Availability
0 likes · 13 min read
Java Fault Analysis Masterclass: 7-Step Review, 6 Root Causes, 8 HA Principles
Internet Backend Architecture Stack
Internet Backend Architecture Stack
Sep 24, 2026 · Backend Development

Circuit Breaker & Degradation Architecture: From Production Avalanche Postmortems to Enterprise-Grade Protection

This article analyzes three real production incidents from e-commerce promotions to expose critical flaws in circuit breaker and degradation architectures, detailing the dangers of half-open state probing, configuration hot-update inconsistency, and unbounded business degradation, then presents a three-layer protection model with custom recovery strategies and governance checklists.

Distributed SystemsFault ToleranceHigh Availability
0 likes · 23 min read
Circuit Breaker & Degradation Architecture: From Production Avalanche Postmortems to Enterprise-Grade Protection
liandk
liandk
Sep 24, 2026 · Backend Development

Redis Production Fault Troubleshooting: 8 Critical Issues & Solutions

This article details eight high-frequency Redis production faults—cache penetration, breakdown, avalanche, big keys, hot keys, connection exhaustion, memory OOM, and consistency issues—providing symptoms, root causes, troubleshooting commands, solutions, and a universal SOP for rapid diagnosis and resolution.

Big KeysCache ConsistencyConnection Pool
0 likes · 14 min read
Redis Production Fault Troubleshooting: 8 Critical Issues & Solutions
Internet Backend Architecture Stack
Internet Backend Architecture Stack
Sep 19, 2026 · Backend Development

Distributed ID Deep Dive: Snowflake vs Segment Mode – Trade-offs, Pitfalls & Production Patterns

This article analyzes real-world e-commerce failures to compare Snowflake and Segment ID generation, exposing architectural pitfalls like time rollback, machine ID collisions, and blocking windows, while providing production-ready code patterns and a decision matrix for choosing between ordered consistency and high-throughput availability.

Architecture DesignDistributed SystemsJava
0 likes · 22 min read
Distributed ID Deep Dive: Snowflake vs Segment Mode – Trade-offs, Pitfalls & Production Patterns
Su San Talks Tech
Su San Talks Tech
Sep 14, 2026 · Backend Development

5 AI Patterns to Pinpoint Flaky Production Bugs

The article presents five practical patterns for using AI to debug intermittent production bugs: time-window log analysis, comparative field diffing, concurrent reproduction scripts, hypothesis validation with evidence tables, and integrating logging/database monitoring via MCP, emphasizing AI for retrieval/enumeration while humans retain fix responsibility.

AI debuggingMCPconcurrency testing
0 likes · 28 min read
5 AI Patterns to Pinpoint Flaky Production Bugs
Internet Backend Architecture Stack
Internet Backend Architecture Stack
Sep 3, 2026 · Backend Development

Nacos Config Center Deep Dive: Core Mechanisms, 4 Fatal Failure Postmortems & Production Standards

This article dissects Nacos configuration center's core mechanisms — long-connection push, local disk caching, and differentiated refresh behaviors — and postmortems four real production incidents: config drift causing instance divergence, large-config push triggering cluster avalanche, K8s IP-based gray release failure, and rollback ineffectiveness due to lost push events and cache pollution. It provides coding standards, enterprise-grade landing specifications, and emergency SOPs.

Distributed SystemsGray ReleaseLong Polling
0 likes · 15 min read
Nacos Config Center Deep Dive: Core Mechanisms, 4 Fatal Failure Postmortems & Production Standards
Efficient Ops
Efficient Ops
Mar 5, 2017 · Operations

Surviving Hundred‑Billion Transactions: Real Production Incident Stories

This article recounts a series of real‑world production incidents—including massive concurrency overloads, DDoS attacks, SQL injection breaches, and critical bugs—encountered by an internet finance platform, and shares the concrete technical fixes and lessons learned to improve system resilience.

DDoSOptimistic LockSQL injection
0 likes · 22 min read
Surviving Hundred‑Billion Transactions: Real Production Incident Stories