How eBay Automates Cross‑Platform Patch Deployment at Scale

Author Mei Cen‑kai, eBay Operations Manager, shares the practice of automating cross‑platform system patch deployment across the entire eBay site.

Background

Having worked at eBay for nearly 11 years, the author maintains the global cloud and application platforms, supporting both Windows and Linux servers that number in the hundreds of thousands. eBay, a 20‑year‑old e‑commerce giant, processes billions of dollars daily and must keep services running 24/7.

The company targets an ATB (availability) of 99.947% and performs weekly database upgrades and migrations. Any outage, including those caused by unpatched vulnerabilities, directly impacts revenue and user trust.

Problem Analysis

Initially, eBay operated only Windows servers using IIS and WSUS for patch distribution. This single‑OS approach caused several issues:

Manual, repetitive scripting for each patch.

Limited automation leading to high operational overhead.

Inability to visualize or centrally manage patches across diverse OSes.

Using WSUS and Active Directory introduced constraints: only three deployment policies (download‑only, download‑and‑schedule, download‑and‑auto‑install) and mandatory reboots that generated uncontrolled service interruptions.

To address these, the team moved from single‑OS scripting to automated workflows, eventually achieving a visual, platform‑based management system that integrates multiple OSes.

Patch Management Process

The workflow includes:

Vulnerability discovery via official vendor notices, CVE databases, and commercial or open‑source scanners (e.g., Qualys).

Impact assessment from both application and host perspectives.

Prioritization and grading of vulnerabilities.

Compatibility testing to avoid system crashes or blue‑screen events.

Automated, staged (gray‑scale) deployment with canary testing and rollback capability.

Security measures such as black‑listing risky packages (e.g., certain PHP versions) and maintaining a CMDB‑like configuration management system ensure that only approved assets receive patches.

System Architecture

The architecture consists of:

Network‑wide scanning agents that feed vulnerability data into a central repository.

A configuration management database (CMDB) that maps assets to applications.

Automated pipelines that pull packages from external repositories, mirror them internally, and trigger CI/CD processes for testing and deployment.

Separate handling for special cases (e.g., databases) that cannot be fully automated.

Images illustrate the end‑to‑end flow from discovery to reporting, as well as the visual management console.