DevOps Practices and Process Optimization: A Case Study of Baidu’s Product Line

DevOps has recently become a buzzword; literally it refers to the relationship between development teams and operations teams. So what exactly is "DevOps"?

According to Wikipedia, DevOps is a collection of processes, methods, and systems that facilitate communication, collaboration, and integration among software development, operations, and quality assurance. It embodies the lean principle of delivering maximum customer value by aligning development and operations as interdependent functions.

To acquire this capability, two key points are essential: a global view that strengthens cooperation across roles, and automation that replaces manual, human‑machine interactions with scriptable, non‑interactive management tools such as version‑controlled scripts, monitoring tools like Nagios, and configuration‑management tools like Puppet or Chef.

Some comment that DevOps still feels distant for most domestic companies, perhaps only top‑tier or newly founded firms attempt it, while many enterprises have not yet embraced agile practices. Is DevOps really that far away, and where should it start?

1. Let the Data Speak

Consider a six‑month change in a Baidu product line. Two Baidu‑specific terms are explained: "提测" (submission for testing) and "上线" (deployment). While these activities may not add direct value for customers, they consume time. After process improvements, the data in the two charts below show a significant increase in testing and deployment efficiency.

The charts clearly demonstrate that testing and deployment cycles have become much faster, which is crucial for an internet company like Baidu that releases new features weekly across many product lines.

2. Process Modeling

Given DevOps’ focus on end‑to‑end value delivery, let’s look at a typical delivery process for a single project.

It follows a classic waterfall flow: requirement collection, MRD (Marketing Requirement Document) or overall design, review, detailed design per module, branch creation from trunk, development, integration at a joint‑testing point, test request, testing, bug fixing, test sign‑off, deployment request, and finally operations‑team deployment. The diagram below illustrates this flow.

Because many projects run in parallel, each project creates its own branch from the trunk and only merges before deployment, as shown in the next figure.

Developers often work on multiple projects simultaneously, leading to repeated cycles of the same development steps for each iteration.

Key problems identified are:

Collaboration among multiple developers on the same role.

Each role focuses on its own deliverables (MRD, code, test cases, deployment tickets).

Human‑machine interaction based internal workflow platforms.

3. Identifying Waste

From a lean perspective, waste must be eliminated to deliver value faster. The main sources of waste include:

Unnecessary multi‑branch development – merging many branches raises the risk of integration problems.

Delayed problem discovery – features remain untested until the final integration, causing unexpected issues.

Communication via workflow platforms – test requests, version approvals, and status updates rely on internal tools, emails, and instant messages, creating bottlenecks.

Routine manual work – deployment still requires manual steps in internal platforms, with configuration scattered across locations.

Frequent algorithm‑strategy adjustments (at least weekly) further lengthen the cycle because each change triggers full rebuild, regression testing, and full binary deployment.

4. Countermeasures

1. Script‑Based Automation without Manual Intervention

Automated test requests – Daily integration provides a testable build; developers simply select a successful build for testing via Hudson plugins.

Unified configuration source – All configuration items are version‑controlled in Subversion, organized into Dev, Test, and Online directories.

Scripted regular processes – Developers separate binary packages from configuration; testers only need to validate changed configs. Operations replace manual deployment with scripts invoked via Hudson’s “Click Button” one‑click deployment.

2. Early Problem Detection and Resolution

“Split requirements, develop and verify promptly” – Break requirements into end‑to‑end user stories deliverable within three days; developers and testers collaborate on acceptance criteria before each story, using automated tests for coverage.

“Trunk development, branch testing” – Consolidate development on the trunk, create a branch weekly for testing, and fix any issues directly on the trunk.

“Continuous Integration” – Enforce a CI environment on the trunk, with developers and test engineers adhering to a strict “Check‑in Dance” discipline.

The new development flow is illustrated below.

Branch development shifts to a Single Branch model.

5. Summary

These improvements have transformed the team’s collaboration from isolated “fortress defense” to unified “front‑line” cooperation.

Previously, each role focused only on its own deliverables: product managers wrote MRDs, developers coded to the MRD without considering testability, testers validated against the MRD, and operations executed deployment tickets, with communication limited to hand‑offs of artifacts.

Now, all roles work together toward the final delivery goal, discussing requirements, optimizing implementations, and sharing knowledge: developers explain technical solutions to product managers, testers co‑define acceptance criteria with developers, and operations receive architectural insights to streamline deployment.

Through these activities, the team recognizes that the internal management platform is merely a document‑flow tool; to boost efficiency it must evolve into a fully automated system that emphasizes end‑to‑end value rather than siloed hand‑offs.