Real‑World Ops Lessons: Cloud Architecture, Security, and Team Management in Action

1. Project Management and Cloud Architecture

On a foggy Friday morning the author checks email, instant messages, and cloud notes to review the previous day's work and plan today's tasks, then reflects on the need to clear the fog of work life and explore new horizons.

He categorizes today's tasks into four quadrants: important and urgent; urgent but not important; important but not urgent; neither important nor urgent.

At 10:00 a weekly project meeting discusses the progress of a cloud computing project aimed at transforming the company's architecture from traditional information systems to a resource‑centric, platform‑supporting, agile delivery model. The proposed cloud architecture is illustrated below.

The project must be completed within strict time, budget, and resource constraints, following standard procedures.

A project‑management knowledge map covering five processes and ten knowledge areas is referenced for further reading.

2. Efficient Meeting Management

To improve meeting efficiency, the author adopts three practices:

Follow Robert’s Rules of Order: assign a host and recorder, set clear actionable items, keep topics focused, enforce time limits, and avoid personal attacks.

Distribute agenda and materials in advance to avoid on‑the‑spot discussions.

Take minutes that follow the SMART principle, clearly stating conclusions, tasks, owners, and deadlines.

Current cloud project issues include internal BGP/OSPF routing problems and VRF‑based multi‑VPN routing, both assigned to a full‑stack SRE named Wang Yi.

Wang Yi is technically strong but tends to work alone, which hampers team coordination.

3. Network Security Management

A security audit reveals several vulnerabilities that must be remediated before an external audit. The common remediation strategies are:

Enforce strict network segmentation and isolation.

Retire problematic systems, preserve evidence, and redeploy patched replacements.

Apply role‑based access control on bastion hosts.

Strengthen iptables policies.

Rotate system passwords.

Apply patches to operating systems and applications.

Remove malware‑infected servers.

Wang Yi is added to the security remediation team to help address these issues.

4. Operations Automation Architecture Design

The team has built a comprehensive operations‑automation management platform, illustrated below.

The solution follows DevOps principles, introduces a lightweight IT service‑management layer centered on CMDB, and emphasizes monitoring, automation, standardization, visualization, intelligence, and productization to improve service delivery.

Key design recommendations are:

Keep functions focused and modules decoupled; avoid over‑design.

Ensure the product is practical and supports business needs.

Pay special attention to security and permission controls in automation.

5. Project Team Management

The author introduces Bruce Tuckman’s team development stages to help the team resolve conflicts and improve performance.

6. Two‑Site Three‑Center Architecture Overview

The existing core database system has three major issues: limited RAC cluster bandwidth, aging storage arrays, and insufficient backup/disaster‑recovery capabilities.

The proposed solution adopts a two‑site three‑center design with same‑city active‑active synchronization and remote asynchronous disaster recovery, using RAC + DataGuard redundancy and multi‑layer data protection.

7. Traditional Operations vs. Internet‑Scale Operations

The author outlines six major differences: architecture, work content, knowledge system, target objects, personnel, and management philosophy, and includes a comparative diagram.

8. Large‑Scale Traffic Incident Handling

A sudden slowdown in multiple domain names is traced to a surge of connections on an outdated load balancer. The root cause is identified as malicious voting traffic overwhelming the device.

Mitigation steps include traffic shifting, IP filtering, session/cookie validation, CAPTCHA/real‑name verification, and optionally moving services to public cloud for protection.

9. Employee Turnover Analysis

The author summarizes common reasons for resignation using Maslow’s hierarchy: physiological (salary), safety (environment), social (culture), esteem (recognition), self‑actualization (career growth), and self‑transcendence (life goals).

10. Change Management Lessons

Avoid unnecessary changes; aim for immutable infrastructure.

Perform one change at a time and document it.

Prepare testing and rollback plans before changes.

Ensure implementation, review, and backup personnel are assigned and informed.

Schedule changes before Friday evenings.

Automation without standardization can be disastrous.

11. Operations Architecture Planning

The author proposes a service‑oriented, continuous‑delivery‑focused architecture built around four pillars: people (roles, training, performance), things (hardware, software, resources), processes (operations, monitoring, security, projects), and standards (procurement, deployment, handover).

12. Empathy in Operations

The author reflects on the hardships of operations work—procurement, deployment, monitoring, fault handling, overtime, and being the “IT firefighter” who often bears the blame—emphasizing the need for empathy and responsibility.