4 Years of Ops Mastery: Practical Tips, Optimization Tricks, and Mindset

Accumulated Experience

After more than four years in operations, the author reflects on how the knowledge and ops system have evolved, noting the rise of big data and artificial intelligence, with Python becoming especially popular.

The author has worked over two years in game ops, one year in security ops, and one year in big‑data ops, gaining solid but not exhaustive expertise.

Beginner Section

For newcomers, see the detailed guide on essential Linux operations engineer skills (link omitted) and the topology diagram below.

Intermediate Section

Interview experiences show that merely installing LAMP/LNMP is insufficient; interviewers look for deeper understanding of architecture, Nginx reverse proxy, load balancing, and caching.

Key points:

Reverse proxy acts as a layer‑7 load balancer, enabling horizontal scaling and improving concurrency and resilience.

Local cache reduces backend load and boosts performance.

1. Primary Troubleshooting

Analyze why a program fails or behaves unexpectedly, trace system calls.

Perform in‑depth system bottleneck analysis.

Typical commands include checking remaining memory, system information, and hardware details (images omitted for brevity).

Log management commands are illustrated in the following diagram:

2. Optimization

Optimization is the most valued skill for ops engineers, but it carries risk; careless changes can cause outages.

JVM tuning parameters (examples):

-Xms: initial heap size

-Xmx: maximum heap size

-Xss: thread stack size

-XX:NewSize, -XX:NewRatio, -XX:SurvivorRatio, -XX:MaxPermSize

-XX:+UseSerialGC, -XX:+UseParallelGC, -XX:+UseConcMarkSweepGC, -XX:+UseCMSCompactAtFullCollection

-XX:+PrintGC, -XX:+PrintGCDetails, -Xloggc:filename

Additional options such as -XX:+AggressiveOpts, -XX:+UseBiasedLocking, -XX:+DisableExplicitGC, and memory page size tuning are also discussed.

Memory tuning results show a near‑three‑fold performance improvement after adjusting Tomcat JVM parameters.

3. Development Skills

Python is recommended over shell for automation due to its speed of learning and rich ecosystem (SaltStack, Fabric, Zenoss, Nagios plugins, libvirt, Supervisor, OpenStack, etc.). Mastering Python can significantly increase salary.

The author is currently converting legacy shell scripts to Python and maintains a Python notes repository.

4. Awareness

Security awareness: protect accounts and private keys, use encrypted storage tools, avoid cloud drives, and add passwords to SSH keys.

Sharpening‑knife mindset: understand the principle behind any configuration before applying it.

Planning awareness: for complex changes, draft detailed operation plans, get peer review, and prepare rollback procedures.

Documentation and sharing: record special cases and analyses for future reference and team knowledge transfer.

Monitoring awareness: monitoring is the eyes of ops; close collaboration with monitoring tools is essential.

Business awareness: know the services each host provides and their interdependencies to narrow fault‑tracing scope quickly.

Topology diagram of ops thinking:

Author: Haozi Source: http://chenhao6.blog.51cto.com/6228054/1949673