4+ Years of Ops Mastery: Troubleshooting, Optimization, and Essential Mindset

Accumulated Experience

After more than four years in operations, the author reflects on how the knowledge and operational systems have evolved, noting the rise of big data and artificial intelligence (especially Python) as current trends.

Beginner Section

See the detailed guide on essential Linux operations engineer skills.

Topology diagram:

Intermediate Section

Interview experiences reveal that merely installing LAMP/LNMP is insufficient; interviewers expect deeper knowledge of Nginx reverse proxy, MySQL optimization, and performance tuning.

Key points about reverse proxy:

Provides layer‑7 load balancing, enabling horizontal scaling and improving concurrency and resilience.

Offers local caching of static resources, reducing backend load and enhancing performance.

Core troubleshooting skills (hard to master in study alone):

Analyzing why a program fails or behaves unexpectedly, tracing system calls.

In‑depth system bottleneck analysis.

Typical log‑management commands and system information are illustrated with screenshots.

1. Primary Troubleshooting

Analyze causes of program failures and trace system calls.

Perform deep system bottleneck analysis.

Examples of log analysis and performance tools (top, sar, vmstat, iostat, ipcs) are shown.

2. Optimization

Optimization is the most valuable skill for ops engineers, directly affecting salary, but it must be done carefully to avoid downtime.

Optimization involves adjusting parameters based on actual hardware and workload. The author shares JVM/Tomcat tuning parameters, including standard, non‑standard, and unstable options, as well as collector settings and garbage‑collection statistics.

Sample JVM options:

-Xms (initial heap size)

-Xmx (maximum heap size)

-Xss (thread stack size)

-XX:NewSize, -XX:NewRatio, -XX:SurvivorRatio, -XX:MaxPermSize

-XX:+UseSerialGC, -XX:+UseParallelGC, -XX:+UseConcMarkSweepGC

-XX:+PrintGC, -XX:+PrintGCDetails, -Xloggc:filename

Memory tuning results show a reduction from ~605 requests/second to ~435 requests/second after applying Tomcat optimizations.

3. Development Skills

Shell scripting is often insufficient; Python is recommended for automation, configuration management (SaltStack), batch execution (Fabric), monitoring plugins, virtualization (python‑libvirt), process management (Supervisor), and cloud platforms (OpenStack).

Mastering Python can significantly increase salary.

4. Awareness

Security awareness: Protect accounts and private keys, use encrypted storage tools, avoid cloud drives, and add passwords to SSH keys.

Preparation awareness: Understand the principles behind any configuration before applying it.

Planning awareness: Document complex changes, obtain peer review, and prepare rollback plans.

Documentation and sharing awareness: Record special cases and share knowledge to avoid repeated mistakes.

Monitoring awareness: Monitoring is the eyes of operations, essential for detecting anomalies.

Business awareness: Know the services each host provides and their interdependencies to speed up fault isolation.

Operations mindset is crucial; technical skill alone is not enough without proper awareness of security, planning, documentation, monitoring, and business impact.