Essential Ops Checklist: Prevent Data Loss, Secure Servers, and Optimize Performance

1. Online Operation Guidelines

Test everything before using production servers; avoid changing SSH settings without a backup of sshd_config. A mistaken rsync command can delete source data, and an accidental rm -rf can wipe entire directories. Always verify commands before execution.

Avoid multiple people editing the same server simultaneously, as conflicting changes can obscure the real cause of an issue.

Backup configuration files (e.g., .conf) before modification, preferably by commenting out original lines and copying the file.

2. Data Handling

Use rm -rf with extreme caution – a single typo can cause irreversible loss.

Backup everything – regular snapshots (e.g., every 2 hours for payment systems, every 20 minutes for loan platforms) dramatically reduce risk.

Prioritize stability over speed – avoid deploying untested software in production; prefer proven stacks.

Maintain confidentiality – protect sensitive data and prevent leaks through proper access controls.

3. Security

Change the default SSH port and disable root login.

Use regular user accounts with key‑based authentication, sudo rules, IP restrictions, and user limits.

Deploy intrusion‑prevention tools (e.g., hostdeny) to block repeated failed attempts.

Audit /etc/passwd for unauthorized users.

Enable a firewall with a default‑deny policy, only opening required ports. Run services with the least privileges possible and enforce fine‑grained permission controls.

Implement intrusion detection and centralized log monitoring (e.g., watch /var/log/secure, /etc/log/message, FTP logs) to quickly identify and respond to attacks.

4. Daily Monitoring

Monitor system resources (CPU, memory, disk, network) and OS‑level events such as logins and critical file changes. Regular hardware health checks help predict failures and guide tuning.

Service monitoring (web, DB, load balancers) tracks key metrics to spot performance bottlenecks early.

Log monitoring for hardware, OS, and application errors provides essential context when incidents occur.

5. Performance Tuning

Understand the underlying mechanisms of software (e.g., why Nginx is faster than Apache) before adjusting parameters.

Follow a structured tuning workflow: identify bottlenecks via logs, define a tuning direction, and adjust one parameter at a time, starting with hardware and OS, then moving to database settings.

Always perform benchmark tests to verify that changes improve performance and meet real‑world workload requirements.

6. Ops Mindset

Maintain composure under pressure; avoid making critical changes when stressed.

Take responsibility for data integrity—regular backups are non‑negotiable.

Conduct thorough root‑cause analysis rather than applying quick fixes.

Separate test and production environments, and double‑check the target machine before executing impactful commands.