Avoid Costly DBA Mistakes: Real‑World Deletion Cases and How to Prevent Them
This article shares a collection of real DBA deletion mishaps—from accidental Oracle home removal to mistaken rm‑rf commands—and offers practical prevention tips such as aliasing rm, monitoring storage, backing up before deletions, and enforcing peer review to improve operational safety.
1. Case Sharing
1. Accidentally Deleting Oracle Software
During archive log cleanup, a hardware staff member mistakenly removed the entire ORACLE_HOME of node 2 by typing
rm -rfafter an unintended upward keystroke.
2. Space‑Induced Deletion Errors
A root user typed
rm -rf abcwith an extra space before
abc, causing the entire OS directory to be removed.
Another incident involved deleting trace files with
rm orclwhile a slow VPN connection hid a stray space, leading to the removal of the whole directory.
3. Mis‑authorized Space Deletion
While installing a database, a command
chmod 777 -R /oraclewas mistyped with an extra space (
chmod 777 -R / oracle), corrupting many system files and requiring a full system restore from tape.
4. Deleting Critical Data Files
In a development environment, after copying all files of a tablespace, a wildcard
rm *mistakenly removed the data files of the SYSTEM tablespace, forcing a night‑long recovery effort.
5. Script‑Based Deletion Mistake
A backup script that should remove old RMAN logs used a variable for the log directory; the variable was unset, pointing to the user’s home directory and deleting it entirely.
6. Deleting Mounted Directories
During a project migration, after copying a project directory, the original directory was removed with
rm, unintentionally deleting a subdirectory that was a mount point to another server.
7. Deleting a Large Tablespace by Mistake
A temporary tablespace name was confused with a large 100 GB tablespace; the missing “_temp” suffix caused the large tablespace to be dropped, requiring a full re‑extraction of data.
Key lessons: never use wildcards like * without careful verification; fatigue greatly increases error risk, so always double‑check commands before pressing Enter.
2. Prevention Recommendations
1. Alias or Redefine the rm Command
Introduce a policy that replaces direct
rmwith a safer workflow, such as moving files to a temporary location first and only deleting after a verification period.
2. Strengthen Storage Space Monitoring
Do not wait for disks to reach 100 % capacity before cleaning; maintain a safety threshold and perform regular space audits to avoid rushed deletions.
3. Backup Before Emergency Deletions
If a rapid file removal is unavoidable, copy the data to another host or storage medium first, ensuring a fallback option in case the deletion proves harmful.
4. Avoid Deletions During Fatigue or Odd Hours
Schedule risky operations when the team is rested; verify the current working directory with
pwdbefore executing destructive commands.
5. Implement Dual‑Operator Oversight
For critical actions, have two qualified personnel present to review and approve the operation, reducing the chance of unilateral mistakes.
Efficient Ops
This public account is maintained by Xiaotianguo and friends, regularly publishing widely-read original technical articles. We focus on operations transformation and accompany you throughout your operations career, growing together happily.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.