How a Single Git Mistake Rolled Back Three Days of Production Code

Incident Overview

On a Friday afternoon the team prepared to release a new version. The normal workflow was:

git checkout main
git pull origin main
git merge feature/new-function
git push origin main

During the merge a conflict occurred. After resolving it, the developer mistakenly executed:

# ❌ Wrong commands
git reset --hard HEAD~3
git push --force origin main

The main branch was reset to three commits earlier, losing three days of work from multiple developers.

Impact Assessment

Running git reflog revealed the lost commits:

a1b2c3d HEAD@{0}: reset: moving to HEAD~3   ← rollback
d4e5f6g HEAD@{1}: commit: feat: payment feature   ← lost commit 1
h7i8j9k HEAD@{2}: commit: feat: order optimization   ← lost commit 2
l1m2n3o HEAD@{3}: commit: fix: user login   ← lost commit 3

Three commits affecting five files impacted three teammates.

Emergency Recovery

Step 1 – Recover the Local Branch

# Find the lost commit via reflog
git reflog
# Example: d4e5f6g is the needed commit
# Create a temporary branch pointing to it
git branch restore-branch d4e5f6g
# Verify the restored content
git log restore-branch --oneline

Step 2 – Restore the Main Branch

# After confirming the content, reset main
git checkout main
git reset --hard d4e5f6g  # back to the lost commit
# Push without forcing
git push origin main

Step 3 – Notify the Team

【Incident Notice】
Main branch was mistakenly rolled back and restored at 15:30.
Affected files: OrderService.java, PaymentService.java, UserController.java
Please verify your code.

Root‑Cause Analysis

Used git reset --hard instead of git revert reset deletes history; revert preserves it.

Had revert been used, the incident would not have occurred.

Used git push --force on a shared branch --force overwrites remote history.

Using --force-with-lease would have checked for conflicts first.

Insufficient branch protection

Main had no protection rules, no mandatory PR reviews, and no restriction on force pushes.

Preventive Measures

1. Branch Protection Rules

# Example GitHub branch protection configuration for "main"
Require pull request reviews before merging
Require status checks to pass before merging
Require branches to be up‑to‑date before merging
Require conversation resolution before merging
Do not allow bypassing the above settings
Restrict who can push to matching branches

2. Personal Operation Guidelines

# ✅ Use revert instead of reset for already pushed commits
git revert HEAD~2..HEAD
git push origin main

# ✅ Use --force-with-lease instead of --force
git push --force-with-lease origin main

# ❌ Avoid this dangerous combo
git reset --hard HEAD~3
git push --force origin main

3. Aliases to Reduce Risk

# .gitconfig example
[alias]
  push-safe = push --force-with-lease
  re = reflog
  undo = reset --soft HEAD~1

Post‑Incident Summary

The team enabled branch protection on all core branches, prohibiting direct pushes. The key lesson is that the accident itself is not scary; repeating the same mistake is.

If your main branch lacks protection, configure it today.

Quick Recovery Cheat Sheet

# 1. View reflog
git reflog

# 2. Recover the lost commit
git checkout -b restore-branch <commit-hash>

# 3. Reset main
git checkout main
git reset --hard <commit-hash>
git push origin main

# 4. Safe push
git push --force-with-lease origin main