4.5 Git Data Recovery & Sensitive-Data Scrubbing

4.5 Git Data Recovery & Sensitive-Data Scrubbing

Two failure modes look similar and demand opposite responses. A developer who destroys local work needs recovery, and Git almost always still holds the objects. A developer who commits a credential needs scrubbing plus rotation, because deletion alone leaves the secret in history and in every existing clone.

Git Disaster Recovery: Salvaging Lost Commits & History

A fundamental architectural strength of Git is that it rarely deletes any object immediately. When an engineer accidentally deletes a local branch, executes a git reset --hard, or loses work in a detached HEAD state, the commit objects remain intact inside the local object database as "dangling commits" until Git's garbage collection (git gc) purges them (typically after 14 to 30 days).

Tool 1: The Reference Log (git reflog)

While git log displays the commit history of the current branch, git reflog tracks every single movement of the HEAD pointer on your local machine. Every commit, checkout, rebase, cherry-pick, and reset appends an entry to the reflog.

# Inspect the chronological movement of HEAD
git reflog

Sample Reflog Output:

7f1a34d (HEAD -> main) HEAD@{0}: reset: moving to HEAD~2
e4b219c HEAD@{1}: commit: Implement Stripe payment webhook
9a8c12b HEAD@{2}: commit: Add user profile database migration
3c4d5e6 HEAD@{3}: checkout: moving from feature/payment to main

Recovery Scenario A: Recovering from an Accidental git reset --hard

1. A developer accidentally executes git reset --hard HEAD~2, obliterating two commits from main.
2. Running git log shows those commits are gone.
3. The developer runs git reflog and sees that prior to the reset, HEAD was at commit e4b219c (HEAD@{1}).
4. The developer restores the commits instantly:

# Move the branch pointer back to the commit before the reset
git reset --hard e4b219c

Recovery Scenario B: Recovering an Accidentally Deleted Branch

1. A developer deletes a branch containing 3 days of unpushed work: git branch -D feature/analytics.
2. The developer runs git reflog and searches for the last commit on that branch:

git reflog | grep feature/analytics
# Output: 9a8c12b HEAD@{4}: commit: Complete event tracking pipeline

1. Recreate the deleted branch pointing to that commit SHA:

git branch feature/analytics 9a8c12b

Tool 2: git cherry-pick for Selective Commit Salvage

git cherry-pick <commit-sha> applies the exact diff introduced by a specific commit from any branch and commits it cleanly onto the currently active branch.

• Operational Scenario: An engineer accidentally commits a critical bug fix to the wrong branch (e.g., committed to an experimental sandbox branch instead of main).
• Remediation: Checkout main, run git cherry-pick <commit-sha>, verify via test suite, and push. Then checkout the sandbox branch and reset or revert the commit.

Safe Rollbacks on Public Branches: git revert vs. git reset --hard

[!WARNING] AZ-400 Exam Rule: Never use git reset --hard on public, shared branches (main, release/*)! Resetting moves the branch pointer backward, rewriting history. Pushing this change requires --force, which breaks the clones of every other developer on the team and violates compliance audit trails.

On any shared branch, always use git revert:

# Safe forward-moving rollback: creates a NEW commit that applies the inverse diff
git revert 4c2a1e8

# Reverting a merge commit (must specify mainline parent index, typically 1)
git revert -m 1 9b3d2f1

git revert preserves the entire historical timeline, passes automated CI/CD branch policies without force-pushing, and provides transparent auditability.

---

Scrubbing Sensitive Data & Leaked Credentials

One of the most dangerous security incidents in DevOps occurs when a developer accidentally commits a private key, API secret, connection string, or .env file into a Git repository.

The Invalidation of "git rm":

Commit 1: Add appsettings.json (Contains Prod SQL Password!) ──► Blob written to .git/objects/
                                                                    │
Commit 2: Developer runs 'git rm appsettings.json'             ──► Blob STILL EXISTS in Commit 1!
                                                                    (Accessible via git checkout)

Why git rm Does NOT Delete Secrets

Running git rm password.txt and committing simply records that the file is not present in the new commit's tree. The secret remains permanently stored in the blob corresponding to the previous commit. Anyone cloning the repository or running git checkout HEAD~1 can inspect the plaintext secret.

The Modern Scrubbing Tool: git-filter-repo

Historically, teams used git filter-branch to rewrite history. However, git filter-branch is deprecated by the official Git project due to glacial performance, shell escaping vulnerabilities, and history corruption risks.

The modern, officially recommended Python tool is git-filter-repo (or the high-speed Java utility BFG Repo-Cleaner).

# Prerequisites: Install git-filter-repo via package manager
pip install git-filter-repo

Scrubbing Scenario 1: Purge an Entire Sensitive File from All History

# Make a fresh mirror clone of the repository first
git clone --mirror https://dev.azure.com/myorg/myproject/_git/myrepo
cd myrepo.git

# Completely delete credentials.json from every commit, tree, and tag
git-filter-repo --path credentials.json --invert-paths

Scrubbing Scenario 2: Redact a Specific Leaked Password Across All Files

Create a replacement file expressions.txt containing the leaked string and its replacement:

password123===>REDACTED_SECRET
Server=tcp:prodsql.database.windows.net===>Server=tcp:localhost

Run git-filter-repo to replace matching strings across all files in all historical commits:

git-filter-repo --replace-text expressions.txt

The Mandatory 4-Step Incident Remediation Protocol

Purging Git history with git-filter-repo is only one technical step in a complete security incident response. Exam AZ-400 strictly tests the end-to-end operational protocol when credentials leak:

End-to-End Leaked Credential Remediation Lifecycle:

Step 1: Credential Rotation (IMMEDIATE) ──► Invalidate leaked key in Azure Key Vault / Entra ID
                 │
Step 2: Rewrite Git History             ──► Run git-filter-repo on a bare mirror clone
                 │
Step 3: Force-Push Cleaned History      ──► git push origin --force --all --tags
                 │
Step 4: Mandatory Team Re-Clone         ──► Delete local developer clones to prevent re-infection

1. Step 1: Immediate Credential Invalidation & Rotation (Highest Priority):
   • The instant a secret is pushed to a remote repository, assume it has been compromised by automated scraper bots or caching proxies.
   • Immediately navigate to Azure Key Vault, Microsoft Entra ID, or the affected database/service and rotate the secret, regenerate API keys, and revoke active tokens.
   • Exam Trap: Scrubbing the Git repository before rotating the key is an anti-pattern; attackers who already cloned the repository can exploit the active secret while you spend hours rewriting history!
2. Step 2: Rewrite History with git-filter-repo:
   • Execute git-filter-repo on a clean bare clone to excise the file or string from every commit and tag.
3. Step 3: Force-Push Rewritten History:
   • An authorized administrator temporarily enables Force push permissions in Azure Repos and pushes the rewritten refs:

     git push origin --force --all
     git push origin --force --tags
     

4. Step 4: Coordinate Mandatory Team Re-Clones:
   • Instruct all developers to delete their local repository folders completely and perform a fresh git clone.
   • Why this is mandatory: If any developer runs git pull from their existing local repository, Git will merge the rewritten remote history with their old local history, resurrecting the deleted secret blobs back into the remote repository!