Before AI Refactors Legacy Code: Lock Down These Behaviors First

Clarify the Scope of Change

Before asking an AI agent to refactor a legacy method, the team must agree on what is being changed and which observable behaviors must remain identical. The author illustrates this with a hypothetical bulk customer import function that currently mixes file parsing, validation, persistence, and result aggregation. Martin Fowler's definition of refactoring — improving internal structure without altering observable behavior — serves as the guiding criterion: method and class names may change, but results that users and callers depend on must not shift silently.

For example, if the original implementation allows valid rows to succeed while invalid rows fail, but the new implementation rolls back the entire batch on any error for "transactional consistency," the business rule has changed. This is no longer pure refactoring and must be treated as a behavioral change with its own justification.

Document Invisible Consequences

External behavior includes more than the UI success message. Database writes, duplicate notifications, batch retry responses, and permission checks are all part of the contract. The author recommends creating a baseline table with stakeholders, listing scenarios, confirmed expectations, and verification evidence. The article provides a concrete example table for the import feature:

All records valid: One batch created, corresponding customer records written. Verified by batch result, record IDs, and count.

One row invalid: Valid rows retained per convention, error rows explained. Verified by row-level results matching final data.

Duplicate batch submission: Returns existing batch and current state, no new import created. Verified by consistent batch ID and no duplicate writes.

Cross-department data import: Rejected per permission rules. Verified by rejection record and absence of unauthorized writes.

Response lost after write: Subsequent query can confirm result. Verified by batch status reconciling with actual data.

This table must be reviewed with business experts because systems differ: some allow partial success, others require full rollback. Duplicate submissions may return progress rather than waiting for final result. Code reveals current implementation but not intent; discrepancies between documentation and code must be resolved by those who know the business and callers, not by assuming documentation updates justify changing behavior.

Create Repeatable Comparison Samples When Tests Are Missing

Legacy systems often lack tests. The approach is to record the current implementation's actual outputs for typical inputs, then turn those recordings into repeatable checks. These tests describe the status quo; they do not automatically prove correctness. The author has the agent map call entry points, data flows, and exception paths, then humans select the most critical business behaviors to protect. Samples should cover normal, boundary, and historically problematic cases. Production data must be authorized and anonymized; synthetic data is preferred where sufficient.

When comparing old and new results, timestamps and random IDs will naturally differ and can be normalized by prior agreement. However, amounts, permissions, states, and notification counts must never be excluded to make tests pass. For external write operations, never run old and new implementations simultaneously against real systems — that would duplicate data and notifications. Instead, replay the same inputs in an isolated environment, or have the new implementation compute results while logging intended writes for inspection.

Refactor Incrementally, Verify Continuously

With baselines established, extract parsing, then validation, then persistence. After each step, rerun the relevant checks to confirm no unapproved differences were introduced. AI tends to tidy adjacent code opportunistically; the task must explicitly constrain which modules may change, which interfaces and business rules are invariant, and that other issues should be reported as suggestions rather than fixed immediately. Otherwise a local refactor can expand into a half-system rewrite.

For interface or data structure migrations, the author references Danilo Sato's Parallel Change pattern (on Martin Fowler's site): allow old and new versions to coexist, migrate consumers, then retire the old version. This requires maintaining compatibility during transition and assigning responsibility for eventual cleanup. Not every small change needs a dual-version mechanism; an internal pure function may need only a few valuable tests, while cross-service interfaces, historical data, and multiple clients demand explicit migration plans.

Ensure Rollback Compatibility for Data and Code

Rolling back code does not guarantee data rolls back. If the new version wrote fields the old code cannot read, or changed identifier formats, a simple rollback may cause a second failure. The article shows an illustration of this risk.

Acceptance records must enable the next person to understand: which version was tested, with which samples, in which environment, and what remains untested. For migrations, document how long old and new versions must coexist, and who will reconcile and repair data if problems arise. Even if an agent claims "logical equivalence," the author insists on seeing the old-vs-new result comparison: are differences explained? Were critical failure scenarios checked? Passing these checks only covers the examined scope; it cannot guarantee no omissions.

Only when these questions are answered can the team decide whether the refactor is mergeable. Cleaner code is a benefit; the required outcome is that dependent business functions still work correctly.

References

Martin Fowler, Refactoring: https://refactoring.com/ Danilo Sato, Parallel Change:

https://martinfowler.com/bliki/ParallelChange.html