The problem
Take an online shop in a monorepo: four Symfony services
(order, stock,
shipping, billing) and the Vue.js
front shop, served by Caddy. A TypeScript SDK
is generated from the OpenAPI document of
order.
A pull request announces a refactoring: the configuration
shared by the four services moves into a common bundle,
app/config. Each service only keeps its
differences. The diff touches several dozen files, and the
description states "Behaviour unchanged, tests pass".
Reading does not allow the reviewer to check that, for three reasons.
The result cannot be read in the diff.
Symfony merges the imported configuration with the service's
own, and the merge rules vary from one option to the next.
Knowing whether order keeps its production
cache pool means replaying that merge in your head, four
times.
The tests do not see everything. They run
in the test environment. A
prod-only setting, lost in the merge, fails no
test.
The description is a claim, not a proof. When a coding agent writes the PR, its prose sounds just as sure whether the check took place or not. The reviewer cannot judge "behaviour unchanged"; they can replay a command.
The same problem comes back with a Caddyfile split into fragments, or a regenerated SDK. By definition, a refactoring does not change external behaviour: a property that can be checked, provided we choose what to compare.
The options
Reading the diff line by line
- What it solves: the reviewer judges names, readability and intent, with no tool to write.
- What it costs: an effort that grows with the size of the diff, for a result that cannot be read in it: merged configuration, imported files, generated code. Google's review practices already advise keeping refactorings apart from other changes.
Relying on the test suite
- What it solves: it already exists, runs in CI and checks the behaviours that matter.
- What it costs: a green suite says that nothing tested broke, not that nothing changed. Production configuration, a Caddyfile or a Dockerfile are rarely covered.
Snapshot tests
-
What they solve: a reference output is
committed, then compared on every run (Vitest's
toMatchFileSnapshot,spatie/phpunit-snapshot-assertionsin PHP). The guard is permanent. -
What they cost: references to maintain,
and
vitest -uaccepts any difference if nobody reviews it. It is a lasting device for a one-off question.
An equivalence proof per refactoring
- What it solves: we capture what the system consumes, before and after, and compare. The diff is empty, or each difference is explained. The reviewer replays the command.
- What it costs: a throwaway script, and a capture of the base to rebuild. The proof only covers what it compares.
Our recommendation
Every refactoring PR carries a replayable equivalence proof: a capture before, one after, their diff, and what the proof does not cover.
The capture covers what the system consumes, not the sources. The compiled Symfony container, the JSON that Caddy runs and the generated code are the real products of a configuration file. Two sources that produce the same artefact are equivalent for the system.
| What we refactor | What we compare | Command |
|---|---|---|
| Shared Symfony configuration | resolved configuration, routes, services |
debug:config, debug:router,
debug:container
|
| Caddyfile split into fragments | adapted JSON | caddy adapt --pretty |
| Configuration or generator of an SDK | generated code |
regeneration, then git status --porcelain
|
| Inherited moon tasks | resolved tasks | moon task <target> --json |
| Reformatting, dependency upgrades | PHPStan baseline, formatter |
sum of the count values,
php-cs-fixer check
|
For a PR written by a coding agent, the rule changes the review. The reviewer no longer judges prose: they replay a command and read a diff. An output pasted into the description can be wrong or stale; a replayed command cannot.
The "Not verified" section completes the proof. It states what no command covered: the application clicked through against a running API, a multi-platform build, the integration in an editor. An empty section with no justification becomes a review question.
The trade-offs we accept
Like any recommendation, this one has a price.
A proof only covers what it compares. A
JSON adapted from the repository says nothing about the
files the image actually copies, nor about production
variables. debug:container compares service
definitions, not their code. The "Not verified" section
names these blind spots.
A throwaway script to write. It is sometimes longer than the change itself. The recipes below give its outline, and it travels in the description, not in the repository.
It does not replace the tests. The proof says what changed; the tests say whether what matters works. A configuration identical before and after can be wrong on both sides.
A difference must be judged. A non-empty diff is not necessarily a failure: two Caddy sites with disjoint hosts can swap places with no effect. The cost of the review moves from reading to judging an explanation.
The signal that should trigger a review of this choice: the same proof rewritten from one PR to the next. It then deserves a permanent place, as a snapshot test or a drift check in CI. Another signal: a regression that got through despite a green proof, which therefore compared the wrong artefact.
Implementation
The examples target Symfony 7.4 LTS and 8.1, Caddy 2.11 and GitHub.
1. A description template
GitHub prefills the body of every pull request with
.github/pull_request_template.md.
<!-- .github/pull_request_template.md -->
## Why
<!-- The need, in two or three sentences. -->
## What
<!-- What changes, what does not. Mechanical commits kept apart. -->
## Verification
<!-- For each proof: the exact command, what it compares, its result. -->
## Not verified
<!-- What no command covered, and why. -->
For the shared configuration, the description reads, for example:
## Verification
- Capture script (collapsed block below) on the base and on the branch,
in dev, test and prod, for `order` and `stock`: `diff -ru` empty.
- PHPStan baseline at the same total; `php-cs-fixer check` exits with 0.
## Not verified
- `shipping` and `billing`: same import, capture not run.
The throwaway script goes into a collapsed
<details> block of the description. The
reviewer can replay it, and the repository keeps nothing of
it.
2. Recipe: a shared Symfony configuration
The script captures, per environment, the resolved
configuration of the extensions, the routes, the services
and the parameters. It runs wherever
bin/console runs, with jq.
#!/usr/bin/env bash
# snapshot-symfony.sh: throwaway, attached to the PR description
# Usage: snapshot-symfony.sh <absolute output folder>
set -euo pipefail
out="$1"
extensions=(framework doctrine doctrine_migrations security api_platform monolog)
# jq -S sorts the keys; the checkout path differs from one capture to the other
snap() {
local file="$1"
shift
php bin/console "$@" --format=json | jq -S . | sed "s#$PWD#<project>#g" > "$out/$file"
}
for env in dev test prod; do
mkdir -p "$out/$env"
for ext in "${extensions[@]}"; do
snap "$env/config-$ext.json" debug:config "$ext" --env="$env"
done
snap "$env/routes.json" debug:router --env="$env"
snap "$env/services.json" debug:container --env="$env"
# --show-hidden only lists the services whose id starts with a dot
snap "$env/services-hidden.json" debug:container --show-hidden --env="$env"
snap "$env/parameters.json" debug:container --parameters --env="$env"
done
The base is the commit the branch started from, not the tip
of main: otherwise the diff mixes in the
changes merged since.
# Base in a separate worktree, branch in the current checkout
git worktree add ../base "$(git merge-base HEAD origin/main)"
(cd ../base/services/order && composer install --quiet && ~/snapshot-symfony.sh /tmp/order-before)
(cd services/order && ~/snapshot-symfony.sh /tmp/order-after)
diff -ru /tmp/order-before /tmp/order-after
Without --resolve-env,
debug:config leaves environment variables as
placeholders: neither their values nor the machine's
.env enter the comparison. Outside debug mode,
as in prod,
debug:container recompiles the container on
every call, without depending on a stale cache.
The script fails loudly, on purpose: two empty captures give
an empty diff. Hashed identifiers, such as
.service_locator.<hash>, derive from
their content: a difference on one of them is explained by
another line of the same diff.
3. Recipe: a Caddyfile split into fragments
The front is served by the official Caddy image, and its Caddyfile becomes a shared file completed by imported fragments. We adapt the configuration inside the image of each revision, to include what the Dockerfile actually copies.
# The JSON that Caddy runs, adapted inside the image of each revision
docker build -t shop:before ../base/fronts/shop
docker build -t shop:after fronts/shop
for rev in before after; do
docker run --rm --env-file fronts/shop/caddy.env.example "shop:$rev" \
caddy adapt --config /etc/caddy/Caddyfile --pretty > "/tmp/caddy-$rev.json"
done
diff -u /tmp/caddy-before.json /tmp/caddy-after.json
{$VAR} variables are replaced before the
Caddyfile is parsed, with the environment of the command:
both captures therefore get the same example file.
{env.VAR} placeholders, resolved at runtime,
stay as they are in the JSON.
A common difference comes from the order of the sites: when
two blocks swap places, their routes can be inverted. The
generated names of their loggers (log0,
log1) also follow that order. With disjoint
hosts, the difference has no effect: the description says
so.
4. Recipe: a regenerated SDK
When the generator configuration moves into a common package, the generated code must not change by a single byte.
# Regenerate with the common configuration, then require an unchanged folder
moon run order-api-sdk:generate-api
changes=$(git status --porcelain -- libs/typescript/order-api-sdk/generated)
[ -z "$changes" ] || { echo "$changes"; exit 1; }
git diff --exit-code ignores untracked files:
an extra generated file would go unnoticed.
git status --porcelain lists them.
Switching generators, from openapi-typescript to @hey-api/openapi-ts for example, changes the output by construction. The proof then targets the input: the old generator, run again on the committed specification, reproduces the old code byte for byte. So the contract did not move, and the type-check of the fronts covers the calls.
5. Two guards: the PHPStan baseline and the formatter
A
PHPStan baseline
records each accepted error with a count. A
refactoring must not raise its total.
# Total errors in the baseline, on the base then on the branch
total() { grep -E '^[[:space:]]+count: [0-9]+$' | awk '{ s += $2 } END { print s + 0 }'; }
git show "$(git merge-base HEAD origin/main):services/order/phpstan-baseline.neon" | total
total < services/order/phpstan-baseline.neon
An equal total does not rule out a swap: one error fixed, another added. If no file moves, the strict proof is an empty diff of the baseline.
php-cs-fixer check exits with 8 when files
still need fixing, 0 otherwise. Reformatting gets its own
commit, which is proven by replaying it: on its parent,
php-cs-fixer fix produces exactly that commit.
6. Requiring it from a coding agent
AGENTS.md, at the root of the repository, is
read by Codex, Cursor or the GitHub Copilot coding agent.
Claude Code reads CLAUDE.md: an
@AGENTS.md line imports it there.
<!-- AGENTS.md (excerpt) -->
## Refactoring pull requests
- Before changing anything, capture what the system consumes: resolved configuration
(`debug:config`, `debug:router`, `debug:container`), Caddy's adapted JSON,
generated code, resolved moon tasks (`moon task <target> --json`).
- After the change, take the same capture again, compare with `diff -ru`
and explain every difference.
- Keep mechanical commits (moves, reformatting) apart from the others.
- Follow `.github/pull_request_template.md`: exact commands and results
under "Verification", blind spots under "Not verified".
These files remain context, not a constraint. An agent that
writes the PR body itself (gh pr create --body)
can also stray from the template. A CI job therefore checks
that the sections are there.
# .github/workflows/pr-description.yml
name: pr-description
on:
pull_request:
types: [opened, edited, reopened, synchronize]
permissions: {}
jobs:
sections:
runs-on: ubuntu-latest
steps:
- name: Verification and Not verified sections
env:
BODY: ${{ github.event.pull_request.body }}
run: |
for section in '## Verification' '## Not verified'; do
grep -qF -- "$section" <<< "$BODY" || { echo "::error::Missing section: $section"; exit 1; }
done
The body goes through an environment variable, as GitHub
recommends against script injection. The job is added to the
checks required on main, next to the
quality gate: kept apart, it reruns when the description changes. It
checks the form; the substance is left to the reviewer.
Checklist
- Refactorings isolated in their own PR, or in their own commits.
- A capture of the consumed artefact, not of the sources, before and after.
- The commit the branch started from as the comparison base.
- The same script, failing loudly, and the same variables for both captures.
- An empty diff, or each difference explained in the description.
- A filled-in "Not verified" section, or a justified empty one.
- Tests still run: the proof does not replace them.
-
The rule in
AGENTS.md, imported byCLAUDE.md, and checked in CI.
Sources
Versions checked on 15 November 2026: Symfony 7.4 LTS and 8.1, Caddy 2.11.9.
- Martin Fowler, Refactoring: the definition of refactoring.
- Google, Small CLs: "Separate Out Refactorings".
- Symfony, debug:config, debug:container, debug:router, recompilation outside debug mode and configuration merging.
- Caddy, caddy adapt, environment variables and logger names.
- Git, git diff, git status and git worktree.
- PHPStan, The Baseline; PHP-CS-Fixer, Usage (exit codes).
- moon, moon task.
- Vitest, Snapshot; spatie/phpunit-snapshot-assertions.
- GitHub, pull request template, collapsed sections, gh pr create, pull_request event and untrusted input.
- AGENTS.md; Claude Code, CLAUDE.md and AGENTS.md.
- Lychen, pull request 243 and pull request 258: an SDK regenerated and compared byte for byte, an open-source implementation of what this article describes, in a moon monorepo.
- Lychen, pull requests 245, 247, 250, 252 and 271: refactorings described with their checks and what was not checked, in the same monorepo.