Fix forward doesn't undo a payment: the limit of AI speed when the bug lives in the data

The Pragmatic Engineer's winning essay shows an engineer merging 11x more PRs with agent-only review. Yet the serious incidents he reports all happened in the data. There, reversibility doesn't come from speed: it has to be designed into the pipeline.

The Pragmatic Engineer published the winning entry of its essay challenge, written by Neil Kakkar. He is a London-based engineer who worked at Bloomberg and PostHog and now works at Tano, an AI-native startup. According to the data he collected himself, Kakkar merges about 11 times more PRs per month than he did at PostHog. In most cases the only review comes from an agent, and the team practises fix forward: if something breaks, you fix it going forward. This is one engineer's account, not market data, and Gergely Orosz himself points out that the average he sees across the industry is closer to 2x.

What caught my attention wasn't the multiplier. It was the incidents the author describes so candidly, because every serious one happened in the data layer, which is where I have worked for two decades. My thesis is this: fix forward works for code, but when the error materialises in data, payments or reports, reversibility has to be designed into the pipeline, and the real gate becomes domain rules and automated data-quality checks.

The serious incidents live in the data

The essay describes three cases. In one report, the agent treated minor currency units as floats multiplied by 100, as if every currency had a hundred cents, when the Bahraini dinar is divided into a thousand fulūs. On a Postgres table of about 6 million rows, an index migration blocked writes and ingestion stopped saving new posts. In a migration of the agent architecture some guardrails were missing, and creators were paid twice for the same work.

The currency bug was fixed within the hour, as soon as the team noticed that the report numbers didn't match. That fixes the code, but it doesn't help whoever read the wrong report in the meantime, or the payment that has already gone out. A deploy replaces a line of code. It doesn't undo a transfer or a decision made on the basis of a number.

Kakkar says the product architecture went from a mostly irreversible decision to a reversible one. For the architecture, I agree. Its effects in the world, though, such as money moved and data consumed by other systems, remain as irreversible as ever. That asymmetry is what “move fast and break things” tends to hide.

Reversibility is design, not a side effect

If data doesn't roll itself back, someone has to build the mechanisms that stop the error from reaching it: idempotent payments, raw layers that allow reprocessing, and migrations that don't lock tables. The Postgres case is instructive. The documentation explains that with the concurrent option the index is built “without taking any locks that prevent concurrent inserts, updates, or deletes on the table”. The rule existed. It just wasn't in the path of whoever was generating the code.

Tano made the right move after the incident: the review agent now has a list of the largest tables and doesn't allow blocking migrations on them. On the currency bug, the author notes that a utility for currency calculations already existed and that a human reviewer would have noticed the generated code ignoring it. This matches Bacchelli and Bird (2013), who found reviews to be “less about defects than expected” and more about “knowledge transfer” and “increased team awareness”. If the reviewer is now an agent, that domain knowledge has to be written down for it before the incident, not after.

At GOL, I managed a two-year Oracle-to-SAP ERP migration covering finance, tax, inventory and procurement, with zero critical downtime. In a tax domain, a wrong entry can't be fixed with a deploy, which is why the heavy lifting lies in planning the rollback. That planning is exactly what an agent can speed up, as long as it is given the rules.

The real gate comes after the merge

Even the best reviewer reads the diff, not the data, and some errors only surface when code meets real numbers. The currency bug was caught by an informal quality check: someone saw that the reports didn't match. Turning that observation into an automated gate is the missing step.

At WEX, in a banking context bound by financial regulations, I automated data-quality checks with Great Expectations, and critical data incidents fell by 30%. At BairesDev, I implemented observability with DataDog and Monte Carlo, using ML-based anomaly detection to enforce data-quality SLAs. None of this makes anyone slower at writing code, but it stops a fast mistake from becoming an expensive one. A check that blocks two payments to the same creator for the same content is cheap to write and acts before the money leaves.

Kakkar himself comes close to this when he imagines an agent that would flag “when we paid one creator twice because of a bug in our migration”. That is data observability, and it doesn't require unlimited tokens. For transparency: I use Claude in the pipeline behind this newsletter, the same family of tools Tano used to build its review agent. So I'm not writing as a sceptic of the technology. I'm writing about where to put the lock.

The speed objection

The strongest objection is that all of this is a brake, and startups live on speed. Kakkar says the trade-off was a conscious one, and for a company of about ten people rewriting its own architecture, it may well be the right call. Gergely adds that businesses that already work have good reason to change their processes more carefully.

The line I'm proposing, however, isn't company size: it's the reversibility of the operation. And the speed gain is less certain than it looks. In a controlled experiment, the group with Copilot finished a task 55.8% faster (Peng et al., 2023). METR, by contrast, measured experienced open-source developers using early-2025 tools and found them 19% slower (Becker et al., 2025). The settings are too different to generalise either result, which is already a reason not to trade safety for uncertain speed. And the same AI that generates the code writes the idempotency test and the migration rule in minutes: the cost of the lock has fallen along with the cost of the code.

What a deploy can't undo

Fix forward is a good strategy for anything a deploy can undo. The essay's incidents show where it stops: at currency calculations, at large-table migrations and at payments. At those points, reversibility is a property of the pipeline, and the real gate combines domain rules in the reviewer with automated checks on the data.

My practical recommendation is this. Before allowing merges with agent-only review, write down the list of operations a deploy can't undo, such as payments, regulatory reports, published data and large-table migrations. Then turn each item into a rule for the review agent and a quality test in the pipeline. Everything else can move at whatever speed the team can sustain.

References