Hand the next owner a broken run

AI helped draft this article and generate its two code-rendered diagrams. I proposed the drill in a comment on Steven Passanante’s article; the results below come only from a synthetic scripted example, not a team handover.

The engineer who owned the revenue pipeline has left. Their name is off CODEOWNERS, someone else’s is on it, and every metric definition is in a readable file with a comment explaining it. By the usual measures, ownership has transferred.

Then a run fails at 6 a.m. because an upstream file arrived late.

Can the new owner fix it? Nothing in the reassignment answers that. File ownership tells you who is responsible. It doesn’t tell you whether they can recover anything.

This piece proposes a cheap way to find out before it matters: hand the next owner a broken run on purpose.

Where this came from

Steven Passanante’s article on the lone wolf problem in data engineering argues that knowledge concentrating in one engineer is an architectural problem, not a cultural one. His fix is structural:

One metric. One file. One owner. Enforced by the version control system, not by culture.

He tests it with a thought experiment: what happens when someone leaves? In his design, “the next steward reads the files, runs dbt docs serve, and understands what they inherited.”

I agree with the direction. In a comment on his article, I suggested making the departure test an exercise rather than a thought experiment:

Give the next steward a known failed run and ask them to diagnose and restore it using the repository, generated catalog, and ordinary team access, without help from the previous owner.

That’s a proposal, not something I’ve run with a team. Steven hasn’t reviewed this piece, and his article’s claims about his own framework are his. What follows is the drill written down, plus a small synthetic version to check that its mechanics do what I say they do.

Reassigning files proves less than it looks

Readable definitions and named owners cover what a pipeline computes. Recovery needs a different kind of knowledge:

• Deployment context. Which bucket, which credentials, which environment variable the job silently expects.
• Recovery decisions. When a late file shows up, do you reprocess it? How far back do you replay? Those are judgement calls someone made once, usually under pressure, and they often live in that person’s head or notes.
• Order of operations. Which upstream job has to run first, and with which flag.

None of that is visible in an ownership file, and none of it shows up until something breaks. A departure test that only checks the files will pass on exactly the platforms where it should fail.

The drill: one failed run, four stages

Pick a failure you’ve actually had and know how to fix. Then hand it to the next owner with one rule: the repository, the generated catalog and ordinary team access are allowed. The previous owner, and anything only they can reach, are not.

The drill runs four stages in order:

1. Find guidance on the component the catalog names for this failure.
2. Check it’s within access: every source the guidance points to is reachable without the previous owner.
3. Execute the steps.
4. Verify the output equals the answer you already know.

Where it stalls is the output. Each stall becomes a missing piece of guidance. The synthetic drill assigns it, by rule, to the component the catalog names for the failure, since that’s where the next steward looks first. Then the drill runs again until it passes.

A synthetic version, with six levels of guidance

To check the mechanics, a small synthetic version was built around a fictional data platform with two owned components, ingest_orders and revenue_daily, a generated catalog, a team-access folder, and the previous owner’s private notes in a folder the drill isn’t allowed to read. One known failed run: revenue_daily aborted because the orders file for October 14 arrived late. The code and results are in a private draft workspace, so this article reports a recorded synthetic run rather than offering a public runnable demo.

The correct recovery takes three things: rerun ingest_orders with a flag that accepts late files, rerun revenue_daily with a two-day replay window to pick up adjustments held during the outage, and set a bucket variable whose value lives in team access.

The steward is a fixed script, a stand-in for a checklist, not a model of a person. The fixture, the drill, the tests and the results are published next to this article. Every file the steward reads goes through a guarded reader that records the path, so the drill can prove it stayed inside ordinary access. Six versions of the same platform differ only in the recovery guidance:

Version	Guidance	Stalled at
D1	Ownership reassigned, no recovery guidance	find guidance
D2	”Rerun revenue_daily” only	execute
D3	Both reruns, no replay window	verify
D4	Complete, but the bucket comes from the previous owner’s notes	within access
D5	Complete, on the failing component	passed
D6	Complete, on the upstream component	find guidance

Source: results.json. The expected stage for each version was written and hashed before the drill code existed; all six matched on the first run, as recorded in the demo’s RUN.md.

Each stall is a different kind of gap

The point of the drill isn’t the pass. It’s that the five failures are distinguishable.

D1 has nothing to find. This is the “ownership reassigned, definitions readable” platform. It’s the one a file-only departure test would pass.

D2 fails loudly. Rerunning the metric hits the same missing partition. That’s the easy kind of gap: the steward knows immediately that they don’t know enough.

D3 fails quietly. Both reruns succeed. Nothing errors. But without the replay window, the restored revenue for the day was 245.5 instead of 231.0, because the adjustments held during the outage were never picked up. Without a known answer to check against, this version reports a successful recovery. That’s the gap I’d worry about most in a real platform: the recovery decision someone made once, never written down, whose absence produces a plausible wrong number.

D4 depends on a person. The guidance is complete, but it says to get the bucket from the previous owner’s notes. The drill stops before running anything, because following that instruction would mean leaving ordinary access. A test checks this boundary too: when the access check is removed, the drill does read the private notes, which the real drill never does.

D6 is in the wrong place. The same complete guidance, written on the upstream component, can’t be found from the failing one. The drill looks one hop upstream when it finds nothing, so it reports this as “guidance exists on ingest_orders, not on revenue_daily” rather than “nothing anywhere”. The fix isn’t more documentation. It’s moving the guidance to where the catalog sends you.

What a pass proves, and what it doesn’t

A pass is narrow evidence: this failure, on this platform, can be recovered by someone using only ordinary access. That’s more than a reassigned ownership file shows, and it’s checkable.

It isn’t proof that operational knowledge has transferred. One failure covers one path. A team that passes a late-file drill can still be stuck on a schema change or a credential rotation. The drill gets stronger with a small rotating set of real past failures, not one. And it measures the guidance plus one steward, not the team.

What I didn’t check

• No real people. The steward is a script that follows a fixed procedure. A person would improvise, search Slack, guess and sometimes succeed for the wrong reasons. Whether real stewards stall in the same places is the hypothesis, not a result.
• No real platform. The job runner is simulated in a few dozen lines of Python. The failure, the late-file flag and the replay window are invented for the fixture.
• The gaps were planted. Each version was built to have one kind of gap. The drill finding them shows the mechanics work. It doesn’t show a real platform’s gaps would be this clean or this findable.
• One failure. Every version uses the same failed run.
• The gap’s owner is a rule, not a finding. The drill always assigns the gap to the component the catalog names for the failure. Deciding where guidance really belongs is still a person’s call.
• The access boundary was hardened after review. The first version of the path check could be fooled by repo/../private/.... It now normalizes paths and rejects .., and a test covers it.
• Sources. Steven’s article and my comment were read on the public page on October 26. His article reports results for his own framework; I haven’t verified them and don’t use them here.

Six steps for a departure drill

1. Pick a real past failure you know how to fix, and write down the correct restored output.
2. Give the next owner only the repo, the generated catalog and ordinary team access.
3. Keep the previous owner out of the room.
4. Run the four stages in order: find guidance, check it’s within access, execute, verify against the known answer.
5. At the first stall, write the missing guidance on the component the catalog names, then run the drill again.
6. Rotate in a different past failure each time ownership changes.

If you’re planning an ownership handover on a data platform and want a second pair of eyes on it, here’s how I work.

---

How this was made: the six versions and their expected stall stages were written and hashed before any drill code. The demo is Python standard library only, and an independent review of the code and claims led to the fixes noted above. The results grid is drawn from the results file; the drill loop is a schematic.