Episode 6. Your interview loop tests a skill your agents already have. Here are the four that actually predict performance.
Where We Are in the Series
Episode 4 laid out the four pillars of agentic engineering – Context, Judgment, Verification, Orchestration. Episode 5 covered the ladder engineers climb to acquire them.
Both assumed something I never examined: that you can find these people. With the loop most orgs are running, you can’t.
The Arithmetic
In April 2026, Sundar Pichai disclosed that around 75% of new code at Google is AI-generated and approved by engineers – up from 50% six months earlier.
Now run that through your interview. If three quarters of production code arrives generated and the engineer’s contribution is approval, then a loop built around unassisted code production is measuring the shrinking quarter – under a stopwatch, on a puzzle, with the reference material removed.
You are optimising your most expensive process for the part of the job that’s disappearing fastest. The part that’s growing deciding, directing, verifying – appears nowhere in it.
Why the Old Loop Broke
Algorithmic interviews were never about algorithms. They were a proxy for problem decomposition under pressure. Imperfect, but not worthless. Three things killed the proxy:
- The skill became a commodity. Any model solves these in seconds. The proxy now separates preparation, not candidates.
- It selects for free time. It always favoured people between jobs over engineers who spent three years shipping. Tolerable when the signal was decent.
- It’s no longer scoreable. Unproctored take-homes and async code tests have become close to unfalsifiable, and every hiring manager knows it.
What I’d keep: one unassisted round still tells you how someone thinks when nothing is filling the gaps. The mistake isn’t having it. The mistake is having it be the whole loop.
The Industry Already Moved – And Is Split
Canva has expected candidates to use AI in technical interviews since June 2025, and rewrote its questions to be ambiguous enough that one prompt can’t finish them. Meta added an AI-enabled coding round in late 2025 while deliberately keeping an AI-free one alongside it. Shopify asks candidates to integrate AI output into unfamiliar codebases. Google is adding a code-comprehension round that explicitly grades prompt quality and output validation.
But Amazon and Anthropic both keep AI out of the live round – and Anthropic sells an AI assistant. They’re not laggards.
The split isn’t a disagreement about whether AI matters. It’s a disagreement about what a single round can isolate. Both positions are defensible. What isn’t defensible is running only the old kind of round and being surprised your hires can’t operate agents.
The Four Signals
Mapped to the four pillars. Each one has a format and a question that’s hard to fake.
1. Context – can they set an agent up to succeed? Unfamiliar repo, deliberately vague ticket, AI encouraged. Watch the first fifteen minutes and stay quiet. Strong candidates gather before they generate and ask what’s out of scope. Weak ones paste the ticket in and press enter. Beware the fake tell: enormous prompts are volume, not context engineering. → “What did you decide not to give it, and why?”
2. Judgment – can they decide what shouldn’t exist? Hand them a 300-line PR that works, passes tests, and is architecturally wrong. Say “review this.” Strong candidates reject it and price the rejection – what it costs in eighteen months, and what the forty-line version looks like. Weak ones approve with naming nitpicks. → “What would you delete?”
3. Verification – can they catch a confident failure? Give them agent output with a subtle bug that passes the supplied tests. Strong candidates read before they run and check the diff against the requirement, not the test suite. This round predicts on-call performance better than anything else in the loop. → “How do you know it did what it said it did?”
4. Orchestration – can they decompose for parallel work? Ask them to split a feature into pieces three agents could run in parallel without colliding. Strong candidates name shared state as the risk immediately and define contracts before tasks. Weak ones split by file and assume the merge is trivial. → “What happens when two of them disagree?”
A Loop You Could Run Next Month
| Round | Time | Tests |
|---|---|---|
| Retrospective conversation | 30 min | “Something you shipped and later regretted” |
| Live agentic build (real repo, seeded bug) | 90 min | Context + Verification |
| The rejection review | 45 min | Judgment |
| Decomposition design | 45 min | Orchestration |
| Fundamentals, AI-free (optional) | 45 min | Raw reasoning – weight it at a fifth, not four fifths |
Three rules: score the process, not the artifact; put the AI policy in the invite; and calibrate on three of your own strong engineers first. If your best people score badly, the rubric is wrong – not them.
The Junior Problem
The apprenticeship ladder ran on juniors doing the small work seniors didn’t want. That work was never valuable in itself , it was the tuition through which judgment got paid for. Agents do all of it now, so a lot of orgs have quietly stopped hiring juniors.
That’s a defensible one-year decision and a bad five-year one: you’re consuming a supply of senior engineers you’ve stopped producing. Interview juniors for reading appetite, verification instinct, and how they respond to correction mid-interview – then build the tuition back deliberately, because it no longer arrives as a byproduct.
Closing Thought
A legacy interview loop isn’t a decision. It’s institutional memory , a record of what used to be true. Nobody chose to keep testing binary tree inversion in 2026. It’s that no one owned the decision to stop.
Hiring last year’s engineers for next year’s work is the most expensive mistake available right now, because every hire made through the old loop is a five-year commitment to the old paradigm.
Don’t redesign everything. Add one round. Put a wrong-but-working PR in front of your next three candidates and ask what they’d delete. You’ll learn more in forty-five minutes than the current loop tells you in four hours.
Then run it on your existing team, and see what that tells you.
Earlier: Ep 1 · Ep 2 · Ep 3 · Ep 4
Khurram Bilal