In short
Companies increasingly ask candidates to code live with an AI agent while the interviewer watches. There is no published standard yet, but hiring managers describing their own rubric consistently grade four things: planning before prompting, delegating narrow subtasks, reviewing the agent's output critically, and insisting on tests.
A format that barely existed a year ago is now showing up in senior engineering loops: the interviewer asks you to share your screen, hands you a problem, and expects you to solve it with an AI coding agent running. You are not being tested on whether you can write a binary search from memory. You are being tested on how you drive the tool.
The problem is that nobody publishes the rubric. In early August a thread in r/ExperiencedDevs asked exactly this question, and the most-upvoted reply was pure exasperation: buy a crystal ball, roll dice, there is no knowing, because some interviewers will fail you for using the agent too much and others for using it too little. That answer collected more support than any concrete advice in the thread, which tells you how unsettled this is.
But underneath the frustration, several people who actually run these interviews described their criteria, and they converged more than you would expect.
The four behaviours that keep coming up
Hiring managers describing their own scoring in that discussion named a consistent set of behaviours:
- Plan before you prompt. Use the agent's planning mode, and iterate on the plan until it is actually right, before any code is generated. Candidates who prompt straight into implementation read as careless.
- Delegate narrow, well-scoped subtasks. Handing the agent a large vague goal is the most common failure. Splitting the work into pieces small enough to verify is the skill.
- Review the output like a reviewer, not a recipient. Reading the diff critically and rejecting parts of it is scored positively. Accepting a passing result without inspecting it is scored badly even when the code works.
- Insist on tests. Adding tests without being asked came up repeatedly as the clearest separator between candidates.
Notice that none of these are about prompt wording. They are the same judgement a good senior engineer applies to a junior's pull request. That is the actual thing being measured.
Ask what they want to see. Out loud. First.
Because the standard is genuinely unsettled, the highest-value thing you can do is refuse to guess. Another well-received comment in the thread framed the unstated-expectations version of this interview as a classic bring me a rock setup, and advised getting the interviewer to disclose their actual expectations in the first few minutes.
A reasonable opening: "Before I start, do you want to see me lean on the agent heavily, or do you want to see how I work when I mostly write it myself? I can do either, and I would rather show you the one you are assessing."
No competent interviewer penalises that question. It reads as senior. And the answer changes your entire approach.
Narrate everything
In a traditional coding interview, silence while you think is tolerated. In this format it is fatal, because the visible artifact is the agent's output rather than your reasoning. If you do not narrate, the interviewer sees a tool producing code and has no evidence about you at all.
Say what you are about to delegate and why. Say what you expect back. When the diff arrives, say what you are checking and what you would reject. When you accept something, say why it is safe to accept. The narration is the interview.
The interview is also information about them
It is worth stating plainly: if a company cannot articulate what a good performance looks like in a format they chose, that is data about the team. Engineers in these threads treated unstated criteria as a signal about internal clarity, and adjusted their read of the company accordingly. You are allowed to do the same.
How to practise
Take a small feature in a personal project and do a timed run with an agent while talking out loud, ideally recording yourself. Write the plan first. Delegate one narrow piece. Read the diff aloud and name one thing you would reject. Write tests. Then watch the recording and ask whether someone listening would conclude you were in control of the tool, or the other way around.
That question is, more or less, the rubric.
Frequently asked
What is a live AI-agent coding interview?
A screen-shared interview where you solve a coding problem using an AI coding agent such as Claude Code, Cursor or Copilot, rather than writing every line yourself. The interviewer is assessing how you direct, constrain and verify the agent, not whether you can recall syntax.
Should I use the AI agent aggressively or barely at all?
Ask before you start. Interviewers in this format disagree sharply with each other, and some will penalise heavy use while others penalise avoidance. Two minutes spent asking what they want to see is the single highest-value move in the interview.
What do interviewers say separates a strong candidate?
Practitioners describing their own rubric point to the same behaviours: writing and iterating on a plan before generating code, scoping work to narrow subtasks, reading the generated diff critically rather than accepting it, and adding tests without being prompted.
Is it a red flag if a company cannot explain their criteria?
It is at least a signal worth noting. Engineers in these threads describe unstated-expectation interviews as a poor sign about how the team defines success internally. You are allowed to evaluate the interview process as evidence about the job.
How do I practise for this format?
Rehearse narrating your process out loud while pairing with an agent on a small feature: state the plan, delegate one piece, read the diff aloud, point out what you would reject, then write tests. The narration is most of what is being graded.