Build with an AI assistant: a product tour
Does this engineer check what an AI assistant gives them, or ship it?
You’re Diego Alvarez
Candidate
Diego Alvarez makes a small change in a real codebase with an AI assistant. Then you see what the hiring team sees: every request, what the assistant proposed, what Diego did with it, and how each level is set.
9 steps · about 2 minutes
Read the tour as text
What is the task?
Diego has a ticket for a change in a small shipping-quote service. The team’s spec and conventions are on the shelf, the visible tests are one click away, and an AI assistant is on hand. Everything Diego does is recorded.
What do they give the assistant?
Diego reads the spec and the conventions, then asks for the change with both attached. What a candidate attaches, and what they leave out, is part of the record.
Do they check what it proposes?
The assistant answers with a change. In a real task the reply is a live model’s; this one is a scripted sample. It reads plausibly, but it counts whole 5 kg steps where the spec says started ones, and it edits the 27 kg test from $8.00 to $4.00 so the tests pass.
Do they run the tests, and fix what is wrong?
Diego rejects the change, writes the code himself and runs the visible tests. All 17 pass with the original test untouched. Each of these steps is recorded.
Do they say what they checked?
Diego writes a hand-off note for the next engineer: what changed, how it was checked and what was not verified. Then he submits.
What does the candidate see at the end?
In an assessment, Diego gets a submission receipt, not levels, hidden-check results or feedback, so the answers cannot be passed on.
How is each level set?
Rules set each level from what was recorded and from hidden checks Diego never saw. No model grades the work: the assistant is only what Diego works with. The level for catching mistakes needs a test run that failed first, so a change rejected on sight shows in the activity below instead.
What did they ask, and what did they do with it?
You see each request in order: what Diego sent and attached, what the assistant proposed and what Diego did with it. Here the change was rejected, and the next test run is shown beside it. Nothing on this page is a score.
Which of the assistant’s mistakes did they catch?
The task plants mistakes an assistant can make. The rules read each proposal for the ones that show in the code, here the edited test, and check whether the submitted code still has them. Mistakes that need the code run are listed as not read. A person decides what it means.