The AI-native interview | Sierra<br>Skip to main contentThe AI-native interview
Vijay Iyengar
Arya Asemanfar
Angie Wang
Share
April 22, 2026<br>Subscribe to the Sierra blog<br>Get notified about new product features, customer updates, and more.
Coding agents like Codex and Claude Code are upending software engineering as we know it. The role is shifting from building the machine to designing and honing it. Much like engineers stopped worrying about how a compiler translates code into machine instructions, we now need to focus less on the precise lines of code that are written and more about whether it produces the right outcomes over time.<br>This shifts what we should evaluate in interviews. When a single engineer can build across the stack, leverage comes from combining technical ability with product thinking and business context. They don’t just write code. They define scope, make tradeoffs, and iterate with customers to deliver impact. We’ve redesigned our engineering interview process from the ground up to reflect this new reality.<br>Framing the problem<br>Sierra’s engineering interview process had been fairly standard: two coding interviews plus interviews for algorithms, system design, and culture fit, followed by reference checks. It’s a well-understood, scalable approach, and for a long time it worked.<br>But recently, something started to feel off. Much of the signal we got from this interview was about mechanics; typing syntax into an editor, remembering algorithm details, stitching frameworks together. This felt increasingly dissonant from the new reality of our work. The gap showed up most clearly in debriefs. In the absence of clear interview signals, hiring managers leaned more heavily on referrals and prior experience.<br>We started building an AI-native interview process with three key attributes:<br>li>ul]:my-2">Representative: Reflects the work engineers actually do day to day, capturing initiative, ownership, judgment, system understanding, and product thinking.<br>High signal: Gives us clarity about where a candidate could excel, and where they may need support.<br>Positive experience: Feels engaging and authentic for candidates, so that when we make an offer, they’re excited to say yes.<br>Introducing the AI-native onsite<br>We removed our coding and algorithms interviews and replaced them with an AI-native onsite:<br>li>ul]:my-2">Plan: A working session with the candidate to define a product to build. The candidate drives ideation, while interviewers ask questions to strengthen it. We focus on an idea in the candidate’s domain so we see their product thinking in action.<br>Build: The interviewer steps out and the candidate brings the idea to life over 2 hours, using the AI tooling and frameworks of their choice. They have complete freedom to pivot or adjust scope as they go.<br>Review: The candidate demos what they’ve built. We debate the key product flows and choices they made; review the code to understand their technical judgment (data model, abstractions, extensibility, etc.); and discuss the path to production. We also dig into how they used AI along the way.<br>We’ve found this new format to be much more effective. Because candidates can actually build during the onsite — rather than just talk about what they might build — it’s both more representative of the work, and produces higher signal. It’s much easier to gauge their agency (do they pivot when they get stuck?) and judgement (how do they scope what to build within the time constraints?).<br>It’s also more engaging, even if candidates are nervous at the start. To set expectations, we share evaluation criteria and advice ahead of time. For example, it’s OK to cut your scope as you build, and to skip boilerplate (CRUD, auth) to focus on what’s unique. As Paul Buchheit, the creator of Gmail, put it: if it’s great, it doesn’t have to be good.<br>Rounding out the rest of the process<br>As we’ve honed the AI-native onsite, we’ve also rethought the rest of the interview process. Our coding phone screen still required candidates to write code without AI into an online editor. But vibe-coding an app is easy. The harder, more relevant, problem is getting it into production in a scalable way. So we replaced the phone screen with a system design interview to better reflect that.<br>While the AI-native onsite tests for product sense and building 0->1, it doesn’t capture taking a feature from 1->N in an existing, messy codebase. To address this, we’re piloting a debugging interview. Candidates are given a medium-sized codebase and a draft PR from a colleague that introduces a cross-cutting feature. Their job is to review and improve it — pulling down the code, inspecting the output, and iterating with coding agents to make it better. The level of AI used in this interview is still TBD, as new models can zero-shot many fixes.<br>What did we learn?<br>We’re hiring for strengths, not just an absence of weakness. This approach gives us much richer signal about a candidate’s spikes and gaps. For...