How AI interview tools actually work
Screen capture, system audio, and overlay windows — a technical look at what these tools do and what they cannot do.
There is a lot of vague marketing in this category and not much explanation. Here is the actual mechanism, which is more interesting than the marketing and also clarifies the limits.
Three inputs
System audio
The interviewer's voice arrives at your machine as audio output. Capturing it is a platform-specific job: macOS exposes it through ScreenCaptureKit, and Windows through output loopback. That audio goes to a speech-to-text model to produce the rolling transcript.
This is why capture works identically across Zoom, Meet, and Teams. The tool is not reading the meeting app — it is reading the sound card.
Screen capture
When you snip a region, the app takes a screenshot of that rectangle and either sends the image to a vision model or runs OCR locally to pull the text out. There is no continuous recording; capture happens when you press the key.
Your context
Resume, job description, role. This is the difference between an answer that cites your actual work and one that could belong to anybody.
The overlay window
The part people ask about most is how the window stays out of a screen share.
Operating systems provide a content-protection flag — set it on a window and the compositor excludes that window from capture APIs. It was built for DRM and for password managers, but it applies to any window that asks.
So the exclusion is done by the OS, not by tricking the meeting app. That is why it behaves consistently across platforms, and it is also why the boundaries are what they are.
What this means for the limits
- The window is excluded from screen capture. It is not hidden from the operating system — the process is visible in Activity Monitor or Task Manager like any other app.
- Tab-level sharing captures a page's own render surface, which is a different path from a desktop capture. That is why every honest guide tells you to share a window instead.
- Proctoring software that specifically looks for overlay windows or background processes is a different problem, and one these tools are not built to solve.
Why latency is the real engineering problem
None of the above is especially hard. The hard part is the clock.
In a live interview, a three-second pause is visible. That budget has to cover audio capture, transcription, a round trip to a language model, and rendering — and the model is most of it. This is why per-surface model selection matters more than it sounds: you want a reasoning model for system design, where you are meant to be thinking anyway, and a fast one for behavioral, where the silence is the problem.