project · 2024 · EPAM Innovation Garage
Automated Interview Assessment Platform
A fully automated interview platform that presents questions to candidates over a live call, records their responses, evaluates answers using NLP-based semantic analysis, and produces a structured assessment report without any human interviewer involvement.
Hiring at scale has a fundamental bottleneck: every first-round interview requires a human to be present, listen, and evaluate. This platform removes that constraint entirely. Candidates receive questions over a live call, record their responses, and receive an assessment, all without a human interviewer in the loop.
How the automated interview works
┌─────────────────────────────────────────────────────┐│ Candidate Session ││ (browser-based, no app install) │└───────────────────────┬─────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────┐│ Question Delivery ││ ││ Questions displayed on screen one at a time ││ Timer per question, configurable per role │└───────────────────────┬─────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────┐│ Response Recording (Audio + Video) ││ ││ Live call recording stored to AWS S3 ││ Per-question segments captured separately │└───────────────────────┬─────────────────────────────┘ │ ┌─────────────┴─────────────┐ ▼ ▼┌──────────────────┐ ┌────────────────────────┐│ NLP Analysis │ │ Integrity Monitoring ││ │ │ ││ Transcription │ │ Eye movement tracking ││ Semantic match │ │ Off-screen gaze flags ││ Summarization │ │ Anomaly scoring │└──────────┬───────┘ └──────────┬─────────────┘ └─────────────┬────────────┘ ▼┌─────────────────────────────────────────────────────┐│ Assessment Report ││ ││ Per-question scores · Answer summaries ││ Integrity flags · Overall recommendation │└─────────────────────────────────────────────────────┘NLP-based answer evaluation
Each recorded response is transcribed and passed through a Hugging Face transformer pipeline for two tasks. Semantic similarity matching compares the candidate’s answer against the expected answer for that question, producing a relevance score rather than a keyword match. A summarization step condenses the response into a concise read for any reviewer who wants a quick scan without listening to the full recording.
The combination means the system produces both a machine score and a human-readable summary, useful for different parts of the hiring workflow.
Integrity monitoring without a proctor
The platform adds a computer vision layer that tracks eye movement patterns during the session. Consistent off-screen gaze, particularly in the direction of a second monitor or notes, is flagged as an anomaly. The flags are included in the assessment report alongside the answer scores, giving reviewers context rather than a binary pass or fail.
Backend and storage
A Flask and Spring backend handles session management, question sequencing, and the evaluation pipeline. Recorded call segments are stored to AWS S3, with each question’s audio tied to its metadata for downstream processing. The architecture was designed to support concurrent sessions, so the platform could run multiple candidates simultaneously without bottlenecks in the evaluation pipeline.