We are seeking a Senior AI QA Engineer to validate the performance of AI-driven video analysis systems, focused on detecting key moments in sports content. The role combines manual and automated testing, working with pre-labeled video assets provided by the customer.
Responsibilities
-
Audit live sports games (NBA, MLB, NFL, NHL) to ensure the AI/Inference Service correctly tracks and labels major sports moments such as touchdowns, home runs, and buzzer-beaters as they happen
-
Work with timecodes, video frames, transcriptions, and captions, applying sports-related contexts, lingo, and metrics
-
Act as the human expert who catches when the AI hallucinates, misinterprets a sports rule, or produces errors
-
Collaborate directly with AWS engineers to detail bugs and validate resolutions
-
Review AI-generated labels and metadata tags to ensure they make sense for the sport and meet advertising industry standards (IAB rules), keeping content brand-safe for ad placement
-
Compare the inference system's outputs against customer-provided labels to determine accuracy and identify missed or incorrectly detected events
-
Execute both manual test cases (for nuanced or edge cases) and automated test cases (for validating outputs at scale and ensuring consistency)
-
Log discrepancies, defects, and gaps between expected and actual results, and collaborate with the inference/development team to triage and resolve issues
-
Maintain clear QC documentation, track accuracy metrics, and help build a smooth testing process that bridges traditional sports broadcasting with new AI technology
-
Serve as a communication bridge between the customer and technical teams, clarifying results and expectations
-
Repeat validation iteratively as new labeled assets or model updates are provided to ensure ongoing accuracy and reliability
Requirements
-
3+ years of experience in both manual and automation QA, preferably in video-focused or AI-driven environments
-
Knowledge of testing LLMs and understanding of their workflow
-
Proficiency in automation scripting with Python, Selenium, or similar tools for comparing JSON outputs to ground truth at scale
-
Understanding of software development and QA cycles, including defect logging, triage, and reporting
-
Ability to interpret labeled data and validate model outputs
-
Familiarity with concepts like computer vision and transcript analysis (no need to understand internal model workings)
-
Strong communication skills for stakeholder interaction and reporting
-
Detail-oriented and iterative approach to testing
-
Proficiency in English at an Upper-Intermediate level (B2) or higher
Nice to have
-
Experience with video testing tools or frameworks
-
Prior work in sports analytics or media technology
We offer
-
International projects with top brands
-
Work with global teams of highly skilled, diverse peers
-
Healthcare benefits
-
Employee financial programs
-
Paid time off and sick leave
-
Upskilling, reskilling and certification courses
-
Unlimited access to the LinkedIn Learning library and 22,000+ courses
-
Global career opportunities
-
Volunteer and community involvement opportunities
-
EPAM Employee Groups
-
Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn
EPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.
