AI Test Engineer — United States

An employer is seeking an experienced AI Test Engineer to lead comprehensive quality assurance and evaluation strategies for advanced AI systems.

The Role

• Define end-to-end testing strategies for non-deterministic systems, including golden datasets, LLM evaluation frameworks, regression suites, and human review workflows

• Develop automated evaluation harnesses to measure accuracy, groundedness, latency, toxicity, bias, and multimodal quality across speech, image, and voice models

• Perform adversarial and red-team testing to identify vulnerabilities, prompt injections, data leakage, and edge cases across varied languages and accents

• Build and maintain automated functional, performance, and load testing pipelines integrated into CI/CD quality gates across web and mobile platforms

• Monitor defect trends, track release-readiness metrics, and communicate quality risks to cross-functional stakeholders

What You'll Need

• 12+ years of relevant experience in software testing and quality engineering

• Proven experience evaluating AI models and non-deterministic systems using modern automated evaluation frameworks

• Strong background in test automation across browsers, iOS, and Android platforms

• Familiarity with adversarial testing, red-teaming, and CI/CD quality integration

What's On Offer

• Full-time employment with a fully remote work schedule

• Opportunity to work at the forefront of AI evaluation and quality engineering

• Collaborative, engineering-driven environment

Apply via X8CvXSAE today!