I AM HIRED

Неделя 310 ₽Месяц 630 ₽3 месяца 1 680 ₽Год 5 680 ₽

Research Scientist, Open Benchmarks Red Team — United States

ML и AI · вчера, 15:58 · 16.11.2026

Зарплата не указана

Откликнуться

Компания: ••••••••

Локация: United States

Формат: не указан

Занятость: Полная занятость

Грейд: Senior

For our client, we are seeking a Research Scientist, Open Benchmarks Red Team to join the team of a leader in the Information Technology space.

This role focuses on strengthening the reliability and rigor of AI evaluation by finding weaknesses in frontier benchmarks and helping improve how they are assessed. You will work at the intersection of research and engineering, developing practical methods to identify issues, validate findings, and support better benchmark releases. The position calls for close collaboration with researchers and engineers while contributing to technical reports, tools, and publications that influence the broader AI evaluation ecosystem.

The work is highly technical and research-driven, with a strong emphasis on LLMs, AI agents, and experimental analysis of model behavior.

Location: Remote

Compensation: $198,000 – $340,000 annually

Responsibilities

• Identify and reproduce issues in frontier benchmarks, such as reward hacking and task ambiguity

• Propose and implement fixes for validated benchmark issues in collaboration with authors

• Develop reusable methods to detect recurring benchmark weaknesses

• Collaborate with benchmark authors to review findings and support improved releases

• Contribute to technical reports, open-source tools, and publications on benchmark evaluation

Qualifications

• Strong research or engineering background in machine learning, LLMs, AI agents, and evaluation

• Proficient in Python programming with experience in AI agents or evaluation systems

• Ability to identify weaknesses in evaluation methods and develop new approaches

• Experience in designing experiments and analyzing model behavior

• Familiarity with LLM/agent benchmarks and evaluation frameworks

• Proven track record of independently driving research and engineering projects

• Strong communication and collaboration skills

Benefits

• Opportunity to work on cutting-edge AI evaluation projects

• Collaborative environment with leading researchers and engineers

• Access to significant funding support for innovative AI research

• Support for professional development through publications and open-source contributions

• Engagement in impactful research that shapes the future of AI benchmarks

Our client is an equal opportunity employer. We encourage you to apply even if you don’t meet every qualification—your background could be exactly what this team needs.

Поиск по вакансиям

Название компании скрыто

Названия компаний и их продуктов мы не публикуем на сайте — они доступны в Telegram-боте I AM HIRED. Откройте эту вакансию в боте: увидите название, сможете откликнуться и настроить рассылку подходящих вакансий.

Открыть вакансию в боте