I AM HIRED

Неделя 310 ₽Месяц 630 ₽3 месяца 1 680 ₽Год 5 680 ₽

Technical Lead Manager, Applied Science & AI Evals - Games GenAI — Los Gatos, CA

ML и AI, AI-инженерия · вчера, 21:45 · 18.11.2026

Зарплата не указана

Откликнуться

Компания: ••••••••

Локация: Los Gatos, CA

Формат: не указан

Занятость: Полная занятость

Грейд: Lead / руководитель

At EVc9SRN, our mission is to entertain the world. Together, we are writing the next episode - pushing the boundaries of storytelling, global fandom and making the unimaginable a reality. We are a dream team obsessed with the uncomfortable excitement of discovering what happens when you merge creativity, intuition and cutting-edge technology. Come be a part of what’s next.

We're looking for a Tech Lead to lead Applied Science execution and AI evaluation strategy across uHZffGR bc5qF' agentic products. This role drives - machine learning model development, fine-tuning, optimization, and the offline/online evaluation work that delivers safe, responsible agentic tools and experiences. You will set the ML quality bar for the games GenAI portfolio, drive model work from data curation through deployment and lead a team that executes.

Key Responsibilities

• Lead ML Applied Science research & execution for uHZffGR bc5qF' GenAI portfolio: model selection, data curation, fine-tuning (SFT, DPO, LoRA/QLoRA), and optimization (distillation, quantization) from prototype through production deployment.

• Define uHZffGR bc5qF' AI evaluation strategy end to end: benchmark what "good" means for a given model or agentic product, how it's measured offline before ship and online after ship, and how that bar gets enforced consistently across projects.

• Define offline evaluation processes: curated golden datasets, adversarial and edge-case test suites, automated scoring pipelines, LLM-as-a-judge frameworks, human-in-the-loop grading rubrics, and regression gates that block quality or safety regressions before they reach players or internal users.

• Support and collaborate with Data Science team in online evaluation metrics for Agentic systems: production telemetry, A/B and holdout experimentation, user feedback loops, and anomaly and drift detection that catches degradation in live model and agent behavior.

• Set and maintain ML execution standards across the games GenAI portfolio: model and vendor benchmarking methodology, experimentation rigor, statistical soundness, and documentation practices that other teams can adopt.

• Apply model development and evaluation discipline across both domains of agentic products: player-facing AI features and internal agentic tools.

• Design safety and red-teaming evaluation protocols to detect toxicity, hallucination, jailbreaks, and out-of-character or off-policy behavior in both player-facing and internal agentic systems.

• Communicate model performance, evaluation findings, quality trends, and risk to both technical partners and non-technical stakeholders, including leadership, in a way that drives decisions.

Qualifications

• PhD in Machine Learning, Deep Learning, Computer Science, or a related quantitative field.

• 5+ years of experience in machine learning, applied science, or software engineering, with hands-on depth across both model development (training, fine-tuning, optimization) and evaluation/measurement of ML and GenAI systems.

• Broad, current technical understanding across model families and modalities - LLMs, vision-language and multimodal models, computer vision foundation models, and reinforcement learning - and the judgment to know which architecture fits a given agentic product problem.

• Direct, hands-on experience evaluating and benchmarking foundation models against each other on inference latency, zero-shot generalization, accuracy, and deployment cost, to inform build-vs-buy and model-selection decisions.

• Direct, proven experience with the full LLM fine-tuning lifecycle - SFT, DPO, LoRA/QLoRA, and RLHF for long-horizon tasks - from dataset curation through deployment and ongoing maintenance.

• Direct, proven experience designing and operating evaluation processes for LLM or agentic systems: offline benchmark suites (including tiered/taxonomy-based evaluation by task complexity), automated scoring pipelines, LLM-as-a-judge frameworks, claim-level hallucination and groundedness verification, and online experimentation (A/B testing, production telemetry, drift detection).

• Experience with on-device or production inference tradeoffs - latency, throughput, memory/battery footprint, and cost - and how those constraints shape model selection and deployment decisions.

• Prior experience managing or leading a team of ML engineers or applied scientists - hiring, mentoring, setting technical direction, and being accountable for a team's output.

• Strong foundation in experimental design, statistics, and causal inference; comfortable defending a measurement methodology under scrutiny.

• Working understanding of agentic system architectures (multi-step reasoning, tool use, multi-agent orchestration) sufficient to develop and evaluate models for them.

• Excellent written and verbal communication; able to translate model and evaluation results into a clear point of view for both engineering partners and leadership.

• Comfortable operating at the intersection of two different problem spaces - consumer-facing game experiences and internal developer/creative tooling - and adapting model and evaluation approach to each.

Nice to have

• Experience developing or evaluating GenAI systems in a games or interactive entertainment context.

• Experience combining reinforcement learning with LLM-based planning for autonomous or agentic tasks.

• Familiarity with Model Context Protocol (MCP) or similar agent-to-system connectivity standards, from a model and evaluation standpoint.

• Background in responsible AI: safety evaluation, prompt injection defense, content policy compliance.

• Published research, patents, or public contributions in LLM evaluation, model fine-tuning, multimodal AI, agentic evaluation, or RLHF.

Generally, our compensation structure consists solely of an annual salary; we do not have bonuses. You choose each year how much of your compensation you want in salary versus stock options. To determine your personal top of market compensation, we rely on market indicators and consider your specific job family, background, skills, and experience to determine your compensation in the market range. The range for this role is $523,000.00 - $1,050,000.00.

EVc9SRN provides comprehensive benefits including Health Plans, Mental Health support, a 401(k) Retirement Plan with employer match, Stock Option Program, Disability Programs, Health Savings and Flexible Spending Accounts, Family-forming benefits, and Life and Serious Injury Benefits. We also offer paid leave of absence programs. Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off. Full-time salaried employees are immediately entitled to flexible time off. See more details about our Benefits here.

EVc9SRN is a unique culture and environment. Learn more here.

Inclusion is a EVc9SRN value and we strive to host a meaningful interview experience for all candidates. If you want an accommodation/adjustment for a disability or any other reason during the hiring process, please send a request to your recruiting partner.

We are an equal-opportunity employer and celebrate diversity, recognizing that diversity builds stronger teams. We approach diversity and inclusion seriously and thoughtfully. We do not discriminate on the basis of race, religion, color, ancestry, national origin, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.

Поиск по вакансиям

Название компании скрыто

Названия компаний и их продуктов мы не публикуем на сайте — они доступны в Telegram-боте I AM HIRED. Откройте эту вакансию в боте: увидите название, сможете откликнуться и настроить рассылку подходящих вакансий.

Открыть вакансию в боте