Founding Engineer — San Francisco Bay Area

Founding Engineer, AI Agent Evaluation | San Francisco (Onsite) | Up to $300k base + significant equity

An early-stage, venture-backed AI startup in San Francisco is hiring its first engineer. The company helps enterprises understand whether their AI agents are actually doing the job: how well they perform, where they break and how to improve them.

This is a chance to shape a new category from the ground up, working side by side with an experienced founder from the AI evaluation world.

What you’ll work on:

• Designing ways to score agent output on complex business tasks where there’s no single right answer

• Building realistic test scenarios and simulated workflows that reflect how agents are used in practice

• Deciding how much testing is enough, and making sure results are statistically sound and trusted by customers

• Taking approaches that work in individual customer deployments and building them into the core product

You’ll likely have:

• Built evaluation systems, benchmarks, graders, reward models, RL environments or synthetic data for LLMs or agents

• Strong engineering skills, paired with a scientific approach to measurement

• Early-stage experience as a founder, founding engineer or early hire

• Around 2 to 6 years of experience

Bonus points for: post-training experience (RLHF, SFT or similar), domain experience in areas like finance, legal or healthcare, or a background in statistics or published research on evaluation.

The package:

• Up to $300k base salary

• Significant founding equity

• Comprehensive health benefits

• Relocation support to San Francisco