Professional AI Model Critic (Opinions Required): Evals, Model Behavior & Interpretability | San Francisco Bay Area | Up to $300K Base, $500K OTE + Equity

"Everyone is entitled to his own opinion, but not to his own facts." Daniel Patrick Moynihan

Do you read a model's answer and immediately know what's wrong with it? Do friends avoid asking you which chatbot is best because the reply takes forty minutes and includes charts? Good news: someone will pay you for that.

Our client is one of the fastest-growing AI data companies anywhere, valued at over a billion dollars, and its data and evaluations are used by nearly every frontier lab. They want two people in the Bay Area whose full-time job is to hold well-evidenced opinions about how AI models should behave, and to share them with the researchers building those models.

WHAT A PROFESSIONAL MODEL CRITIC DOES

• Works out what "good" means for a personal assistant, an enterprise agent or a general chatbot, user by user.

• Runs the experiments and gathers the examples that prove it.

• Writes proposals and analyses that lab researchers read to the end.

• Presents the findings live, first to the commercial team and later to the labs themselves.

• Brings back what the labs are thinking, so the next dataset is the right one.

You're part of the commercial team, but nobody will ask you to "circle back" or "touch base". You're there as the researcher in the room.

THE SCORECARD (THIS IS HOW YOU'LL BE JUDGED)

Fair is fair. You judge models for a living, so here is how the hiring team will judge you.

The four that matter most

• You've been inside the frontier. A frontier or neo lab, or an AI evals or data company. OpenAI, Anthropic, Scale AI, Snorkel AI, METR and Redwood Research are the kind of names we mean.

• You did the model work yourself. Evals, model behavior, post-training, answer quality or interpretability, with your own hands on the outputs.

• You write clearly and present well. A sharp written proposal for a lab researcher, followed by a live presentation to a room. You need both.

• You'll be in the office five days a week. Monday to Friday in San Francisco's Financial District. Relocation support is available.

The baseline

• About a year or more of model behavior, interpretability or evals work. A shorter stint counts if it was model behavior at a frontier lab.

• Strong, specific opinions about what makes a model good or bad for a given user. "It depends" is where the conversation starts, never where it ends.

Nice extras

• Work on benchmarks, datasets or evaluation projects, or with human annotator and expert pools.

• A technical degree in computer science, machine learning, statistics, data analytics or economics, or the equivalent learned on the job.

Probably one to skip if

• Your experience is product-facing with no hands-on model work.

• Your AI experience comes only from a large cloud provider, with no frontier or neo lab time.

THE REWARD FOR ALL THAT CRITICISM

• Up to $300K base and up to $500K on-target earnings, plus equity.

• Your thinking reaches almost every frontier lab, instead of a single model.

• A founding seat: two hires, a new function, plenty of room to shape it.

• A front-row view of how different labs think about behavior, evals and data.

• Existing US work authorization is strongly preferred.

ru4 rKQe v5X3Wuv: YOUR TRUE PARTNERS IN AI AND TECH ROLES

hdS yVyW xc2dMRe is a specialist AI and technology recruitment firm. Our desk is small on purpose, so you speak with someone who knows the client, has studied the scorecard and will give you a clear read on where you stand. We prepare you properly and stay with you through to a signed offer.

YOUR MOVE

Apply here on LinkedIn. Tell us where you've worked, what you did yourself, and your strongest opinion about model behavior. Bring the facts to go with it.