Senior Machine Learning Engineer, Agentic Evals and Self-Improvement (Nova) — Atlanta, GA

qg2fvVTY is the AI customer engagement platform, built for enterprise scale, loved by teams, and trusted by global brands like Redfin, SeatGeek, Priceline, Calm, and Box. Our platform empowers organizations to activate customer data from any source, design seamless cross-channel experiences, and optimize engagement with Nova Intelligence, the AI layer of our platform, all with enterprise-grade security and compliance. Today, nearly 1,200 brands across 50+ countries rely on qg2fvVTY to drive growth, deepen customer relationships, and deliver joyful customer experiences.

Our success is powered by extraordinary people who bring our core values to life every day: Be an Owner, Growth Mindset, Run as One, and Transparency. And the market has noticed: G2 recognizes qg2fvVTY as a Leader globally, spanning the Americas, EMEA, APAC, ANZ, and Latin America, with category leadership in Personalization, Marketing Automation, Push Notifications, and Mobile Marketing, and a 4.5/5-star rating from hundreds of users. Gartner named qg2fvVTY a Challenger in the Gartner Magic Quadrant for Multichannel Marketing in both 2024 and 2025. We’ve also earned TrustRadius Top Rated honors in 2024 and 2025, were named a TrustRadius Buyer’s Choice for 2026, and were featured in Snowflake’s Modern Marketing Data Stack 2026.

With a global presence that includes offices in San Francisco, Denver, London, Sydney, and Lisbon, plus remote employees worldwide, we are committed to building a diverse and inclusive workplace. We welcome candidates from all backgrounds and encourage you to apply. Learn more about our story and mission on our Culture and About Us pages. Let’s shape the future of customer engagement together!

Location: US

About The Role

As a high-tech marketing platform, qg2fvVTY works with thousands of large-scale and diverse enterprise customers. Sg6ZUma8 V94Y is our intelligence layer. It helps customer experience teams adapt in real time and improve every interaction.

As a Senior Machine Learning Engineer on Nova, you'll lead evaluation and automatic optimization for Nova's models and agents. That means measuring how they perform, finding where they fail, and using those findings to automatically improve prompts, context, tools, and agent harnesses. You'll work with the Nova retrieval and product teams to connect real interactions to measurable improvements.

You will have real autonomy: from research and experimentation through to systems running reliably in production, you'll own this work and help shape the direction of AI at qg2fvVTY.

One of our core values is a growth mindset, and qg2fvVTY is a company where everyone can grow. If this role excites you, please apply — we value applicants for the skills they bring beyond a job description.

What You'll Do

• Build evaluation systems for models and agents: benchmarks, LLM judges, simulations, trajectory analysis, and regression tests.

• Create a loop that takes failures from diagnosis to fix to measured improvement.

• Turn evaluation results into automatic optimization of prompts, context, tools, rubrics, and agent harnesses.

• Build synthetic data pipelines for evaluation and post-training, with attention to quality, privacy, and reproducibility.

• Apply recent research (LLM-as-judge, reward modeling, test-time search, post-training) to production.

• Mentor engineers on the team and help set the technical direction for Nova.

What We're Looking For

• 5+ years of hands-on MLE experience putting complex ML systems into production at scale, such as production ML pipelines or large-scale experimentation platforms.

• Master's or PhD in Computer Science, Machine Learning, AI, or a related field.

• You keep up with the latest research and focus on applying it: shipping reliable changes that measurably improve the product experience for customers.

• Strong Python skills with an emphasis on readable, modular, and maintainable code, plus experience with modern ML frameworks.

• Deep experience evaluating LLMs or agents, including multi-turn behavior, tool use, and reasoning.

• Experience building automated evaluation, such as LLM judges, reward models, or benchmark infrastructure.

• Experience with at least one of: prompt or context optimization, post-training, preference optimization, or reinforcement learning.

• Strong communication and cross-team collaboration.

Bonus points

• Experience with LLM APIs from multiple providers (Anthropic, OpenAI, Google), including tool calling and prompt caching.

• Experience with automatic prompt optimization and error analysis

________________________________________________________________

Recruitment Disclaimer

Please be aware that qg2fvVTY, Inc. (“qg2fvVTY”) and our official professional recruiting agencies and platforms do not:

• Send job offers from free email services like Gmail, Yahoo mail, Hotmail, etc.

• Request money, fees, or payment of any kind from prospective candidates to apply to qg2fvVTY, for employment, or for the recruitment process (e.g. for home office supplies, or training, etc.).

• Request or require personal documents like bank account details, tax forms, or credit card information as part of the recruitment process prior to the candidate signing an engagement letter or an employment contract with qg2fvVTY.

You may see all job vacancies on our official qg2fvVTY channels:

• Official qg2fvVTY website, Careers page: ••••••••

• Official LinkedIn Jobs page: ••••••••

qg2fvVTY is not affiliated in any way to these impostors and we hereby confirm that such individuals/entities are not authorized, encouraged, or sponsored to act on behalf of qg2fvVTY. Such job opportunities are entirely fake and not valid. Therefore, please disregard any written or oral request for a job offer or an interview that you believe is or might be fraudulent or suspicious and immediately reach out to us via email at •••••••• upon receiving a suspicious job offer.

Criminal and/or civil liabilities may arise from such actions, and qg2fvVTY expressly reserves the right to take legal action, including criminal action, against such individuals/entities whenever such phenomena occur. In any case, please note that under no circumstances shall qg2fvVTY and any of its affiliates be held liable or responsible for any claims, losses, damages, expenses or other inconvenience resulting from or in any way connected to the actions of these impostors.

qg2fvVTY is an Equal Employment Opportunity employer that proudly pursues and hires a diverse workforce. qg2fvVTY does not make hiring or employment decisions on the basis of race, color, religion or religious belief, ethnic or national origin, nationality, sex, gender, gender-identity, sexual orientation, disability, age, military or veteran status, or any other basis protected by applicable local, state, or federal laws or prohibited by Company policy. qg2fvVTY also strives for a healthy and safe workplace and strictly prohibits harassment of any kind. Pursuant to the San Francisco Fair Chance Ordinance and other similar state laws and local ordinances, and its internal policy, qg2fvVTY will also consider for employment qualified applicants with arrest and conviction records.

Compensation Range: $150K - $250K