Content Analyst — California, United States

Our client, a leader in FinTech, is looking to hire an AI Content Analyst. This is a fully remote role that will align to PST time zones!

*This role is a 1-year W2 contract opportunity with benefits & paid holidays!*

As an AI Content Analyst, you will join the AI Evaluations team, analyzing AI-generated answers, building golden datasets, and providing insights on chatbot quality. Your work will improve chatbot accuracy, FAQ accessibility, and technical support operations for a well-known accounting platform!

• Build AI conversation inputs, outputs, driving quality of golden datasets ready for AI evaluation

• Perform offline and online AI answer evaluations at scale with LLM-as-a-judge models and human-in-the-loop

• Analyze datasets to identify improvement opportunities for conversational products, including chatbots and voice assistants

• Support development of conversational AI principles, strategies, playbooks in collaboration with cross-functional Product, Engineering & Data Science teams

Required Skills & Experience

• Bachelor’s degree

• 2+ years of experience in AI Content Analysis, AI Conversation Design, Prompt Engineering, or related field

• Experience with user research, usability testing, and data analysis of agentic AI, chatbots, or LLMs

• Familiarity with AI/ML and/or LLM frameworks such as NLUs, RAG databases, TensorFlow, and/or PyTorch

• Prior experience in SaaS, FinTech, Financial Services, Banking, or related industries highly valued!

pJ5C6v QV98375qVvD xf6HB9sC is an Equal Opportunity Employer, including disability/vets. All applicants must be currently authorized to work on a full-time basis in the country for which they are applying, and no sponsorship is currently available. Employment is subject to the successful completion of a pre-employment screening. Accommodation will be provided in all parts of the hiring process as required under FeKpE5 vC5pDN98cym x9fC4sWnsH Accommodation policy. Applicants need to make their needs known in advance.