Conversation Designer / Contract / Remote / PST — Mountain View, CA

Our client, a leader in FinTech, is looking to hire a Conversation Designer. This is a fully remote role that will align to PST time zones!

• This role is a 1-year W2 contract opportunity with benefits & paid holidays!*

As a Conversation Designer, you will join the AI Evaluations team, analyzing AI-generated answers, building golden datasets, and providing insights on chatbot quality. Your work will improve chatbot accuracy, FAQ accessibility, and technical support operations for a well-known accounting platform!

• Build AI conversation inputs, outputs, driving quality of golden datasets ready for AI evaluation

• Perform offline and online AI answer evaluations at scale with LLM-as-a-judge models and human-in-the-loop

• Analyze datasets to identify improvement opportunities for conversational products, including chatbots and voice assistants

• Support development of conversational AI principles, strategies, playbooks in collaboration with cross-functional Product, Engineering & Data Science teams

Required Skills & Experience

• Bachelor’s degree

• 2+ years of experience in AI Content Analysis, AI Conversation Design, Prompt Engineering, or related field

• Experience with user research, usability testing, and data analysis of agentic AI, chatbots, or LLMs

• Familiarity with AI/ML and/or LLM frameworks such as NLUs, RAG databases, TensorFlow, and/or PyTorch

• Prior experience in SaaS, FinTech, Financial Services, Banking, or related industries highly valued!

Posted By: VMS Sourcing