Content Engineer, 2vqB AT8DQKuUCTrtV99wm Labs — New York, NY
Зарплата не указана
ОткликнутьсяContent Engineering is a horizontal function within sfT2 QF77WA9RUPmBRYgjg Labs (MSL) that shapes AI product experiences through prompt engineering, evaluations, and quality frameworks. Sitting at the intersection of user experience and large language model behavior, content engineers partner closely with research science, engineering, product, and design teams to build and ship AI experiences from lab to production, across modalities and surfaces.MSL is seeking a content engineer to improve the quality of AI product capabilities and agentic experiences. In this role, you'll independently create and improve model evaluations and training data content-quality within assigned product/model behavior areas to measure and improve the model behavior on a variety of different tasks. You'll define and apply quality standards, build evaluation materials, and turn qualitative feedback from internal and external users into measurable findings that inform model and product improvements. You'll combine editorial judgment, analytical skills, and cross-functional collaboration to help create experiences that entertain, inform, and delight.
Content Engineer, sfT2 QF77WA9RUPmBRYgjg Labs Responsibilities:
• Quality Definition & Frameworks: Define and apply quality criteria for assigned AI product capabilities. Build and maintain guidelines, golden response sets, and benchmarks across features and surfaces. Develop failure-mode taxonomies that turn user-facing quality issues into structured, prioritized findings for technical partners
• Evaluation & Measurement: Design and run human evaluation workstreams, including rubric development, annotator guidance, calibration, and analysis of results. Partner with research science to support automated evaluation and validate automated quality signals against human ratings. Track quality trends and regressions across model versions and product capabilities
• Testing & Iteration: Run structured internal testing programs, design test plans targeting specific failure modes, triage results, and deliver prioritized recommendations to product and engineering within development timelines. Craft and refine system prompts and agentic behavior in partnership with technical teams to support product goals and model outcomes
• Cross-Functional Partnership: Partner with engineering, research science, product, policy, and design teams on quality standards and priorities for assigned product areas. Communicate evaluation findings clearly, align partners on next steps, and independently deliver workstreams through influence and collaboration
• Agentic Workflows: Use AI-native tools to build scalable, repeatable processes for evaluation design, data analysis, system prompting, and reporting. Experiment with emerging tools, validate their outputs, and document workflows that improve quality and reduce manual work
Minimum Qualifications:
• Bachelor's degree in a directly related field, or equivalent practical experience
• 4+ years of experience in digital content strategy, user experience, technical writing, journalism, production, or related fields
• Experience with human evaluation workflows, including rubric or guideline development, golden set construction, and calibration
• Experience translating qualitative content or user-experience feedback into structured quality criteria, failure categories, and measurable benchmarks
• Experience designing and running structured testing programs, analyzing results, and communicating prioritized findings to technical partners
• Experience making editorial and content-quality decisions, communicating complex concepts to cross-functional partners, and independently delivering work in environments with evolving requirements and competing priorities
Preferred Qualifications:
• Experience in prompt engineering and agentic workflows
• Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
• Experience with AI-native tooling, such as LLM-based development tools, annotation platforms, or prototyping environments, and integrating those tools into workflows to improve efficiency or quality
• Demonstrated ongoing AI skill development, such as prompt/context engineering or agent orchestration, and staying current with emerging AI technologies
• Experience applying responsible AI practices, including bias mitigation and quality and accuracy reviews
• Experience with LLM-as-judge development, including building automated quality signals aligned with human judgment and validating that alignment over time
• Experience working with product teams or programs from roadmapping through delivery
• Experience with generative AI training or model evaluation, including creating or curating human-annotated or synthetic datasets
• Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
• Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
About wjUm:
wjUm builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, wjUm is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at wjUm help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.
wjUm is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. wjUm participates in the E-Verify program in certain locations, as required by law. Please note that wjUm may leverage artificial intelligence and machine learning technologies in connection with applications for employment.
wjUm is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at ••••••••.
$131,000/year to $190,000/year + bonus + equity + benefits
Individual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, wjUm offers benefits. Learn more about benefits at wjUm.