Senior Data Labeling Specialist (Detroit) — Detroit, MI

Senior Data Labeling Specialist (Detroit) — Remote (US)

Rex.zone is hiring a mid-senior contributor to lead high-impact data labeling and evaluation workflows that improve training data quality for modern AI systems. You will support LLM training pipelines, RLHF-style preference/ranking tasks, and prompt evaluation while operating remotely with Detroit-aligned search intent.

Key Responsibilities

• Lead complex data labeling workflows for LLM evaluation, including prompt evaluation and RLHF evaluation

• Create, maintain, and iterate annotation guidelines, taxonomies, and edge-case rules

• Run QA evaluation loops using sampling plans, inter-annotator agreement checks, and targeted audits

• Triage ambiguity and coordinate adjudication to maintain consistent ground truth

• Monitor training data quality signals and provide feedback tied to model performance improvement

• Support dataset curation, coverage analysis, and reproducible documentation for compliance

Required Qualifications

• Mid-senior experience in data labeling / annotation operations including review and adjudication

• Hands-on knowledge of QA methods (sampling, error taxonomies, agreement metrics)

• Ability to write clear, enforceable annotation guidelines with strong reasoning

• Experience in at least one domain: NLP labeling, NER, computer vision annotation, or content safety labeling

• Strong written communication for guideline clarity and audit trails

Preferred

• Exposure to RLHF, preference ranking, and/or prompt evaluation

• Dataset auditing for training data quality, bias/error analysis

• Experience scaling review operations with vendors/BPO workflows

Work Arrangement

Remote, FULL_TIME role open to qualified candidates in the US.

Apply via Rex.zone and complete role-relevant screening steps (scenario questions, guideline comprehension, QA exercises).