Senior Data Labeling Specialist (Nashville) — Nashville, TN

Senior Data Labeling Jobs in Nashville (Remote)

Rex.zone is hiring a Senior Data Labeling Specialist to own end-to-end data labeling and evaluation workflows for AI/ML training datasets across NLP, computer vision, and content safety. You will lead advanced annotation and reviewer-level QA, and run RLHF-style evaluations (preference ranking, rubric scoring, prompt evaluation) that connect label quality to model performance improvement.

What You Will Do

• Label and review complex text, image, and multimodal datasets while maintaining high inter-annotator agreement.

• Run QA evaluation programs: sampling plans, error taxonomy, root-cause analysis, and corrective actions tied to guideline compliance.

• Perform RLHF evaluation workflows such as pairwise preference ranking, rubric-based scoring, and prompt evaluation for instruction-following and safety behavior.

• Execute NLP labeling tasks including named entity recognition, classification, summarization evaluation, and groundedness/faithfulness checks.

• Execute computer vision annotation such as bounding boxes, polygons, keypoints, segmentation, and attribute tagging; validate edge cases.

• Support content safety labeling (e.g., harassment, self-harm, hate) with consistent escalation paths and audit trails.

• Partner with engineering/ML teams to translate model failure modes into updated guidelines, new label schemas, and targeted re-annotation.

• Document decisions, dataset specs, and QA findings to support reproducibility and dataset lineage.

Required Qualifications

• 3+ years in data labeling/annotation or AI/ML evaluation, including QA evaluation and reviewer workflows.

• Hands-on experience with LLM evaluation, RLHF evaluation, or structured prompt evaluation using rubrics and preference judgments.

• Strong guideline compliance, error analysis, and understanding of how training data quality impacts model performance.

• Experience with NLP tasks (e.g., named entity recognition, text classification) and common dataset formats/tooling.

• Familiarity with computer vision annotation concepts and ambiguity resolution patterns.

• Ability to write clear guidelines, make consistent decisions, and communicate edge cases cross-functionally.

Remote & Location Notes

• Workplace: Remote, open to US-based candidates including Nashville, TN.

• May align to team collaboration hours for calibration and QA sign-off meetings.

Compensation

Competitive hourly base pay: $30–$50 per hour (USD).

Apply via Rex.zone to enter the matching workflow.