Senior AI Data Annotation Jobs in Miami — Miami, FL

About The Role

Senior AI data annotation work focused on expert data labeling and large language model evaluation to improve training data quality, annotation guidelines compliance, and downstream model performance. This is a US-based Remote full-time role aligned with Miami talent demand.

What You Will Do

• Perform data labeling for NLP tasks (e.g., named entity recognition, classification, summarization quality checks, instruction-following evaluation)

• Conduct RLHF evaluations including pairwise ranking, preference labeling, and writing clear feedback rationales

• Execute prompt evaluation and QA evaluation for helpfulness, correctness, and policy adherence

• Run content safety labeling for sensitive content, toxicity, and policy violations

• Audit training data quality to reduce label noise; flag duplicates, leakage risks, and low-quality sources

• Document edge cases, propose rubric improvements, and support dataset curation for LLM training pipelines

• Collaborate asynchronously with reviewers, QA, and project leads; participate in calibration sessions

Requirements

• Mid-senior experience in data annotation or AI evaluation with strong attention to detail

• Ability to apply strict rubrics consistently and communicate clearly in writing

• Familiarity with RLHF concepts, prompt evaluation, QA evaluation, and training data quality practices

Preferred Qualifications

• Experience with LLM evaluation rubrics, policy-based content safety labeling, and dataset curation for instruction tuning

• Exposure to inter-annotator agreement, calibration sessions, and ambiguity resolution

• Computer vision annotation experience (bounding boxes, segmentation, attribute tagging) is a plus