Machine Learning Data Evaluator (Remote) — Crewlia, United States

• Role: Machine Learning Data Evaluator (Remote)

• Location: Remote (Australia, Canada, Ireland, New Zealand, United Kingdom, United States)

• Pay: US$18 per hour

Applicants must currently reside in Australia, Canada, Ireland, New Zealand, the United Kingdom or the United States.

Overview

We are recruiting analytical contributors to assess examples used to train or measure machine learning systems. You will evaluate labels, inputs, expected outputs, and example difficulty so the data represents the intended task accurately.

Key Responsibilities

- Confirm that each example's input, expected output, and label express the same task definition.

- Identify ambiguous examples that permit multiple reasonable answers under the supplied criteria.

- Check for shortcuts, leaked answers, irrelevant cues, and duplicates that could distort learning or measurement.

- Evaluate whether example difficulty and topic coverage match the stated dataset purpose.

- Recommend accept, revise, exclude, or escalate outcomes with a task-grounded rationale.

Requirements

- Must be at least 18 years old and able to communicate in English.

- Ability to reason about inputs, labels, and expected outputs as connected parts of a learning example.

- Ability to spot ambiguity, answer leakage, irrelevant cues, duplication, and coverage gaps.

- Reliable high-speed internet and a suitable home-working setup.

- Previous experience with machine learning data, annotation, or model evaluation is helpful but not required.

Equal Opportunity

We welcome applications from qualified individuals of all backgrounds. Applicants are considered based on relevant skills, qualifications and project eligibility requirements, without discrimination on grounds protected by applicable law.

Apply Now!