Coding Agent Evaluation Specialist — United States

Key Responsibilities：

Apply and adapt coding rubrics to evaluate LLM code responses and agent trajectories (correctness, code quality, efficiency, security, tool usage)

Run and test code, reproduce issues, and verify outcomes against test cases and acceptance criteria

Identify and attribute errors (reasoning, implementation, tool use, environment) and write structured feedback

Complete rubric interpretation and golden-task training; pass the post-training qualification before starting work

Must-Have Skills：

Relevant software development experience with evidence: professional role or project experience (internship or project acceptable; suggested 1+ year, 3+ years preferred)

LLM code evaluation / zJ training or code annotation / data labeling experience (code response scoring, ranking, rubric-based review) with proven quality delivery

Proficient in Python or another main language; able to read, run and debug code; solid Git and terminal skills; able to apply rubrics under established standards, attribute errors and give reliable feedback

Task execution capability; high ability of guideline understanding and attention to detail

Device: laptop/PC with stable internet connection

Nice-Have Skills：

Education: PhD, professional degree, or equivalent recognized senior expertise in CS / related field

Nice-to-have: Education: relevant Bachelor's / Master's / PhD (CS, software engineering or related field), or an equivalent professional / experience route

Nice-to-have: experience with coding agents / dH-rQZj5dfQ development tools; personal or open-source projects

Language Skills：

Fluent English (B2+ required, C1+ preferred); written English for feedback, training and collaboration