RL Data QA Engineer
- Джерело:
- djinni.co
Що робити
- Review RL tasks end to end: Examine instructions, environments, reference solutions and tests, and decide whether each task meets our quality bar.
- Break the verifier: Look for the ways an agent could pass without really solving the task, such as weak tests, leaked solutions or loopholes in the environment.
- Read agent trajectories: Analyze how frontier coding agents attempted each task, and tell a genuine solve from a lucky or illegitimate one.
- Judge difficulty with data: Interpret pass rates across models and trials, and decide when a result is solid and when it needs more evidence.
- Separate task faults from platform faults: Make sure contributors are judged on their work, never on infrastructure issues.
Що пропонуємо
- Experience with RL environments, LLM evaluations, benchmarks or reward modeling.
- Deep technical knowledge in at least one domain: For example QA, software engineering, networking, security, data or ML engineering.
- Experience authoring tasks or benchmarks for AI agents.
- Fluency with AI coding agents (Claude Code, Codex, Cursor or similar) as review tools.
- Experience writing review guidelines or calibrating a team of reviewers.
Схожі вакансії
З блогу Trackr
Усі статті →
Знайдено через trackr.help/jobs · Канал: @trackrhelp · Бот для персональних сповіщень: @trackrhelpBot