Rate model responses
Score answers for accuracy, helpfulness, tone and safety using the project's rubric, and explain your ratings.
Remote · Project-based
AI evaluator and AI rater jobs
AI evaluators read what a model says and judge it: is it correct, helpful, safe and well written? They test models with tricky prompts and rate responses against a rubric so developers know where the model falls short. One6 contributors do this work remotely on AI evaluation projects.
What the work involves
Compare responses
Read two or more answers to the same prompt and decide which is better and why.
Prompt testing
Write realistic and adversarial prompts to find where a model fails, and document what happened.
Content and data review
Check datasets and generated content for accuracy and policy compliance.
You may be a good fit if
- You can explain clearly why one answer is better than another
- You fact-check instead of trusting confident-sounding text
- You apply a rubric consistently, even when it's not how you'd judge on your own
- You write clear, concise English
Skills and setup
- Strong reading comprehension and writing
- Research and fact-checking
- Familiarity with AI chat assistants
- Subject expertise for specialist evaluation (see AI training jobs for experts)
How to get started
- 01
Apply
Share your location, expertise, skills, experience, languages and work preferences.
- 02
Qualify
Complete a skills test or trial task. When a project needs your profile, we contact you with the details.
- 03
Train
Review the project's instructions and terms, including how work is reviewed and paid, and train on its guidelines.
- 04
Work
Work remotely. Your accuracy and output are tracked, and contributors below the quality bar are retrained or removed.
Frequently asked questions
What does an AI rater actually do all day?
Most of the work is reading a prompt and one or more AI responses, scoring them against written criteria and writing a short justification. Some projects focus on writing prompts that test the model instead.
How do projects keep ratings consistent?
Reviewers are calibrated on shared examples against the project's rubric before production work, and agreement between reviewers is monitored throughout.
Is the work remote?
Yes. One6 contributors work remotely, from wherever they are.
How much does it pay?
Compensation depends on the project. Before you start any project you receive its instructions and terms, including the tasks, how work is reviewed and how it is paid, and you are free to decline.
Does applying guarantee work?
No. Contributors are selected for each project based on its requirements, such as languages, domain and experience. We contact you when a project matches your profile.
Read the Contributor Terms and Privacy Policy, learn more about One6, or see all remote AI training jobs.
Related work and guides
Ready to apply?
Share your information once and we'll contact you when a project matches your profile.