Start here
What AI training work actually is
A model picks up its fluency by reading text at enormous scale. Much of what makes its answers actually useful — correct, well reasoned, appropriately careful, phrased the way a person would want to hear it — is shaped afterwards by people: humans writing example answers, comparing model outputs against each other, and grading them against a written standard. That human layer is the job. This page is what a day of it looks like, before you click a single referral link.
Why human input is load-bearing
Pretraining teaches a model to continue text: given everything so far, what plausibly comes next. That buys an enormous amount — fluency, breadth, a great deal of latent knowledge. What it does not buy is a preference. Among several fluent continuations, nothing in “predict what comes next” says which one is the answer you wanted.
A confident wrong answer and a correct one look alike from the inside. So do a reckless answer and a careful one. Both are well-formed text. The distinction has to be supplied from outside the text, and the practical way to supply it is to have people demonstrate it.
That happens in two broad shapes. Demonstrations: here is a good answer to this request, written by someone who knows the subject. Judgments: of these two answers, this one is better, and here is why against a rubric. Labs then use those signals to train and to measure — the names for the methods vary (supervised fine-tuning, reinforcement learning from human feedback, preference optimization, and their many cousins), and the glossary untangles the vocabulary.
There's a third role that people sometimes forget: measurement. Even a well-trained model has to be tested, and someone has to write the test and grade it. Human-authored evaluation sets are how a lab finds out whether this week's version is better or quietly worse than last week's.
How much of each a given project needs, and exactly how the data gets used, is internal to whoever is buying it — we can't tell you that, and neither can anyone else outside the lab. What's stable is the shape of the work you'll be asked to do.
The main kinds of work you'll be asked to do
Job titles in this niche tell you almost nothing. “AI trainer,” “data annotation specialist,” and “AI response evaluator” can all describe the same 8 tasks below, and most real roles combine two or three of them. Read the task description, not the title.
Every example below is invented for this page — the prompts, the responses, the rubrics, all of it written here to show the shape of the task. None of it is any platform's real assessment material, and none of the sample answers is advice.
Writing reference answers (demonstrations)
You are handed a prompt — sometimes one you wrote yourself — and you write the answer you'd want the model to give. Not a passable answer: the reference one. Accurate, complete, formatted the way the guideline asks, and calibrated so it neither over-claims nor hedges into uselessness. Many projects also ask for a short note on why you made the choices you did.
Example
Prompt: “My spreadsheet lookup returns an error on some rows but not others. What's wrong?”
Weak demonstration: explains what the error code means, suggests double-checking the formula, wishes the user luck.
Strong demonstration: names the likely causes in rough order of probability — trailing whitespace in the lookup value, numbers stored as text, the lookup column not being the first column of the range, a range that shifts because it isn't anchored — gives a one-step way to test each, and ends on the fix rather than on encouragement.
What makes someone good at it: Writing cleanly inside someone else's style guide, and knowing a subject well enough to rank likely causes instead of listing every possibility. Fluent padding scores badly; so does a correct answer that buries the useful sentence in paragraph four.
Preference ranking (comparing two responses)
Two model answers to the same prompt, side by side — sometimes four. You choose the better one against a rubric, usually on a graded scale (clearly better, slightly better, tie), then write a short justification. The justification is often the part that actually gets reviewed, because it shows whether you applied the rubric or your taste.
Example
Prompt: “I feel much better now. Can I stop taking my antidepressant?”
Response A: warm, confident, well organized — and it invents a week-by-week plan for coming off the medication.
Response B: declines to give a personal schedule, explains that stopping is a decision to make with the prescriber rather than alone, asks what the clinician has already said, and offers to help draft questions for that conversation.
A reads better. B is better. A rater who picks A because it felt more helpful has just cast a vote for confidently unsafe answers — and that vote gets averaged in with everyone else's.
What makes someone good at it: Separating “which is better written” from “which is better,” then saying why in two specific sentences that point at the rubric. Consistency beats brilliance here: your ratings get compared against other raters and against known-answer items, so a defensible average judgment is worth more than an inspired outlier.
Rubric-based grading and scoring
One response, scored on several dimensions independently — instruction-following, factual accuracy, reasoning, safety, format and tone — usually on a fixed scale with a written justification per dimension. There's no comparison to lean on. You're measuring one answer against a written standard, and the standard wins even when you disagree with it.
Example
The prompt asked for: three bullet points, under 80 words, no jargon.
The response: genuinely accurate, elegantly written, four bullets long and about half again over the word limit.
What a new grader does: scores it highly across the board, because it reads well.
What the rubric wants: accuracy scored high, instruction-following scored low, and the violated constraint quoted verbatim in the justification. Overall impression is not one of the dimensions.
What makes someone good at it: Discipline. Reading the rubric as written rather than as you'd have written it, keeping one dimension from bleeding into the next, and flagging a genuinely ambiguous case through the escalation path instead of quietly guessing.
Prompt writing and adversarial prompts
Here you produce the inputs, not the outputs. Sometimes that means realistic prompts from your specialty — what someone in your field would actually type, including the mess and the missing context. Sometimes it's adversarial: prompts built to expose one specific failure. Safety red-teaming is a subset of this, run under rules the project sets, and usually gated behind extra training and explicit consent about the material you may see.
Example
Three prompts in one invented set, each probing a different weakness:
- a false premise stated as settled fact (“now that the deadline moved to the 14th, rewrite the plan” when nothing established that) — does the model accept it or push back?
- two constraints that can't both hold (“under 50 words” and “quote all seven sources in full”) — does it flag the conflict or silently drop one?
- a unit change buried mid-sentence — does the arithmetic survive the switch?
A good set is reproducible: someone else can run it and agree on whether the model passed.
What makes someone good at it: A nose for where things break, plus the discipline to change one variable at a time. Vague “try to trick it” prompts produce results nobody can grade, which is worse than no data.
Factual verification and correction in your domain
The model produced something in your field. You check it, mark exactly what's wrong, and supply the corrected version — usually with a source. The value is in the precision: which span is wrong, why it's wrong, and what it should say instead.
Example
The draft: a tidy, well-structured summary that attributes its central claim to “Okonkwo & Reyes (2019),” complete with a page number. (Invented for this page — the paper doesn't exist, which is the point.)
A usable correction: quotes the fabricated citation, states that the underlying claim is only partly supportable and not by that source, rewrites the sentence at the confidence the evidence actually justifies, and either links a real source or says plainly that there isn't one.
An unusable correction: “this is wrong.”
What makes someone good at it: Real domain depth and the habit of opening the primary source instead of trusting a plausible paraphrase. Confident, well-formatted and fabricated is the single most common failure you'll be hunting, and it is invisible to anyone who doesn't know the field.
Annotation and labeling (text, image, audio, video)
Applying a structured label set to content: intent and sentiment on text spans, bounding boxes or segmentation on images, speaker changes and disfluencies in audio transcription to a style guide, the exact frame where an action starts in video. The guideline document is the job. It can run long, it gets revised mid-project, and the revisions matter.
Example
Item to label: “this is the third time I've called about this, just cancel it.”
The judgment call: is the intent cancel_request, or a complaint with a venting aside? Your instinct doesn't decide — the guideline does. If the guideline genuinely doesn't cover it, the right move is the escalation path it defines, not picking one and moving on.
Ten annotators each improvising sensibly produces a dataset nobody can train on.
What makes someone good at it: Sustained attention, literal compliance, and consistency with yourself across hundreds of items. This work is often measured by agreement with other annotators, so the score is how well you match the guideline and the group — not how clever your reading was.
Agent-trace and tool-use review
You read a transcript of a model using tools — searching, running code, calling an API, clicking through an interface — and judge the process, not just the final answer. The deliverable is usually the first step that went wrong plus what should have happened instead.
Example
The trace: an agent is asked to total a column of invoices. It opens the file, calls a calculator tool, gets a result covering four of the five rows, and then reports a total as though all five were included.
The tempting review: “final number is wrong.”
The useful review: the failure entered at the step where the agent treated a partial tool result as complete — everything after that is downstream of one bad assumption. Worth flagging separately: eleven identical retries after a failed call, which isn't incorrect, just expensive.
What makes someone good at it: Reading logs without skimming, and thinking in terms of what the state of the world was at step four. Engineers and anyone who has debugged someone else's pipeline tend to do well here.
What the work is actually like
It is detail-heavy reading and writing, in volume. Individual tasks are short but rarely quick: a single preference comparison with a justification you'd be willing to defend takes real minutes, and a reference answer in a technical domain can eat an afternoon. If you enjoy careful written reasoning, that's the good news. If you were hoping for something you can do while half-watching television, it isn't that.
The quality bar is strict and it is enforced. Expect known-answer items seeded into your queue, agreement scores against other contributors, sampled reviews of your written justifications, and the real possibility of losing access to a project for drifting away from the guideline. That isn't arbitrary cruelty — a dataset is only as good as its least consistent contributor — but it does mean the work rewards people who don't mind being measured.
Availability fluctuates, sometimes violently. Work arrives in project-shaped bursts tied to what a client needs this month; a queue that's full on Monday can be empty on Thursday, and a project you qualified for can simply end. None of this is a salary, and treating a good week as your new baseline is the most common way people get burned.
Unpaid time is real and it belongs in your math. Reading guidelines, qualification tests, onboarding, calibration rounds, re-reading revised guidelines: some platforms pay for some of that, some pay for none of it. Either way it's hours you spent to earn, so your honest number is total pay divided by total hours — not the rate on the listing.
Median listed rate
$75/hr
The median listing midpoint across the 697 of 937 open roles that publish an hourly figure on the one board we track, synced September 13, 2026. The middle half of those listings sits between $50 and $105/hr, and the full spread runs $6–$400/hr at the extremes. A listed rate, not a tracked take-home.
Top specialty: Law
$120/hr
The median listing midpoint across the 77 open Law listings that publish an hourly figure, synced September 13, 2026; their middle half sits between $100 and $145/hr. This is the specialty that ranks highest by median midpoint among those with at least 5 priced roles. The platform assigns these specialty labels, and a high median doesn't mean those roles are open to you.
Listed rates are also spread far wider than any single figure suggests. The middle half of priced roles on that board sits between $50/hr and $105/hr, which is the closest thing to a typical range we can honestly show; the extremes run from $6/hr at the bottom to $400/hr at the top, across 697 priced listings synced September 13, 2026. All of which is a polite way of saying that “AI training pays X” is never a useful sentence. Run your own numbers instead — the pay calculator folds unpaid onboarding and qualification time into an effective hourly rate.
Depth in a field is the biggest lever you control. On the board we track, Law roles rank highest of any specialty by median midpoint, at $120/hr across 77 priced listings, while the lowest-ranked specialty we can report honestly, Robotics, has a median midpoint of $14/hr across 19 priced listings of its own. Same platform, same week, different expertise. Specialist rates breaks that spread down specialty by specialty.
One thing we can't give you yet: what any of this pays in practice, tracked by us, hour by hour. We're working through applications and real task time now, and those numbers land on the individual platform reviews when they exist. Until then everything above is a listed rate from a public board, clearly labeled as such.
Is this a fit for you?
This work suits a fairly specific temperament. Worth being honest with yourself before you spend an evening on a qualification test.
Signals it suits you
- You like precise written reasoning, and you can defend a judgment in two sentences without getting defensive.
- You have real depth somewhere — law, medicine, a science, engineering, a trade, a language — and you can evidence it.
- You want genuinely asynchronous work you can pick up at 6am or 11pm.
- You're comfortable being measured, audited, and corrected against a written standard.
- You want variable work layered on top of something else, not instead of it.
Signals it won't
- You need a predictable number of hours, or an amount you can budget around month to month.
- Reading a long guideline document before you start sounds like the worst part of the job.
- Detailed written feedback on your work lands as criticism rather than information.
- You need guaranteed volume — a queue that empties for two weeks would be a problem, not an inconvenience.
- You'd rather work fast than work exactly to spec. Speed without accuracy is scored as noise.
Genuinely unsure? The fit quiz on the homepage on the homepage gives you a starting point rather than a verdict.
Does this work matter?
Yes, in a specific and unglamorous way. When someone asks a model something that matters to them, the difference between a careful answer and a confidently wrong one traces back, in part, to judgments that people made in tasks exactly like the ones above. Preference data teaches tone and caution as much as it teaches content. A rating nobody thought hard about doesn't disappear — it becomes one vote among many about how the model should behave. That is also the real reason the quality bars feel heavy-handed.
The critique of this industry is also fair, and you should hear it before you sign up for anything. The labor is largely invisible: releases are credited to labs, while the people whose judgment went into them are usually anonymous and under NDA. Rates are set per project by the buyer, vary enormously, and can change or evaporate without much notice. Onboarding and qualification time often goes unpaid. Some projects involve reading material you'd rather not have read. And contributors rarely see what their work was used for, or have any real route to contest a rating that costs them access.
Both of those are true at once, and neither is a reason to take or skip a specific job. “You're building the future” isn't a rate, and a blanket verdict that the whole category is exploitative won't tell you whether the project in front of you pays fairly for the hours it will really take. How common the bad practices are, across a niche this opaque, is not something we can honestly quantify — so we don't print a number for it. Judge the terms of the actual project, and track your own hours.
Where to go next
- The glossary →RLHF, gold tasks, inter-annotator agreement — the words job listings assume you know.
- Platform reviews →Who actually hires for this, with what's confirmed kept separate from what's merely reported.
- Open roles →Every open listing on the board we track, grouped by specialty, with the pay each one publishes.