Reference
Glossary of AI training work
Onboarding documents in this niche are written for people who already work in it. Here are 46 of the words they use without explaining, in plain English — the queue, the work, the money, and the scores that decide what you get to see next. Every definition below is ours, describing how this work generally operates; none of it is any platform's policy.
Start here
If you only learn one pair of terms, make it advertised rate vs effective hourly rate. Of the 937 open roles we track, 697 publish an hourly figure; the median listing sits at $75/hr at the midpoint of its own range, and the middle half of them fall in the $50–$105/hr band. Those are the numbers on the postings: what the listings advertise, not what anyone has been shown to earn. What reaches your bank account depends on unpaid onboarding, rejected tasks, dry spells, and the tax nobody withheld — which is what the rest of this glossary is about.
Platform mechanics
How work reaches you: queues, gates, trackers, and the levers that decide whether there is anything to do today.
- Task queue
- The list of available work a platform shows you once you are qualified. You are usually not assigned anything: you open the queue and take what is there, which means your week depends as much on what the queue holds as on how fast you work.
- Batch
- A bundle of similar tasks released together under one set of instructions and one rate. Batches tend to appear and empty out quickly, so a lot of this work is about being at the keyboard when one drops.
- Qualification
- A gate that unlocks one particular kind of work — sometimes a quiz, sometimes a sample task a human reviews. Passing one usually opens that project only, not the whole platform, so expect to qualify repeatedly.
- Screening / assessment
- The test that decides whether you get in at all: writing samples, coding exercises, domain questions, sometimes a recorded interview. It is often unpaid and rarely quick, so count those hours when you judge what a platform really pays.
- Onboarding
- Everything between "you're accepted" and "you're earning": paperwork, identity verification, payment setup, instruction reading, sometimes a calibration exercise. Some platforms pay for parts of it and some do not, and that choice is theirs.
- Project
- One client's body of work, with its own instructions, rubric, rate, and quality bar. Because you are typically qualified per project, "what does this platform pay?" almost always resolves to "which project are you on?".
- Work order
- The formal record of a specific engagement — what you agreed to do, at what rate, for how long. Where a platform issues one, treat it as the contract and read the rate and end date before you accept.
- Time tracker
- Software, or a timer built into the task screen, that logs the minutes you are allowed to bill. Trackers commonly pause on idle, cap the time allowed per task, or exclude instruction reading — which is why tracked hours come in under the hours you actually sat there.
- Effective hourly rate vs advertised rate
- The advertised rate is the number in the listing. Your effective hourly rate is what you were paid divided by every hour you really spent — unpaid onboarding, instruction reading, rejected tasks, and time spent waiting for work included. The gap between the two is the most important figure in this line of work, and only your own log can tell you what it is.
- Payout cycle
- How often a platform actually pays and how far behind the work it runs — weekly, twice monthly, monthly, or on project milestones. Worth confirming before you start, because the first payment is usually the longest wait.
- See alsoGross vs net pay
- Deactivation
- Losing access to a project or the whole platform. Sometimes it follows quality scores, sometimes the client's contract simply ended, and the explanation you get can be thin — which is the main argument for not letting one platform carry all of your income.
- Task availability / dry spell
- A stretch where the queue is empty even though your account is in good standing. Demand comes from client contracts, so it moves on their schedule, not yours, and a quiet fortnight is not necessarily a signal about you.
- Platform stacking
- Keeping accounts live on several platforms at once so a dry spell on one does not zero out your week. The cost is more unpaid onboarding, more rubrics to hold in your head, and more context switching.
- Domain / specialty
- How platforms slice work by field — software engineering, law, medicine, languages, and so on. Your domain usually drives both what appears in your queue and what it pays, which is why a specialist board is worth reading by section rather than as one list.
- NDA
- A nondisclosure agreement, which most projects in this niche run under. It typically covers the client's identity, the instructions, and the content of the tasks — the reason honest write-ups of this work (ours included) describe process and pay rather than showing you real tasks or assessment questions.
The work itself
What you actually do at the keyboard, and the model-training vocabulary the instructions assume you already know.
- Annotation
- Adding structured information to raw data so a model can learn from it: tagging objects in an image, marking the entities in a sentence, flagging whether an answer is unsafe. It is the umbrella term for most data work.
- Labeling
- Attaching a category from a fixed list to an item — spam or not spam, positive or negative, safe or unsafe. Narrower than annotation, and usually the fastest-paced and lowest-priced end of it.
- RLHF
- Reinforcement learning from human feedback: a training approach where people rate or rank model outputs, and those judgments are distilled into a reward model that nudges the system toward the preferred behavior. Most gigs that ask you to pick the better of two answers are feeding an RLHF pipeline.
- Preference ranking / A-B comparison
- You are shown two or more responses to the same prompt and asked which is better, almost always with a written justification. The justification is frequently what is really being graded, so a bare "B is better" is the classic way to fail an otherwise correct judgment.
- SFT demonstration
- Supervised fine-tuning demonstration: a worked example you write yourself, pairing a question with the ideal answer so the model can imitate it. Slower and more demanding than rating someone else's output, and usually priced accordingly.
- Prompt writing
- Authoring the inputs used to train or test a model — realistic questions, instructions, or scenarios, normally to a brief ("ten questions a small-business owner would ask about payroll"). Briefs reward realism and variety, not cleverness.
- Red-teaming
- Deliberately trying to make a model misbehave — produce something unsafe, break its own rules, or reveal instructions it should not — so the failures can be fixed. It is often specialist work, and it can mean sustained exposure to unpleasant material, which is worth weighing before you take it on.
- Rubric
- The written standard your work is graded against on a project: what counts as accurate, complete, well-formatted, and safe. Read it before your first task and read it again whenever it changes, because a quiet rubric update is a common cause of a sudden score drop.
- Grader / evaluator
- A role centered on scoring output against a rubric — model responses, or sometimes other workers' submissions — rather than producing new content. Expect to justify scores in writing.
- Gold standard / ground truth
- The answer a project treats as correct, usually written by an expert or settled by several reviewers agreeing. Hidden gold-standard items are commonly mixed into real work to measure how close you are to it.
- Inter-annotator agreement
- How often independent workers give the same answer to the same item. When it is low, the usual cause is ambiguous instructions rather than careless people — and saying so, with examples, is a legitimate thing to raise with a project lead.
- Edge case
- An item that sits at the boundary of the instructions, where the rubric does not clearly apply. Handling edge cases well — flagging them with a reason instead of quietly guessing — is much of what separates a high scorer from an average one.
- Hallucination
- A model stating something false as though it were established fact: an invented citation, a confident wrong figure, a function that does not exist. Checking the claims inside a fluent answer is a large share of evaluation work.
- Model response
- One output the model produced for one prompt. It is the unit you will most often rate, edit, or compare, and it is sometimes called a completion or generation.
- Reward model
- A model trained on human preference judgments that then scores new outputs automatically, standing in for a human rater at scale. It is where your ranking work ultimately ends up.
- Eval / benchmark
- A fixed set of test items used to measure how well a model does something, so versions can be compared on the same footing. Writing and grading eval items is its own category of work and often needs genuine subject expertise.
- Adversarial prompt
- An input crafted to push a model into failing — trick framing, a harmful request disguised as something benign, instructions hidden inside content the model is asked to summarize. The building block of red-teaming.
Money and status
What these words mean. The practical side — what to set aside and when it is due — lives in the taxes guide.
- Independent contractor
- You are in business for yourself rather than employed by the platform: nothing is withheld from your pay, there are no employer benefits, and you control your own hours. The same status means the arrangement can end with little notice.
- 1099-NEC
- The US information return a business uses to report money it paid to a non-employee. If a platform issues you one, the IRS has a copy — and the income is reportable whether or not any form ever arrives, which catches out a lot of first-year contractors.
- Self-employment tax
- The Social Security and Medicare contribution that employees split with an employer and that contractors owe on their own net profit. It sits on top of income tax rather than replacing it. The rate, the earnings thresholds, and the deduction that partly offsets it are all published by the IRS; look up the current figures at irs.gov.
- Estimated / quarterly taxes
- Because nobody withholds tax from contractor pay, the US system expects you to pay in during the year instead of settling up once at filing. The IRS sets who has to do it, the due dates, and the penalty for coming up short — check the current rules at irs.gov before you plan around them.
- Gross vs net pay
- Gross is what the platform paid you. Net is what survives your deductible business expenses and the tax owed on the profit. Budget from net: the difference is big enough that treating gross as income is the most common money mistake in gig work.
- Referral bonus
- A one-time payment for bringing someone to a platform, usually conditional on them being accepted and sometimes on them completing a minimum amount of work first. Terms vary by platform and are not always published, so read them before you count on the money.
- See alsoRevenue share
Quality and review
The scoring machinery that decides which work you see next, and what to do when it goes against you.
- Spot check
- A review of a sample of your work rather than all of it. Because it is a sample, one weak task reviewed at the wrong moment can move your score further than its share of your output deserves.
- Audit
- A deeper, systematic review of a body of work — yours or a whole project's — usually triggered by a quality concern or a client complaint. It can end in re-work, unpaid rejections, or removal from the project.
- Rejection
- A submitted task the reviewer refuses, normally unpaid. Rejections are the quietest drain on an effective hourly rate, which is why it is worth asking for the specific reason rather than shrugging and moving to the next item.
- Appeal
- The route for contesting a rejection or a score. Where one exists it is usually time-limited and wants specifics: the task identifier, the rubric line you are relying on, and a short factual argument rather than a complaint.
- Accuracy score
- A rolling quality number a platform keeps on you, built from gold-standard comparisons, spot checks, or reviewer ratings. It often controls which work you can see and whether you keep access, and it typically recovers more slowly than it falls.
- Calibration
- A round of practice items with known answers, run before or during real work so everyone grades the same way. If your answers drift from the calibration set, that is information about the instructions as much as about you.
- Attention check
- An item planted in a batch with an obvious correct answer, there to catch people clicking through without reading. Failing one reads as bad faith rather than tiredness, so the items that look trivially easy are the ones to slow down on.
Where to go next
- What this work actually is, day to day →
- Where specialist expertise changes the rate →
- Taxes for contract AI training work →
Spotted a word a platform used on you that isn't here? It probably belongs on this page — we add terms as they turn up in the platforms' own published material.