Human data glossary

Plain-language definitions of the terms you will meet when specifying, collecting, labeling and evaluating data for AI.

Acceptance criteria

The written conditions an item must meet to be accepted into a dataset, such as format, accuracy and completeness. They make quality reviews objective rather than a matter of opinion.

Adversarial prompt

A prompt written deliberately to make a model fail, for example by being misleading, ambiguous or designed to provoke unsafe output. Used in prompt testing and red-teaming.

Annotation

Adding labels or structured information to raw data, such as tagging entities in text, drawing boxes around objects in images or marking speakers in audio.

ASR (automatic speech recognition)

Technology that converts spoken audio into text. ASR models are trained and tested on recorded speech paired with accurate transcripts.

Bounding box

A rectangle drawn around an object in an image or video frame to show where it is. One of the most common computer vision annotation types.

Calibration

A step in which contributors or reviewers work through shared examples and compare their answers against an agreed standard, so that everyone applies the guidelines the same way before production work begins.

Code-switching

Moving between two or more languages within a conversation or sentence. Common in many multilingual communities, and a frequent weak point for language models.

Data specification

A written document describing exactly what a dataset should contain: format, metadata, contributor profile, guidelines and acceptance criteria. It is the reference every later quality check is measured against.

Fine-tuning

Further training of an existing model on a smaller, targeted dataset to change or improve its behavior for a specific task or domain.

Gold-standard item

An item with a known, verified answer, mixed into regular work to measure how accurately contributors are performing.

Held-out test set

Data kept separate from training and used only to evaluate a model. Newly collected, unpublished data reduces the risk that a model has already seen the test items.

Human evaluation

Assessment of model output by people, such as rating answers against a rubric or having experts check correctness, used where automated metrics cannot judge quality.

Instruction data

Examples of instructions paired with high-quality responses, used to fine-tune language models to follow requests helpfully and accurately.

Inter-annotator agreement

A measure of how often different annotators give the same label to the same item. Low agreement usually signals unclear guidelines or a genuinely ambiguous task.

Localization

Adapting content for a specific language and market so that it reads naturally and is culturally appropriate, rather than only translating it word for word.

Pilot

A small first batch of work used to test the specification, guidelines, contributors and quality process before committing to full production volume.

Prompt testing

Systematically probing a model with realistic and adversarial prompts and documenting where it fails.

RLHF (reinforcement learning from human feedback)

A training approach in which people compare or rate model outputs, and those preferences are used to steer the model toward more helpful and safer responses.

Rubric

A structured set of criteria and scoring levels used to rate model output consistently across reviewers.

Segmentation

Labeling every pixel of an image according to the object or region it belongs to, giving a more precise outline than a bounding box.

Speaker labeling

Marking which person is speaking at each point in an audio recording, often together with timestamps.

TTS (text-to-speech)

Technology that turns written text into spoken audio. TTS voices are trained on clean, consistent recordings, often from professional voice talent.