Distillation
Training a smaller model to imitate the behavior or outputs of a larger model.
Plain English
A big model teaches a smaller one.
Example
A team uses a frontier model to generate labels, then trains a cheaper classifier on those outputs.
Why it matters
Distillation helps reduce cost and latency while preserving enough task quality.