Skip to content
Back to Glossary
◉ ai

Distillation

Compressing knowledge from a larger model into a smaller, faster one

What is Distillation?

A technique where a smaller 'student' model learns to replicate the behavior of a larger 'teacher' model, retaining most of the capability at a fraction of the cost and latency. Distillation is why you can run surprisingly capable models on a laptop. For builders, it means you don't always need the biggest model — you need the right-sized model for your task.

Take the next useful step

LV’s AI hub

Original context on AI applied to creation, research, and systems.

Creator OS

Useful when AI is part of a content workflow that still needs human judgment.

💡

In plain words

"Think of it like a senior mentor transferring know-how to a junior — cheaper, faster, still sharp."

How it works

Key takeaways

  • Small model learns from a larger teacher model

  • Maintains quality while cutting inference cost

  • Core technique for deploying AI at scale

▸

Real-world example

You run 10,000 queries through GPT-4 (the teacher) and collect its outputs. Then you fine-tune a smaller model like Llama 8B on those outputs. The small model captures most of GPT-4's quality at a fraction of the inference cost.

Related terms

Inference

Using a trained AI model to generate outputs

Fine-Tuning

Training an AI model on your specific data

Open Weights

AI models anyone can use and modify

Copied