◉ ai

Tokens

The basic units AI uses to process text

The fundamental units of text that AI models process — roughly ¾ of a word in English, though the ratio varies by language. Every input and output is measured in tokens, which determines both cost and context window usage. A 1,000-word article is approximately 1,300 tokens. Understanding token counting is essential for managing API costs and fitting content within a model's context window.

Related terms

Context Window

Maximum text an AI can process at once

LLM (Large Language Model)

AI models that understand and generate text

Inference

Using a trained AI model to generate outputs