What is…
Token
The chunk of text a model actually reads and writes — roughly ¾ of a word on average.
Models don't see letters or words; they see tokens. A token might be a whole short word ("cat"), part of a longer word ("un" + "believ" + "able"), a space, or punctuation. In English, 100 tokens is about 75 words.
Tokens matter because everything is measured in them: how much text fits in the model's memory (the context window), how fast it responds, and — if you use an API — what you pay.
💡 Think of it like
Lego bricks of language. The model builds every sentence one brick at a time.
📌 Example
"Tokenization isn't hard" might split into: Token · ization · isn · 't · hard.
🧠 Test yourself
Which of these describes Token?
Related terms
Context window
How much text a model can 'see' at once — your conversation, files and instructions, all measured in tokens.
LLM (Large Language Model)
A model trained on huge amounts of text to predict the next word — the engine behind ChatGPT, Claude, Gemini and Grok.
Inference
Running a trained model to get an answer — as opposed to training it.
🎮 Learn AI by playing
40 bite-size missions, boss battles and a certificate. Free.
Start the bootcamp →☀️ One AI term every morning
Plus the day's top stories, in your inbox by 8am.