The basics · Part 1 — The AI Dictionary

A model trained on huge amounts of text to predict the next word — the engine behind ChatGPT, Claude, Gemini and Grok.

A large language model is a neural network trained on a vast slice of the internet, books and code to do one thing: predict what text comes next. Do that well enough, at enough scale, and you get something that can explain, summarize, write, translate and code.

Think of it like: The world's most well-read autocomplete — one that has read so much it can finish your thought, your essay, or your code.

Example: ChatGPT, Claude, Gemini and Grok are all apps built on top of LLMs.

The chunk of text a model actually reads and writes — roughly ¾ of a word on average.

Models don't see letters or words; they see tokens. A token might be a whole short word ("cat"), part of a longer word ("un" + "believ" + "able"), a space, or punctuation. In English, 100 tokens is about 75 words.

Think of it like: Lego bricks of language. The model builds every sentence one brick at a time.

Example: "Tokenization isn't hard" might split into: Token · ization · isn · 't · hard.

How much text a model can 'see' at once — your conversation, files and instructions, all measured in tokens.

The context window is the model's working memory for a single conversation. Everything counts against it: your messages, its replies, pasted documents, uploaded files, and hidden instructions. Modern models range from roughly 100,000 to a million-plus tokens.

Think of it like: A desk. A bigger desk holds more papers at once, but a cluttered desk still makes it harder to find the one that matters.

Example: A 200K-token window fits roughly a 500-page book.

The billions of internal numbers a model learns during training — its 'knowledge', stored as weights.

Parameters (also called weights) are the adjustable numbers inside a neural network. Training nudges them, trillions of times, until the model gets good at predicting text. A "70B" model has about 70 billion of them.

Think of it like: The knobs on a gigantic mixing board. Training is turning billions of knobs until the song sounds right.

Example: "8B" and "405B" in a model's name refer to billions of parameters.

The neural-network design (from Google, 2017) that nearly every modern AI model is built on.

The transformer is the architecture introduced in the 2017 paper "Attention Is All You Need." Its breakthrough was attention: letting every word in a passage look at every other word to figure out what matters, all in parallel. That made it possible to train on enormous datasets efficiently.

Think of it like: Instead of reading a sentence one word at a time like a person, it reads the whole page at once and draws lines between the words that relate.

The mechanism that lets a model weigh which earlier words matter most for the next one.

In "The trophy didn't fit in the suitcase because it was too big," attention is what lets the model connect "it" to "trophy" rather than "suitcase." Every token scores its relevance to every other token, and the model blends information accordingly.

Think of it like: A highlighter the model runs over the conversation, over and over, deciding what to focus on for each word it writes.