How models are made · Part 2 — The AI Dictionary

A cheap fine-tuning method that trains a tiny add-on instead of the whole model.

Full fine-tuning updates every parameter — expensive. LoRA freezes the original model and trains a small set of extra weights that sit alongside it. The add-on is often just megabytes and can be swapped in and out.

Think of it like: Clip-on lenses for a camera: you don't rebuild the camera, you snap on a specialty lens.

Shrinking a model by storing its numbers with less precision, so it runs faster on smaller hardware.

Model weights are normally stored as 16-bit numbers. Quantization rounds them to 8, 4, or even fewer bits. The model gets 2–4× smaller and faster with surprisingly little quality loss.

Think of it like: Saving a photo as a high-quality JPEG instead of a giant RAW file. Slightly less detail, dramatically smaller.

A model built from many specialist sub-networks where only a few 'experts' activate for each token.

Instead of one giant network doing all the work, an MoE model has many expert blocks plus a router that picks the best few for each token. The model can have a huge total parameter count while using only a fraction per step — big-model knowledge at closer to small-model cost.

Think of it like: A hospital with a triage nurse: you don't see every doctor, just the two specialists your case needs.

Making AI systems reliably do what people actually intend — helpful, honest, and not harmful.

Alignment is the research field (and the practical work) of getting models to pursue the goals we mean, not just the goals we literally typed — and to refuse genuinely harmful requests. RLHF, constitutional training, red-teaming and interpretability research all fall under it.