Almost every large language model in use today descends from one 2017 paper. Google researchers proposed the transformer, an architecture “based solely on attention mechanisms, dispensing with recurrence and convolutions entirely”. Most of the vocabulary below either describes parts of that design or the techniques built on it to train, shrink and serve such models.
This glossary keeps definitions short and, where a term comes from a specific paper or specification, links to it. Terms are grouped alphabetically.
A to C
AI agent. A system that uses a model to plan and take actions, such as calling tools or APIs, over several steps toward a goal, rather than producing a single reply. Coding agents are the most common kind in software work; our comparison of AI coding tools looks at Cursor, Windsurf and GitHub Copilot.
AGI (artificial general intelligence). A hypothetical system able to match humans across most intellectual tasks. There is no agreed test for it.
Alignment. Work to make a model’s behaviour match the intentions and values of the people using it, including refusing harmful requests.
Attention. A mechanism that lets each token in a sequence weigh the relevance of every other token when building its representation. Each token produces a query, a key and a value; queries are compared with keys to decide how much of each value to use.
Backpropagation. The algorithm that computes how much each weight contributed to a model’s error, so the weights can be adjusted.
Benchmark. A fixed test set used to compare models. Scores can be inflated if test data leaks into training data.
BF16, FP16 and FP8. Number formats with 16 or 8 bits per value. Fewer bits mean less memory and faster maths, with some loss of precision. BF16 keeps the range of 32-bit floats with less precision.
Chain-of-thought. Prompting a model to write out intermediate reasoning steps. The original paper found this improved large models’ performance on arithmetic and reasoning tasks.
Checkpoint. A saved copy of a model’s weights at a point in training.
Context window. The maximum number of tokens a model can take in at once, covering the prompt, any documents and its own output.
D to G
Diffusion model. A generative model that learns to reverse a gradual noising process, turning random noise into an image or other output step by step (Ho et al., 2020). Latent diffusion runs that process in a compressed representation to cut compute (Rombach et al.).
Distillation. Training a smaller “student” model to match the output probabilities of a larger “teacher” (Hinton et al.).
Embedding. A list of numbers representing a piece of text, image or audio so that similar items sit close together. Embeddings power semantic search.
Epoch. One full pass through the training data.
Few-shot learning. Giving a model a few worked examples in the prompt. The GPT-3 paper showed a 175-billion-parameter model could perform new tasks this way “without any gradient updates or fine-tuning”.
Fine-tuning. Further training a pretrained model on a narrower dataset to adapt it to a task or style.
Foundation model. A large model trained on broad data that can be adapted to many tasks.
Generative AI. Models that produce new content, such as text, images, audio or code, rather than only classifying or ranking existing data.
GGUF. A binary file format for running models with the GGML library and tools such as llama.cpp. According to its specification, it succeeds the older GGML, GGMF and GGJT formats, holds everything needed to load a model in one file, and can be memory-mapped for fast loading.
Gradient descent. The method of repeatedly nudging weights in the direction that reduces error.
H to L
Hallucination. Output that is fluent but false or unsupported, such as an invented citation.
Hidden layer. Any layer of a neural network between the input and the output.
Human-in-the-loop. A design in which a person reviews or approves a model’s output or action before it takes effect.
Inference. Running a trained model to produce output, as opposed to training it.
Instruction tuning. Fine-tuning on examples of instructions and good responses so a model follows requests.
Jailbreak. A prompt crafted to make a model ignore its safety rules.
JSON mode or structured output. An API option that constrains a model to return valid JSON, often matching a supplied schema.
KV cache. Stored keys and values from earlier tokens, so the model need not recompute them for each new token. It speeds generation but uses a lot of memory on long contexts.
LLM (large language model). A transformer trained on very large amounts of text to predict the next token. Pretraining is usually called self-supervised, because the labels come from the text itself.
LoRA (low-rank adaptation). A fine-tuning method that freezes the original weights and trains small added matrices. The LoRA paper reported 10,000 times fewer trainable parameters and three times less GPU memory than full fine-tuning of GPT-3 175B.
M to P
MCP (Model Context Protocol). An open protocol, originally published by Anthropic, for connecting AI applications to tools and data. The specification uses JSON-RPC 2.0 messages between hosts, clients and servers and takes inspiration from the Language Server Protocol used by code editors. Our MCP explainer covers setup.
MLOps. Practices and tools for deploying, monitoring and updating models in production.
MoE (mixture of experts). An architecture in which a gating network sends each input to a few specialised sub-networks, so only part of the model runs each time (Shazeer et al., 2017).
Multimodal. Able to take in or produce more than one type of data, such as text and images. Vision-language models, which answer questions about pictures, are the kind most often run at home; our guide to running vision LLMs locally covers LLaVA, BakLLaVA and others.
Neural network. Layers of simple units connected by weighted links, trained to map inputs to outputs.
Ollama. An open-source tool for downloading and running language models on a local machine.
Open-weight model. A model whose trained weights are published. That is not the same as open source, which would also cover training code and, ideally, data, under an open licence. Our self-hosting guide walks through running open-weight models such as DeepSeek-R1 and Llama 3 on your own hardware.
Parameter. A learned number inside a model, mostly weights. Model sizes such as “7B” count parameters in billions. Fewer parameters means less memory, which is why small models suit modest machines; our guide to self-hosting small LLMs, from Raspberry Pi to MacBook Pro covers what runs where.
Perplexity. A measure of how well a model predicts text; lower is better.
Physical AI. AI built into machines that act in the physical world, such as robots, vehicles and factory equipment. Such systems are often trained and tested on a digital twin, a simulated copy of a real machine or site, before they touch the real thing. Investors are betting on the idea: TSN covered the Wall Street Journal’s report that Jeff Bezos was seeking $100bn for a fund to buy manufacturing companies and automate them with AI.
Post-training. Everything after pretraining, including instruction tuning and reinforcement learning from human feedback.
Prompt injection. Hidden instructions in content a model reads, such as a web page or email, that try to hijack its behaviour.
Q to S
Quantisation. Storing weights with fewer bits, such as 4-bit integers, to cut memory and speed up inference. Our quantisation deep dive compares the main formats.
QLoRA. LoRA fine-tuning on top of a 4-bit quantised model. The QLoRA paper fine-tuned a 65-billion-parameter model on a single 48GB GPU, using a 4-bit NormalFloat data type, double quantisation and paged optimisers.
RAG (retrieval-augmented generation). Fetching relevant documents and adding them to the prompt so the model answers from them. The original RAG paper combined a generator with a dense vector index of Wikipedia. Vector databases usually supply the retrieval step; see our vector database guide.
Reasoning model. A model trained to spend extra tokens working through a problem before answering.
Reinforcement learning. Training by trial and error, in which a model is rewarded for good outcomes rather than shown correct answers.
RLHF (reinforcement learning from human feedback). Training a reward model on human rankings of outputs, then optimising the language model against it. In OpenAI’s InstructGPT paper, human raters preferred outputs from a 1.3-billion-parameter RLHF model to those of the 175-billion-parameter GPT-3.
RNN (recurrent neural network). An older architecture that processes a sequence one step at a time. Transformers largely replaced RNNs for language.
Scaling laws. Empirical rules linking model performance to parameters, data and compute. Kaplan et al. found loss falls as a power law with each. DeepMind’s Chinchilla paper argued parameters and training tokens should grow in equal proportion; its 70-billion-parameter model, trained on four times more data, outperformed the 280-billion-parameter Gopher.
Self-attention. Attention in which a sequence attends to itself; the core operation of a transformer.
Semantic search. Search by meaning, using embeddings, rather than by matching keywords.
T to Z
Temperature. A setting that controls randomness when picking the next token. Low values make output more predictable.
Token. The unit a model reads and writes: a word, part of a word or a symbol.
Tokeniser. The component that splits text into tokens and maps them to numbers.
Transformer. The attention-based architecture described above, used by nearly all current language models.
Vector database. A database that stores embeddings and finds the nearest ones to a query quickly.
vLLM. An open-source inference engine built around PagedAttention, which manages the KV cache like virtual memory pages. Its paper reported two to four times higher throughput than earlier systems at similar latency.
Weights. The learned numbers that define a trained model.
Whisper. OpenAI’s open speech-recognition model, trained on 680,000 hours of weakly supervised audio (Radford et al.).
Zero-shot learning. Asking a model to do a task with no examples in the prompt, relying only on an instruction. One-shot means a single example.
Definitions drift as fast as the models
Several terms here have shifted meaning within a few years. “Open source” is now routinely applied to models whose weights are downloadable but whose training data and code are not, which is why “open-weight” has become the more precise label. “Agent” covers everything from a chatbot with one tool to systems that run unattended for hours. Even the numbers move: claims that a technique runs a given model “on consumer hardware” are often looser than the source paper, as with QLoRA, whose headline result used a 48GB card. The safest habit is to go back to the paper or specification, which is why each definition above links to one where it exists. Developers comparing tools should treat marketing uses of these words as starting points for questions, not as definitions.
Most of this jargon names a trade-off
Read together, many of these terms describe the same bargain: give up a little accuracy or flexibility to save memory, compute or money. Quantisation trades precision for size. LoRA and QLoRA trade the freedom of full fine-tuning for far smaller training costs. Distillation trades a large teacher’s capability for a cheaper student. Mixture of experts runs only part of a model per token. The KV cache spends memory to save compute, and PagedAttention manages that memory more efficiently. RAG swaps retraining for a retrieval step that can go stale or fetch the wrong passage. Knowing which side of each trade a project can afford is more useful than knowing the definitions alone, and it is the practical starting point for getting good results from small self-hosted models.
Sources
- Vaswani et al., Attention Is All You Need (2017)
- Brown et al., Language Models are Few-Shot Learners (2020)
- Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2022)
- Ouyang et al., Training language models to follow instructions with human feedback (2022)
- Hu et al., LoRA: Low-Rank Adaptation of Large Language Models (2021)
- Dettmers et al., QLoRA: Efficient Finetuning of Quantized LLMs (2023)
- Kwon et al., Efficient Memory Management for LLM Serving with PagedAttention (2023)
- Hinton et al., Distilling the Knowledge in a Neural Network (2015)
- Shazeer et al., Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (2017)
- Ho et al., Denoising Diffusion Probabilistic Models (2020)
- Rombach et al., High-Resolution Image Synthesis with Latent Diffusion Models (2021)
- Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision (2022)
- Kaplan et al., Scaling Laws for Neural Language Models (2020)
- Hoffmann et al., Training Compute-Optimal Large Language Models (2022)
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)
- GGUF specification (ggml-org on GitHub)
- Model Context Protocol specification
Related stories
- MCP Explained: The USB-C for AI Tools (Model Context Protocol Guide)
- Quantization Deep Dive: GGUF, AWQ, GPTQ, EXL2 Compared (2026 Guide)
- Vector Databases for RAG: From Chroma to Production (2026 Guide)
- Prompt Engineering for Self-Hosted LLMs: Getting the Most from Small Models
- AI Coding Tools 2026: Cursor, Windsurf, Copilot Compared
- Running Vision LLMs Locally: LLaVA, BakLLaVA & Beyond (2026 Guide)
- Self-Hosting LLMs in 2026: The Complete Setup Guide (DeepSeek-R1, Llama 3, and Beyond)
- Self-Hosting Small LLMs: From Raspberry Pi to MacBook Pro (2026 Edition)
- Bezos’s 100 Billion Bet: The AI Manufacturing Takeover Explained
Follow @tsnmediahub on X for breaking AI news as it happens.
Follow on X
