AI Terms, Properly Explained

Large Language Model

At its core, an LLM is a next-word predictor: given everything written so far, it produces a probability distribution over what token comes next, samples one, and repeats. The "large" refers to the number of tunable parameters and the volume of text it was trained on — scale is what turns a crude autocomplete into something that can hold a coherent conversation, write code, or summarize a contract.

It has no persistent memory of you, no beliefs, and no awareness between requests — every reply is reconstructed from the conversation you send it, fresh each time.

Transformer & Attention

The transformer is the neural network architecture behind essentially every modern LLM. Its key idea is attention: for every word in a sentence, the model computes how relevant every other word is to it, and weighs them accordingly. That's how "bank" is interpreted correctly in "I deposited my paycheck at the bank" — the model isn't reading the sentence in isolation, it's weighing "deposited" and "paycheck" heavily when deciding what "bank" means.

Tokens

Models don't read whole words — they read tokens, which are often word fragments. "Unbelievable" might split into pieces like un, believ, able. This lets a fixed vocabulary represent any input, including typos, rare words, and other languages, without needing a dictionary entry for every possible word.

Pricing, rate limits, and context window size are all measured in tokens, not words or characters — a useful fact once you're budgeting API costs.

Parameters

Parameters are the model's internal tunable numbers — weights adjusted during training. Individually, a single parameter means nothing; collectively, billions of them encode everything the model "knows" about language, facts, and patterns. When you hear a model described as "70B" or "400B," that's a parameter count, and it's a rough proxy — not a guarantee — for capability.

Context Window

The context window is the maximum amount of text — measured in tokens — a model can consider at once: your prompt, the conversation history, any documents you've pasted in, and the response it's generating, all sharing one fixed budget. Once that budget is full, the oldest content has to be dropped or summarized to make room for new input.

Prompting, Zero-Shot & Few-Shot

Prompting is simply the instruction you give the model — its entire "understanding" of a task comes from the wording and examples in that one message. A zero-shot prompt asks for a task with no examples ("translate this to Spanish"); a few-shot prompt includes a handful of worked examples first, letting the model infer the pattern before it sees your real input.

Temperature

Temperature controls how much randomness goes into picking the next token. At low temperature, the model almost always picks its highest-probability guess — consistent, a little predictable. At high temperature, it's more willing to pick a less likely token — more varied, occasionally incoherent. It's a dial between the safe, expected answer and the more adventurous one, not a measure of "creativity" in any deeper sense.

Training, in Three Stages

"Training" usually bundles together three distinct jobs:

Fine-Tuning

Fine-tuning takes an already-trained model and continues training it on a smaller, specific dataset, adjusting its existing parameters rather than starting over. It's how a general-purpose model becomes noticeably better at a narrow task — a particular writing style, a support-ticket format, a company's internal terminology — without the cost of training from scratch.

Hallucination

A hallucination is a fluent, confident, and wrong answer — a fabricated citation, a nonexistent API method, a plausible-sounding but false fact. It happens because the model is fundamentally predicting likely-sounding text, not consulting a database of verified truth. Nothing in the architecture distinguishes "I actually know this" from "this is a very plausible-sounding sentence."

Reasoning Models

Reasoning models are trained or prompted to work through a problem step by step before producing a final answer, rather than jumping straight to a response. That extra deliberation — thinking it through on purpose — tends to improve performance on multi-step problems like math and logic puzzles, at the cost of slower, more expensive responses.

Multimodal

A multimodal model can take in — and sometimes produce — more than one kind of input: text, images, audio, or video, processed through the same underlying architecture. Instead of a text-only model, it has more than one sense to perceive the world with.

AI Agents

An AI agent is a model wired up with the ability to take actions — call an API, run code, query a database — and observe the results, typically in a loop: perceive, reason, act, observe, repeat. That's the line between a chatbot, which only produces text, and an agent, which can actually do something with the world instead of just describing what it would do.

Context Engineering

Context engineering is the practice of deliberately deciding what goes into a model's limited context window — which documents, which conversation history, which tool outputs — rather than dumping everything available onto the table before it opens its mouth. As context windows fill with tool results and history, curating what actually gets sent in becomes as important as the prompt itself.

AGI vs. ASI

AGI (artificial general intelligence) refers to a system with human-level ability across essentially any intellectual task, not just the narrow ones it was trained for. ASI (artificial superintelligence) refers to a hypothetical system that exceeds human ability broadly. Both remain aspirational terms — current systems, however capable at specific tasks, are neither.

Temperature, Live

Drag the slider to see how the same prompt — "Describe a cat" — might shift as temperature increases, from the safest possible completion to a more unusual one.