AI
Software that finds patterns in data and uses them to answer
AI is software that looks at a huge number of examples, finds the patterns in them, and uses those patterns to answer or decide. It is not alive and it does not think the way a person does. It calculates, very fast, what is most likely to come next.
Learn more
LLM
An AI trained on enormous amounts of text to continue sentences
A large language model has read an enormous amount of text and learned which words tend to follow which. When you write to it, it keeps choosing the next likely word until an answer is formed. That is why it can sound fluent and still be wrong.
Learn more
Prompt
What you write to the AI — your instruction
A prompt is simply what you type to the AI. It is not a magic spell, but wording matters: the clearer you say who you are, what you want, and in what form, the better the answer. Rewriting a weak prompt is usually faster than arguing with the answer.
Learn more
System prompt
Standing instructions given to the AI before the conversation starts
A system prompt is the instruction placed before your message: who the assistant should be, what it must always do, what it must never do. You usually do not see it, but it shapes every answer. Lyen's teaching rules live in one.
Token
The small pieces of text an AI actually counts
An AI does not read letters or whole words; it cuts text into small pieces called tokens. A short word may be one token, a long one several. Length limits and API prices are counted in tokens, not in characters, so this word shows up whenever cost or capacity is discussed.
Context window
How much text the AI can hold in view at one time
The context window is the desk the AI works on: your question, the conversation so far, and any documents you pasted must all fit on it. When the conversation grows past that size, the oldest part slides off and the AI truly cannot see it any more. That is why a long chat starts forgetting what you said at the beginning.
Hallucination
A confident answer that has no basis in fact
In this field the word means one thing only: output that is wrong or unsupported, stated as if it were certain. It has nothing to do with a person seeing things — the machine is not experiencing anything. It happens because the model is choosing likely-sounding words, not checking a source, so always verify names, numbers and quotations.
Temperature
A dial between safe, repeatable answers and surprising ones
Temperature decides how boldly the model picks the next word. Set it low and it keeps choosing the most likely word, so answers become steady and repeatable — good for facts and code. Set it high and it reaches for less likely words, so answers become varied and more error-prone — useful for brainstorming.
Parameter
One of the many numbers a model adjusts while learning
A parameter is any number a model adjusts by itself while it learns. Weights are the largest group, and biases, embeddings and normalization coefficients are parameters too. A 7B model has about 7 billion of them. More makes a model heavier to run, but not automatically better.
Learn more
Training
Showing a model examples until its numbers settle
Training is where a model is built: it is shown example after example, guesses an answer, is told how far off it was, and its numbers are nudged a little closer. Repeat that billions of times and the patterns stick. It is slow and expensive, which is why models are trained once and then used many times.
Fine-tuning
Teaching an already-trained model your own examples
Instead of building a model from nothing, you take a finished one and keep training it on a smaller set of your own examples, so it picks up your tone or your subject. It is far cheaper than training from scratch. For most beginners, though, a clearer prompt solves the problem without any fine-tuning at all.
Learn more
Inference
Actually running a trained model to get an answer
Inference is the everyday use of a model: you send text in, the model does its calculation, an answer comes out. Nothing is learned during inference — the model is exactly the same before and after. When people talk about the cost or speed of an AI service, they usually mean inference.
RAG
Look the facts up first, then let the AI answer
With RAG, the system first searches your documents for passages related to the question, then hands those passages to the AI and asks it to answer using them. The AI is no longer answering from memory, so made-up answers drop sharply and you can check the source it used.
Agent
An AI that can use tools and take steps on its own
A plain chatbot only writes back. An agent is given tools — read a file, search the web, run a command — and works through a goal step by step, deciding what to do next after each result. That is more useful and also riskier, so an agent should be told clearly what it may and may not touch. In the broad sense, an agent is a model plus a harness — LangChain writes it as "Agent = Model + Harness".
Sub-agent
A helper agent that the main agent starts for one part of a job
A sub-agent works in its own fresh context window, so it can read many files or search widely and report back only the result, keeping the main conversation short. Give a sub-agent one narrow job and only the tools it needs — and check its report like any other draft.
Learn more
Ralph Loop
Giving an agent the same prompt again and again until the work is really done
When the agent tries to stop, a hook gives it the same prompt again, and it carries on from the work it has already saved in files and Git. A loop needs a finish line that can be checked and a limit, such as a maximum number of rounds, or it can run for ever.
Learn more
Skill
A reusable set of instructions an AI loads when needed
A skill is a small folder of instructions, and sometimes scripts, for one kind of job. The AI reads it only when that job comes up, so its knowledge can grow without every conversation carrying everything. Writing the same explanation into a prompt over and over is the sign that it should become a skill.
Learn more
Neural network
A web of simple math steps that learns patterns from examples
A neural network is a large stack of very simple calculations wired together. Nobody writes the rules by hand; the network is shown many examples and slowly adjusts its own numbers until its answers match. The name comes from a loose analogy with brain cells, but it is ordinary arithmetic running on a computer, not a thinking creature.
Learn more
Machine learning
Teaching a computer with examples instead of written rules
In ordinary programming you write the rule yourself: if this, do that. In machine learning you instead show the computer thousands of solved examples and let it find the rule on its own. That is why these systems handle messy things like photos and speech well, and why they can also be confidently wrong when the examples were poor.
Learn more
Deep learning
Machine learning with many stacked layers — what powers today's AI
Deep learning is machine learning done with neural networks that have many layers stacked one after another. Each layer picks up something slightly more abstract than the one before, so early layers might notice edges in a photo while later ones recognise a face. Almost every AI you hear about today, including chat models, is built this way.
Learn more
The network design behind almost every modern language model
The transformer is a particular way of arranging a neural network, published in 2017, that lets the model weigh every word in the input against every other word at once. That is what makes long sentences hold together instead of drifting. The T in GPT stands for it, and nearly every chat model you can name is a transformer.
Learn more
Learn more about GPT
Weight
A learned number that sets how strongly one input counts
A weight is a number on a connection inside a neural network. It is multiplied by an input, so it decides how much that input affects the result. Nobody types weights in: training finds them by nudging them until the answers improve. A model file is mostly these numbers, which is why an open model is one whose weights you can download.
Learn more
Prompt engineering
The craft of asking so the AI can actually answer well
Prompt engineering is the practical skill of writing your request so the model has what it needs: who it should be, what the goal is, what the answer should look like, and any facts it cannot guess. It is closer to briefing a new colleague than to programming. The same model can look brilliant or useless depending only on how it was asked.
Learn more
Chatbot
A program you talk to in ordinary sentences
A chatbot is any program you interact with by typing or speaking normal sentences instead of clicking buttons. Older ones followed fixed scripts and broke as soon as you strayed from them; today's are usually a language model behind a chat window. Lyen on this site is one, and like every chatbot it is software running rules, not a person.
Learn more
MCP
A common plug for connecting AI to your tools and data
MCP, short for Model Context Protocol, is an agreed way for an AI assistant to reach outside itself — to read a file, query a database, or call a service. Before it, every tool needed its own custom hook-up. Think of it as the USB port of AI assistants: one shape, so anything built to it simply plugs in. The agent is the MCP client, and a program that offers its own features for agents to use is an MCP server — for example, with WebMCP a website offers its own features to agents through the web browser.
Learn more
MCP server
A program that offers tools and data to AI apps through MCP
An MCP server is a program that offers what it can do — tools to call, data to read, ready-made prompts — through MCP. Any app that speaks MCP can connect to it: Claude Code, Codex, Claude Desktop or a program you wrote. A server can run on your computer and talk through standard input and output, or run on the web and talk over HTTP. The agent runs what the server offers, so read a server before you install it.
Learn more
Probability
A number from 0 to 1 for how likely something is
Probability says how likely something is, from 0 (never) to 1 (certain), and is often written as a percentage. A language model gives every possible next token a probability, and all of them together add up to 1. A high probability only means a word fits the learned pattern well; it does not mean the sentence is true.
Learn more
Language model
A program that calculates which token is likely to come next
A language model is a program that takes some text and calculates a probability for every possible next token. It writes by picking one token, adding it to the end of the text and calculating again. Today's large language models (LLMs) are neural networks trained on huge amounts of text, but a language model can also be tiny — even a single table of word counts, like the bigram on this site.
Learn more
Sampling
Picking the next token by a weighted random draw
When a language model writes, it gives every possible next token a probability. Sampling draws one of them at random, with likelier tokens drawn more often, instead of always taking the top one. That is why the same question can get a different answer — and why sampling can also repeat the same answer. Temperature, top-k and top-p change how the draw is made.
Learn more
Greedy decoding
Always picking the single most likely next token
Greedy decoding skips the random draw: at every step it takes the candidate with the highest probability. The same input then gives the same result each time, which suits facts and code but can sound flat and repetitive. Taking the top word at each step does not guarantee the best sentence overall, and it does not make the answer true.
Learn more
Bigram
A pair of neighbouring words, counted to predict the next one
A bigram is a pair of words that appear side by side. A bigram model counts how often each word follows another in its training sentences, then turns those counts into probabilities. It looks only at the one word just before, so it is a teaching toy, not how modern neural language models work. You can build one yourself on the Build AI screen.
Learn more
Next-token prediction
Guessing which token comes next, again and again, to write text.
The basic way a language model generates text. Given the context so far, it computes a probability for every possible next token, chooses one, adds it to the end and repeats with the longer context until the answer ends.
Learn more
Probability distribution
A list of possible outcomes and how likely each one is, adding up to 100%.
A probability distribution gives every possible outcome a probability so that all of them add up to 1 (100%). A language model produces a distribution over its vocabulary for the next token, and greedy decoding or sampling then picks from it.
Learn more
Autoregressive
Generating one token at a time, each fed back in as input
An autoregressive model writes one token at a time. Each new token is added to the end of the text, and the whole text so far becomes the input for choosing the next one. This loop is why answers appear word by word, and why one early wrong token can pull everything after it in the wrong direction.
Learn more
Context
All the text the model is given for one answer
The context is everything a model receives for one answer: the system prompt, the conversation so far, attached files and any search results the app adds. The model's weights stay the same — a different context is what makes a different answer. The app decides what goes in, and anything left out simply is not there for the model to use. How much can fit is limited by the context window.
Learn more about weights and parameters
Reasoning mode
A setting that spends extra computation before answering
In reasoning mode a model first produces intermediate steps — working, checking, trying other routes — and only then the final answer. This often helps with maths, code and multi-step problems, but it uses more tokens, takes longer and still does not guarantee a correct answer. A ‘Thinking…’ label on screen is a product display, not evidence of human-like thought.
Learn more
Tokenizer
The tool that cuts text into tokens and numbers them
A tokenizer splits text into tokens and replaces each one with an ID number from its own vocabulary. Tokens can be whole words, parts of words, single characters or bytes, and spaces and punctuation count too. Each model family has its own tokenizer, so the same sentence can become a different number of tokens. The AI Lab uses a simple word splitter for teaching, not a real LLM tokenizer.
Learn more
Attention
How a model mixes in information from other tokens
Attention is the step inside a Transformer where each token gives a score to every other token in the context, and those scores decide how much of each one's information is blended in. This is how ‘it’ in a sentence gets linked to the noun it refers to. In text generators the attention is causal: a token can use only earlier positions, never words that have not been written yet.
Learn more
Softmax
Turns raw scores (logits) into probabilities that sum to 1
At each step a model gives every token in its vocabulary a raw score called a logit. Softmax turns that list of scores into probabilities between 0 and 1 that add up to 1, keeping the order: the highest score gets the highest probability. Temperature divides the scores before softmax, which makes the result sharper or flatter.
Learn more
Top-k and top-p
Limits on which candidates sampling may draw from
Before sampling, these settings cut away unlikely candidates. Top-k keeps only the k tokens with the highest probability. Top-p keeps the smallest group of top tokens whose probabilities add up to at least p, so the number kept changes from step to step. Both make odd words rarer, but neither of them checks facts.
Learn more
Gradient descent
Nudging the weights a little in the direction that lowers the loss
Gradient descent is how most models learn. After a round of predictions it calculates the gradient: which way, and how strongly, each weight should move to make the loss smaller. Every weight then takes a small step that way, scaled by the learning rate. Repeating this many times brings the weights close to values that fit the data — on the Hands-on AI development screen, w goes from 0 to about 2.
Learn more
Learn more about weights and parameters
Learning rate
How big a step each weight takes per update
The learning rate multiplies every correction in gradient descent. Too small, and training crawls; too large, and the weights jump back and forth or blow up instead of settling. The model does not learn this value itself — it is a hyperparameter that people choose and test. The Hands-on AI development screen uses 0.01, and 0.05 for the input × w + b task.
Learn more
Learn more about weights and parameters
Loss
A single number for how wrong the predictions are
The loss measures the gap between a model's predictions and the correct answers, boiled down to one number. Training means changing the weights so that this number goes down. The formula is called the loss function: the Hands-on AI development screen uses the average of squared errors, and other tasks use other formulas. A low loss on the training data alone does not prove the model works on new data.
Learn more
Learn more about weights and parameters
Training data
The examples a model learns its weights from
Training data is the collection of examples shown to a model during training — sentences for a language model, labelled photos for an image classifier, input–answer pairs for a small regression. The model can only pick up patterns that are in this data, including its mistakes and gaps. Data set aside for checking (validation and test data) must not be used for training.
Learn more about weights and parameters
Evaluation metric
A score for how well a model does on data it has not seen
Evaluation means measuring how well a trained model performs, using data it did not learn from and a metric — a fixed way of scoring the results. Accuracy, the share of correct answers, is the simplest; other metrics count different kinds of mistakes. A good score on one test set is evidence, not proof: look at the wrong answers too, and keep the test data out of training.
Classification
Predicting which category something belongs to
Classification is a task where the answer is one of a fixed set of categories: spam or not spam, which of three iris species, which digit is in a picture. The model gives each category a score and picks the highest one (argmax). With two possible answers it is binary classification; with three or more, multi-class. When the answer is a number on a scale instead, the task is regression.
Learn more
Regression
Predicting a number on a continuous scale
Regression is a task where the answer is a number rather than a category: a house price, tomorrow's rainfall in millimetres, the result of input × 2. The model is scored by how far its numbers are from the true ones, often with squared error. The small program on the Hands-on AI development screen is a regression; deciding which iris species a flower is would be classification.
Learn more
Overfitting
Fitting the training data so closely that new data goes wrong
A model overfits when it learns the quirks and noise of its training data instead of the general pattern. It then scores very well on the examples it trained on and noticeably worse on new ones. You spot it by comparing the training score with the score on held-out validation or test data; more data, a simpler model or stopping training earlier can help.
Learn more
Hyperparameter
A setting people choose before training, not learned
Hyperparameters are the settings people choose before training starts: the learning rate, how many times to train, the number of layers, the number of neighbours k, the depth of a tree. Training does not change them — it changes the parameters (weights). Good values are found by trying several and comparing them on validation data, never on the final test data.
Learn more
Learn more about weights and parameters
Validation data
Held-out data for choosing settings, separate from the test set
Data is usually split three ways: training data to learn from, validation data to compare settings such as hyperparameters, and test data used once at the end for the final score. Cross-validation splits the training data into several parts and takes turns holding one out, which gives a steadier comparison when data is scarce. Tuning while watching the test score makes that score look better than it really is.
Learn more
Activation function
The function a neuron applies after adding up its inputs
A neuron in a neural network first combines its inputs as z = w·x + b, then passes z through an activation function. Sigmoid squeezes any number into the range 0 to 1; ReLU turns negative numbers into 0 and keeps positive ones. Without such a non-straight step, stacking many layers would still add up to one straight-line calculation, and the network could not learn curved patterns.
Learn more
Runner (inference engine)
The program that loads a model and makes it write text
A trained model is only a file of numbers. A runner loads those numbers, turns your text into tokens, runs the model to score the next token, picks one and repeats, showing the words as they come. Ollama and llama.cpp are runners you can use on your own computer.
Learn more
KV cache
Saved attention results so earlier tokens are not recomputed
In attention, every token produces a key and a value. During autoregressive generation the earlier tokens do not change, so their keys and values are kept in memory — the KV cache — and reused for each new token instead of being recalculated. This makes generation much faster, but the cache grows with the length of the context and can take up a lot of GPU memory.
Learn more
Learn more about weights and parameters
A structured request from the model for the app to run a tool
A model can only produce text, so it cannot act on its own. In a tool call it outputs a structured request — the tool's name and its arguments, usually as JSON — and the app checks it, asks for permission if needed and runs the tool. The result goes back to the model as more context. Repeating this loop is what people call an agent.
Learn more
Multimodal
Handling more than one kind of data, such as text and images
A multimodal model takes in or produces more than one kind of data — text, images, audio or video. For example, a model can receive a photo and answer a question about it in text. Every kind of input is turned into numbers the model can calculate with. Not every feature that looks multimodal is one model: an app may first turn speech into text with a separate speech-recognition model.
Learn more
Vibe coding
Describing what you want and letting AI write the code
Vibe coding means describing in plain words what you want to make — how it should look and behave — and letting an AI tool write the code, then checking the result and asking for changes. It is a quick way to try ideas and build small web apps or games. You still have to test what comes out, because generated code can contain mistakes or security problems. On BaeumNova's Vibe Coding screen, Lyen builds the app and shows a preview.
Learn more
Query, key and value
The three vectors attention uses to look things up
In attention, every position makes three vectors from its token. The query says what this position is looking for, each key says what an earlier position contains, and each value is the information that position passes on. The model compares the query with every key, turns the scores into weights that add up to 1, and mixes the values with those weights.
Learn more
BPE
A tokenizer method that keeps merging the most frequent pair
BPE (byte pair encoding) builds a tokenizer from training text. It starts with single characters or bytes, finds the pair of neighbouring pieces that appears most often, merges it into a new piece, and repeats until the vocabulary is big enough. Common words end up as one token, while rare words are split into several pieces, so any text can still be written.
Learn more
Learn more about GPT
Byte-level BPE
BPE that starts from bytes instead of characters
Byte-level BPE starts from the 256 possible UTF-8 byte values instead of from characters. So the first vocabulary is always 256, whatever the language. Every character and emoji is stored as bytes, so no text is ever out of vocabulary. GPT-2 and many later tokenizers work this way.
Learn more
Learn more about GPT
OOV (out of vocabulary)
A word the tokenizer does not have in its vocabulary
OOV means out of vocabulary: a word that is not in the tokenizer's list. A word-level tokenizer has to replace such a word with an "unknown" mark, so its meaning is lost. BPE cuts a rare word into smaller pieces it does know, and byte-level BPE never meets an OOV word at all.
Learn more
Corpus
A collection of text used for training
A corpus is a large collection of text gathered to train a tokenizer or a language model. A tokenizer counts the pairs in its corpus to decide what to merge, so the corpus shapes the vocabulary. Train on Korean text and Korean pieces become tokens.
Learn more
Merge rules
The ordered list of pairs a BPE tokenizer joins
While BPE learns, it writes down every pair it merged, in order. That list is the merge rules. To tokenize new text, the tokenizer applies the same rules in the same order. The rules are saved with the vocabulary; if you change them, the token numbers change too, so the model weights must be trained again.
Learn more
Learn more about weights and parameters
Learn more about BPE
Chat history
The earlier messages sent to the model with each new question
A language model does not keep anything between requests. A chat app seems to remember because it sends the earlier messages again together with the new question. That list is the chat history. It has to fit in the model's context length, so long chats are cut or summarised. Without it, every question is answered on its own.
Learn more
Computation graph
A record of which calculation used which numbers
While a model computes its answer, a training library records every step: which numbers went into which addition or multiplication and what came out. That chain of steps is the computation graph. Backpropagation walks it backwards from the loss and works out, step by step, how much each number affected the result.
Learn more
Decoder
A Transformer that writes the next token from earlier ones
A decoder is a Transformer that sees only the tokens before the current position and predicts the next one. It then adds that token and repeats, which is how GPT-style models write text. The original Transformer also had an encoder that reads a whole input at once, but most chat models today are decoder-only.
Learn more
Learn more about GPT
Diffusion model
A model that makes pictures by removing noise step by step
A diffusion model starts from an image of random noise and removes a little noise at each step, guided by your text, until a picture appears. It is trained by adding noise to real pictures and learning to undo it. Many image and video generators, such as Stable Diffusion, work this way, unlike language models that write one token at a time.
Learn more
encode / decode
Turning text into token numbers, and numbers back into text
encode is the tokenizer step that cuts text into tokens and replaces each with its number from the vocabulary. decode does the reverse: it looks up each number and joins the pieces back into text. A good tokenizer gives back exactly the original text after encode and then decode, which is a simple test worth running.
Learn more
Execution provider
The ONNX Runtime plug-in that runs a model on a given chip
ONNX Runtime does not talk to every chip by itself. An execution provider is the part that hands the model's calculations to one kind of hardware, such as an NVIDIA GPU (CUDA or TensorRT), an Apple chip (Core ML) or plain CPU. You list the providers you want in order, and parts a provider cannot run fall back to the next one.
Learn more
FreeToken
A runtime for running very large MoE models on a personal PC
FreeToken is a runtime introduced on the /ai/internals page. It is built to run very large MoE models, which use only some of their experts for each token, on a personal computer. It treats the NVIDIA GPU, the CPU, main memory (RAM) and the links between them as one inference machine.
Learn more
Gradient clipping
Shrinking the gradient when it gets too large
Sometimes one batch produces a huge gradient, and one update throws the weights far off so that the loss jumps or becomes nan. Gradient clipping measures the size of the whole gradient and, if it is above a set limit, scales it down to that limit before the update. The direction stays the same; only the step gets shorter.
Learn more
Learn more about weights and parameters
Normalization
Rescaling numbers inside a network so training stays stable
As numbers pass through many layers they can grow or shrink until training breaks. Normalization rescales them at each step. Transformers use layer normalization: for each token it shifts the numbers to an average of 0 and a spread of 1, then multiplies and adds two learned sets of numbers, which are also parameters.
Learn more
Learn more about weights and parameters
MLP
A basic neural network of layers stacked one after another
An MLP (multilayer perceptron) passes numbers through two or more layers. Each layer multiplies by weights, adds a bias and applies an activation function such as ReLU or GELU. Inside every Transformer block there is a small two-layer MLP that works on each token separately and changes its features after attention has mixed in information from other tokens.
Learn more
Learn more about GPT
Learn more about weights and parameters
Momentum
Carrying part of the last update into the next one
Plain SGD moves the weights only by the current gradient, so it can zigzag. Momentum keeps a running sum of past updates and adds part of it to each new step, so the weights keep rolling in a steady direction and small bumps matter less. That running sum is extra state the optimizer must save in a checkpoint; plain SGD has none.
Learn more
Learn more about weights and parameters
Negative log
−log(p): a number that grows as a probability gets smaller
Taking the log of a probability and putting a minus in front gives a number that is 0 when the probability is 1 and grows quickly as the probability drops toward 0. Training uses it as the loss for the correct token: a confident right answer costs almost nothing, a confident wrong one costs a lot.
Learn more
ONNX
A shared file format for AI models
ONNX (Open Neural Network Exchange) is an open file format for models. A model trained in PyTorch or another framework can be exported to an .onnx file and then run by different programs on different devices, without the original training code. Microsoft and Facebook started it in 2017.
Learn more
ONNX Runtime
An engine that runs ONNX models on many devices
ONNX Runtime is Microsoft's open-source engine for running .onnx models. It works on Windows, macOS, Linux, phones and in the browser, and uses execution providers to reach GPUs and other accelerators. On BaeumNova it runs the voice model that reads text aloud.
Learn more
Output layer
The last layer: one score for each vocabulary token
The output layer turns the final vector of a position into one score per token in the vocabulary. Softmax then turns those scores into next-token probabilities. Because it needs one row of weights per token, a large vocabulary makes this layer large.
Learn more
Learn more about weights and parameters
3B, 7B, 8B (parameter count)
A model size: B means billion parameters, not bytes
Names like 3B, 7B or 70B give the number of parameters: 7B means about 7 billion single numbers. It is not the file size and not the number of vectors. The size in memory depends on how many bytes each number uses, so the same 8B model is about 16 GB in 16 bits and about 4 GB in 4 bits, before extra overhead.
Learn more
Quantization scale
The multiplier that turns stored integers back into values
A quantized file stores weights as small integers. To use them, the runtime multiplies each integer by a scale saved next to it. Formats like GGUF keep one scale for each small block of weights, such as 32 numbers, so the file holds these scales and other metadata on top of the integers. That is why a real file is a little larger than the simple estimate.
Learn more
Learn more about weights and parameters
Residual connection
Adding a layer's input back to its output
A residual connection adds what went into a layer to what came out of it: output = input + layer(input). The original information is kept, and the layer only has to learn what to change. It also gives gradients a short path back through many layers, which is why deep Transformers can be trained at all.
Learn more
RoPE
Telling attention a token's position by rotating its vectors
RoPE (rotary position embedding) gives position information by rotating each query and key vector by an angle that depends on the token's position. When attention compares two tokens, the result then depends on how far apart they are. It was proposed in 2021 and is used by many open models, including Llama.
Learn more
SGD
Nudging weights against each small batch's gradient
SGD (stochastic gradient descent) is the simplest training rule: compute the gradient of the loss on a small random batch, then move every weight a small step against it, the step size being the learning rate. Repeat with the next batch. Optimizers such as Adam add extra state, like momentum, on top of this idea; plain SGD has nothing extra to save.
Learn more
Learn more about weights and parameters
sherpa-onnx
An open-source toolkit for offline speech recognition and TTS
sherpa-onnx is an open-source project from the Next-gen Kaldi team that runs speech models with ONNX Runtime. It can turn speech into text and text into speech without an internet connection, on computers, phones and small boards. On BaeumNova it is the engine that turns your voice into text.
Learn more
SLM
A small language model that can run on ordinary devices
An SLM (small language model) works like an LLM but has far fewer parameters, so it can run on a laptop or phone, answer faster and cost less. There is no fixed cut-off; models with a few billion parameters or less are usually called small. They are weaker at broad knowledge and long reasoning, but good for narrow tasks.
Learn more
Learn more about weights and parameters
Speech-to-text
Turning spoken words into written text
Speech-to-text (STT), also called speech recognition, turns the sound of speech into letters. A model listens to short slices of sound and writes the most likely words. The text can then be sent to a language model. Some newer models can take sound directly without writing it down first.
Learn more
TensorRT-LLM
NVIDIA's library for running LLMs fast on its GPUs
TensorRT-LLM is an open-source library from NVIDIA that optimizes large language models for NVIDIA GPUs. It compiles the model for the exact GPU and adds tricks such as batching many requests and quantization, so servers can answer more users at once. It only targets NVIDIA hardware.
Learn more
Text-to-speech
Turning written text into a spoken voice
Text-to-speech (TTS), or speech synthesis, reads text aloud. A model turns the letters into sound, choosing pronunciation, rhythm and tone. On BaeumNova, Lyen's voice is made this way with a model run by ONNX Runtime.
Learn more
Training step
One update of the weights during training
A training step is one round of: take a batch, run the model forward, compute the loss, run backpropagation and update the weights. Training logs usually print the step number with the loss, so you can watch the loss fall as steps go by. One pass over the whole dataset is a different unit, called an epoch.
Learn more
Learn more about weights and parameters
vLLM
An open-source server engine for serving LLMs fast
vLLM is an open-source engine, started at UC Berkeley, for serving language models to many users. Its PagedAttention method stores the KV cache in small pages, like a computer's memory, so little GPU memory is wasted and many requests can be batched together. It offers an API in the same shape as OpenAI's.
Learn more
VLM
A language model that also reads images
A VLM (vision language model) takes pictures as well as text and answers in text. An image part turns the picture into vectors the language model can read alongside the tokens. You can ask it what is in a photo, read a chart or a screenshot. It still writes text; making pictures is the job of other models such as diffusion models.
Learn more
Attention head
One complete set of attention calculations
An attention head has its own query, key and value weights and computes attention once. A layer usually has several heads side by side, so each can learn to look for a different kind of link, such as the previous word or the subject of the sentence. Their results are joined and passed on. Seeing where one head looks is a useful clue, not a full explanation of the model.
Learn more
Learn more about GPT
Learn more about weights and parameters
llama.cpp
A C/C++ runner for language models on your own computer
llama.cpp is an open-source program, started by Georgi Gerganov, that runs language models in plain C/C++ on CPUs and GPUs, including Apple chips, without Python. It reads GGUF files and comes with tools to convert and quantize models. Many other apps, including Ollama and phone apps, are built on it.
Learn more
Model checkpoint
A saved copy of a model's weights during or after training
A checkpoint is a file that saves the weights, and often the settings, optimizer state and step number, at one moment of training. If training stops, you can continue from it; when training ends, the last checkpoint is the model you run. A model checkpoint is different from a Claude Code checkpoint, which rewinds file edits.
Learn more
Learn more about GPT
Learn more about weights and parameters
Positional embedding
A vector that tells the model where a token sits in the text
Attention by itself does not know the order of tokens, so "dog bites man" and "man bites dog" would look the same. A positional embedding adds a vector for each position to the token's own vector. GPT-2 learns one vector per position, up to 1,024 positions; other models use different methods such as RoPE.
Learn more
Learn more about GPT
One repeated layer of a Transformer: attention plus an MLP
A Transformer block is the unit that is stacked again and again. Inside, attention mixes information between tokens, then a small MLP changes each token's features, with normalization and residual additions around both. GPT-2 small stacks 12 identical blocks; bigger models stack more and make each one wider.
Learn more
Learn more about GPT
Backpropagation
Tracing the error back to find each weight's gradient
After the forward pass gives a loss, backpropagation walks the computation graph backwards and uses the chain rule to find the gradient of the loss for every weight: how much the loss would change if that weight changed a little. Gradient descent then uses those gradients to update the weights. In PyTorch one call, loss.backward(), does it.
Learn more
Learn more about weights and parameters
PyTorch
The leading Python library for neural networks
PyTorch is an open-source deep learning library for Python, first released by Meta (then Facebook) and now run by the PyTorch Foundation under the Linux Foundation. It gives tensors that can live on a GPU and automatic differentiation, so loss.backward() computes all gradients for you. It is imported as torch.
Learn more
GPT
A Transformer language model that predicts the next token
GPT stands for Generative Pre-trained Transformer. It is a decoder-only Transformer trained on a large amount of text to predict the next token, then used to write text one token at a time. OpenAI named its models this way, from GPT-1 in 2018 to today's GPT models behind ChatGPT, and the same design is used by many other models.
Learn more
Vocabulary
The list of token pieces a tokenizer knows, each with an ID
A tokenizer's vocabulary is the fixed list of pieces it can produce, each with a number (ID). The model has one embedding row and one output score per entry, so the vocabulary size shapes the model. A model only works with the vocabulary it was trained with: swap in another and the numbers mean different things.
Learn more
Learn more about GPT
Cross-entropy
The usual next-token loss: −log of the right token's probability
Cross-entropy loss looks only at the probability the model gave to the correct answer and takes its negative log. If the right token got probability 1, the loss is 0; the lower that probability, the larger the loss. Averaged over many tokens, it is the number language-model training tries to push down. A low loss means good guesses on this data, not that the text is true.
Learn more
Derivative
How much an output changes when an input changes a little
A derivative measures how fast one value changes when another changes by a tiny amount. In training, the derivative of the loss with respect to a weight says which way and how strongly to move that weight. A gradient is just the derivatives for all weights together. You do not have to compute them by hand: libraries do it automatically.
Learn more
Learn more about weights and parameters
Dropout
Switching off random parts of a network during training
Dropout sets a random share of a layer's outputs to zero at each training step, for example 10%. The network cannot lean on any single unit, which helps against overfitting. At inference dropout is turned off and every unit is used. A rate set too high makes training slow and the model weak.
Learn more
Forward pass
Running the input through every layer to the output
The forward pass is the model doing its normal calculation: the input goes through every layer in order and comes out as scores or a prediction. In training, the loss is computed from that output and backpropagation then runs the other way. At inference only the forward pass is needed.
Learn more
Random seed
A starting number that makes "random" results repeatable
Computers make random numbers from a starting value called the seed. The same seed gives the same sequence, so training, data shuffling or sampling can be repeated exactly for checking. A seed only makes a run repeatable; it does not make an answer correct. Some GPU operations can still differ slightly between runs.
Learn more
OCR
Reading the letters in a picture or scanned page
OCR (optical character recognition) turns letters inside an image, such as a photo or a scanned PDF, into real text a computer can search and copy. A scanned PDF is only pictures of pages until OCR reads them. Results can contain mistakes, especially with handwriting or blurry images, so important text should be checked.
Learn more
Adam
A widely used optimizer that adapts the step for each weight
Adam is an optimizer published in 2014 and now the default choice for training most neural networks, including Transformers. For every weight it keeps a running average of recent gradients (like momentum) and of their size, and uses them to choose a suitable step for that weight. It usually trains faster and more smoothly than plain SGD, at the cost of storing that extra state.
Learn more
Learn more about weights and parameters
Adapter
A small set of extra weights trained on top of a frozen model
Instead of changing all of a model's weights in fine-tuning, you can freeze them and train a small adapter that is added alongside. LoRA is the best-known kind: it learns two thin matrices whose product adjusts a weight matrix. The adapter file is tiny compared with the model, and you can keep several adapters for different tasks on the same base model.
Learn more
Learn more about weights and parameters
Learn more about fine-tuning and LoRA
Agent memory
What an agent saves and reloads about earlier work and rules
A language model forgets everything between sessions. Agent memory is the part of the harness that writes down useful facts, rules and progress, usually in files, and loads them back into the context next time. Claude Code, for example, reads CLAUDE.md and its auto memory at the start of each session. This memory is text in files, not the computer's RAM.
Learn more
Agentic system
A language model joined with tools to do work
An agentic system is any setup that joins a language model with tools, files and a loop so that it can carry out tasks, not just answer once. Anthropic uses the term for both fixed workflows, where code decides the steps, and agents, where the model decides its own steps. The phrase is used when talking about how such a system is designed.
Learn more
argmax
The position of the largest value, not the value itself
max returns the largest number in a list; argmax returns where it is. A classifier uses argmax to pick the class with the highest score, and greedy decoding uses it to pick the most likely next token. Positions are usually counted from 0.
Learn more
Baseline model
A very simple model that sets the score to beat
Before trusting a clever model, compare it with a simple baseline, such as always guessing the most common class or using an average. If your model barely beats the baseline, it has learned little. Reporting the baseline next to the score shows what the number really means.
Learn more
Batch
A group of examples processed together in one step
Training does not look at one example at a time. It takes a batch, for example 12 chunks of text, computes the loss for all of them together and makes one update. Larger batches use the GPU better but need more memory; if training runs out of memory, lowering the batch size is the first thing to try.
Learn more
Learn more about GPT
Block size
How many tokens the model reads at once — its context length
In nanoGPT and similar code, block size is the maximum number of tokens in one training example, which is also the longest context the model can use. Memory for attention grows quickly with it, so a smaller block size is a common fix when training runs out of memory. A model cannot look further back than its block size.
Learn more
Learn more about GPT
BM25
A classic search formula that ranks by matching words
BM25 scores how well a document matches a search by the words they share. A rare word counts more than a common one, repeating a word helps less and less (saturation), and long documents are adjusted so they do not win just by being long. It is fast and good at exact names and codes; embedding search is better at the same meaning in different words, and many systems use both.
Learn more
Calibration
Whether a model's scores match how often it is really right
A model is well calibrated when its scores can be read as real chances: among all answers it gave 0.8, about 80% are correct. Many models output numbers between 0 and 1 that look like percentages but are not calibrated, so check them on held-out data before treating a score as a probability.
Learn more
Causal mask
Hiding later tokens so attention can only look back
During training the model sees a whole sentence at once, but it must predict each token using only the ones before it. The causal mask blocks every position from attending to positions after it, by setting those scores to minus infinity before softmax. Without it the model would copy the answer from the future and learn nothing useful.
Learn more
Character-level model
A language model whose tokens are single characters
A character-level model treats each letter, space and punctuation mark as one token, so its vocabulary is tiny and no tokenizer has to be learned. It is easy for practice, such as nanoGPT's Shakespeare example, but needs many more steps per sentence than models with word-piece tokens. Its weights only fit its own character vocabulary, so they cannot be swapped into a runner that expects GPT-2 tokens.
Learn more
Learn more about GPT
Learn more about weights and parameters
Chat template
The exact format a chat model expects messages in
A chat model was trained on conversations written in one fixed format, with special tokens marking where the system, user and assistant parts start and end. The chat template turns your list of messages into that text. Runners such as Ollama and llama.cpp read it from the model files. With the wrong template, a model can ramble, repeat itself or answer strangely.
Learn more
Saved instructions
Preferences ChatGPT applies to every new chat
In ChatGPT's settings you can save instructions about yourself and how you want answers, for example "I am a beginner, use short sentences". ChatGPT adds them to every new conversation, so you do not have to repeat them. They are still only instructions: they steer answers but do not guarantee them, and you should not put secrets in them.
Learn more
Learn more about GPT
Closed model
A model you can use only through its company's app or API
A closed model keeps its weights private. You use it through the company's app or API, and the company decides the price, the limits and when old versions disappear. Claude, Gemini and the GPT models behind ChatGPT are closed models. They are often the strongest, but your data travels to the company's servers.
Learn more
Learn more about GPT
Learn more about weights and parameters
Completion promise
A phrase the agent may say only when the job is truly done
In a Ralph Loop you choose a completion promise, a short phrase such as DONE. The loop keeps going until the agent writes exactly that phrase, and the prompt tells the agent to write it only when it is really true. It works with a maximum number of rounds and a budget, because an agent may never reach the finish, so check the work yourself afterwards.
Learn more
Context management
Choosing what goes into the model's limited context
A model can read only so much at once. Context management is the harness's job of deciding what to put in: which files, which earlier messages, which tool results, and what to summarise or drop when the conversation gets long. Good context management keeps the important facts in view and leaves out noise, which matters more as tasks get longer.
Learn more
Core ML
Apple's framework for running models on iPhone, iPad and Mac
Core ML is Apple's built-in framework for running machine-learning models inside apps on Apple devices. It can use the CPU, the GPU and the Neural Engine chip, and runs on the device without sending data to a server. Models from PyTorch and other tools are converted to Core ML format first, for example with Apple's coremltools.
Learn more
Cumulative probability
A running total of probabilities, added from the largest down
Sort the candidates from most to least likely and keep adding their probabilities: that running total is the cumulative probability. Top-p sampling uses it: with p = 0.9 it keeps the smallest group of candidates whose total reaches 90% and draws only from them, so the number of candidates changes with how sure the model is.
Learn more
Decision tree
A model that classifies by asking yes/no questions in a row
A decision tree learns a chain of questions about the features, such as "petal length < 2.5 cm?", and each answer leads to the next question until a leaf gives the class. It is easy to read. Left to grow without a depth limit, it can memorise the training data perfectly and do worse on new data, which is overfitting.
Learn more
Embedding model
A model that turns text into a vector for searching by meaning
An embedding model does not write text. It reads a sentence or a document and outputs one vector, so that texts with similar meaning get vectors that point in similar directions. Search, recommendation and RAG systems use these vectors to find related text. A small one can even run on a phone to search your own notes.
Learn more
Embedding search
Finding text by close meaning vectors, not shared words
Embedding search turns the question and every document into vectors with the same embedding model and returns the documents whose vectors are closest, often by cosine similarity. It finds "car repair" when you search "fixing my automobile", which word matching misses. Close vectors mean related meaning, not that a document is true or answers the question.
Learn more
Embedding space
The many-dimensional space where embedding vectors sit
Every embedding vector is a point in a space with hundreds of dimensions. In that space, things the model treats as similar end up close together. Distances only make sense between vectors from the same model: two different embedding models build different spaces, so their vectors cannot be compared.
Learn more
Feed-forward layer
The part of a Transformer block that works on each token alone
After attention mixes information between tokens, the feed-forward layer processes each token's vector separately with the same small network: usually a wider layer, an activation function such as GELU, and a layer back to the original width. It is the block's MLP, and in large models it holds most of the parameters.
Learn more
Learn more about GPT
Learn more about weights and parameters
GELU
A smooth activation function used in many Transformers
GELU (Gaussian Error Linear Unit) is an activation function like ReLU, but with a smooth bend instead of a sharp corner at zero: large positive inputs pass almost unchanged, large negative ones become almost 0, and values near zero are softly reduced. GPT-2 and BERT use it in their feed-forward layers.
Learn more
Learn more about GPT
Generalization
Doing well on new data the model has never seen
The point of training is not to score well on the training examples but to do well on new ones. That ability is generalization. You measure it on data held back from training. A model that is perfect on its training data but poor on new data has memorised instead of generalised, which is overfitting.
Learn more
GPT-2
OpenAI's 2019 Transformer model, a classic for learning
GPT-2 is a decoder-only Transformer language model that OpenAI released in 2019, in sizes from 124 million to 1.5 billion parameters. Its design — token and position embeddings, stacked Transformer blocks, an output layer over a 50,257-token vocabulary — is simple and well documented, so projects such as nanoGPT rebuild it to teach how LLMs work.
Learn more
Learn more about GPT
Learn more about weights and parameters
Handoff
One agent passing a task, with its context, to another agent
In a handoff, an agent hands the rest of a task to another agent that is better suited, along with what it needs to know. The harness decides how much context goes with it, which model takes over and how the result comes back. Too little context and the next agent repeats work; too much and it gets lost.
Learn more
Hidden layer
A layer between a network's input and its output
A hidden layer sits between the input and the output layer. It takes numbers in, multiplies them by weights, applies an activation function and passes new numbers on, building features that are not directly visible in the input or the answer. Deep learning simply means having many hidden layers.
Learn more
Learn more about weights and parameters
Imagen
Google's earlier family of text-to-image models
Imagen is a family of Google models that turn a text description into an image. As of October 2026, the BaeumNova course notes that the older Imagen models in the Gemini API have been retired and Google's image generation and editing there is offered through the Nano Banana models.
Learn more
Instruction following
How faithfully a model does what it was told
Instruction following is how well a model obeys the instructions in a prompt or system prompt: the format, the length, the language, the things it must not do. Models differ a lot here and are tested on it. Even a strong model can drift, so an app must not rely on instructions alone for safety; real limits such as permissions have to be enforced by code.
Learn more
k-nearest neighbours (k-NN)
Classifying by the k most similar training examples
k-NN has almost no training: it keeps all training examples. To classify a new one, it finds the k closest examples and takes the most common class among them. The neighbour count k is a hyperparameter you choose outside training: small k follows noise, large k blurs differences.
Learn more
LangChain
An open-source framework for LLM apps and agents
LangChain is an open-source framework, available for Python and JavaScript, for building applications on top of language models: connecting a model to tools, documents, memory and other models. Its companion LangGraph is used to build agents with explicit steps. LangChain describes the harness as everything around the model, which the BaeumNova course quotes.
Learn more
LiteRT
Google's runtime for models on phones and small devices
LiteRT is Google's on-device runtime for machine-learning models, the new name since 2024 for TensorFlow Lite. It runs models on Android, iOS, the web and small boards, using the phone's GPU or other accelerators when available, without sending data to a server.
Learn more
llama.rn
A library that puts llama.cpp inside React Native apps
llama.rn is an open-source React Native library that wraps llama.cpp, so a phone app written in React Native can load a GGUF model and generate text on the device, without a server. PocketPal AI is one app built this way. Flutter apps use other llama.cpp bindings for the same job.
Learn more
Main agent
The agent you talk to, which hands parts to sub-agents
The main agent is the one in your conversation. For a part of the job, such as searching a whole project, it can start a sub-agent with its own fresh context and fewer tools, then receive only the summary. That keeps the main agent's context clean. The main agent stays responsible for the final answer.
Learn more
Maximum number of rounds
The limit on how many times a loop may run
An agent loop may never reach its finish line by itself, and every round costs tokens and money. The maximum number of rounds is a hard stop: after that many rounds the loop ends no matter what. Set it every time, together with a checkable finish line and a token budget.
Learn more
MCP host
The AI app the person uses, which holds the MCP clients
In MCP there are three roles. The host is the AI app a person uses, such as Claude Code, Claude Desktop or a code editor. Inside it, one client per server keeps the connection, and each server offers tools and data. The host decides which servers to start and should ask the user before a tool is used.
Learn more
initialize
The first MCP request that agrees on a version (spec 2025-11-25 and earlier)
In MCP specs up to 2025-11-25, every session starts with initialize. The client sends the protocol version it speaks and what it can do; the server answers with the version it will use, what it offers (for example tools) and its name. The client then sends the notifications/initialized notice, and only after that can tools/list and tools/call be used. The latest spec (2026-07-28) drops this handshake: every request carries its version in _meta.
Learn more
The MCP request that asks a server to run one of its tools
tools/call names a tool and gives its arguments. The server runs the tool and replies with the result, usually as text content, or with an error. The model decides which tool to use, but the host app sends the request and should ask the user before running anything risky.
Learn more
The MCP request that asks a server which tools it has
tools/list asks an MCP server what it can do. The server returns each tool's name, a description the model reads to decide when to use it, and an input schema describing the arguments. Because the model trusts these descriptions, only connect to servers you trust.
Learn more
Nano Banana
Google's Gemini image generation and editing models
Nano Banana is the popular name Google uses for the Gemini models that create and edit images from text. You can describe a new picture or upload one and ask for changes, such as a different background, while keeping the same person or object. It is available in the Gemini app and through the Gemini API.
Learn more
nanochat
Karpathy's full-stack project for a small chat LLM
nanochat is an open-source project by Andrej Karpathy, released in 2025 as the successor to nanoGPT. It covers the whole path in one small codebase: training a tokenizer, pre-training, fine-tuning for chat, evaluation, and a web page to talk to the result. It is meant for learning how a ChatGPT-like model is built end to end.
Learn more
Learn more about GPT
Learn more about fine-tuning and LoRA
nanoGPT
Short PyTorch code for training GPT-2-style models
nanoGPT is Andrej Karpathy's open-source code for training and fine-tuning GPT-style models, in a few hundred lines of PyTorch. It can reproduce GPT-2 on big GPUs or train a tiny character-level Shakespeare model on a laptop CPU in minutes. Its author now marks it as old and points to nanochat, but it is still a clear way to see a GPT-2-style model.
Learn more
Learn more about GPT
Learn more about fine-tuning and LoRA
Ollama
An app that downloads and runs open models on your computer
Ollama is a free runner for macOS, Windows and Linux. One command downloads an open model and starts chatting with it, and Ollama also serves a local API that your own programs can call. It handles quantized model files and chat templates for you, so it is an easy first way to run a model offline.
Learn more
Open model
A model whose weights you can download and run yourself
An open model publishes its weights, so you can download it and run it on your own computer or server, often for free and without sending data away. Many are open-weight rather than fully open source: the weights are shared, but not always the training data or code, and the licence may set limits. Llama, Gemma and Qwen are examples.
Learn more
Optimizer
The rule that turns gradients into weight updates
Backpropagation tells you the gradient for each weight; the optimizer decides how to change the weights with it. Plain SGD takes a fixed step against the gradient. Adam and others also keep a memory of past gradients to choose better steps. The optimizer's state is saved in checkpoints so training can resume.
Learn more
Learn more about weights and parameters
PocketPal AI
A free phone app for chatting with small models offline
PocketPal AI is an open-source app for iPhone and Android that downloads small language models in GGUF format and runs them on the phone, so you can chat without an internet connection. It is built with llama.rn on top of llama.cpp. Phone memory is small, so it works best with small, strongly quantized models.
Learn more
Post-training
Training after pre-training to teach specific behaviour
Pre-training teaches a model to continue text from huge amounts of data. Post-training comes after: fine-tuning on example conversations, learning from human or automated feedback, and practising tasks such as using tools. This is where a raw text predictor becomes a helpful assistant, and where coding agents learn to work well with their harness.
Learn more
Learn more about fine-tuning and LoRA
Q4_K_M
A common 4-bit quantization type for GGUF files
Q4_K_M is one of llama.cpp's "k-quant" types. It stores most weights in about 4 bits, keeps some more sensitive parts in more bits (the M stands for medium), and groups weights into blocks that share scales. It is a popular balance between file size and quality, at roughly 30% of the FP16 size.
Learn more
Learn more about weights and parameters
Recommender system
AI that predicts what you will like and suggests it
A recommender system predicts which videos, songs or products a person is likely to want, from what they and similar people did before, and shows those first. It is one of the most widely used kinds of AI, even though it does not chat or write text. It is a good reminder that AI is much more than ChatGPT.
Learn more
Learn more about GPT
Regression coefficient
The learned number a regression model multiplies each input by
A linear regression predicts with a formula such as price = a × size + b. The numbers a and b are found from the data during training: they are the coefficients, and they are parameters, unlike settings you choose yourself such as tree depth. A coefficient shows how much the prediction moves when that input grows by one.
Learn more
Learn more about weights and parameters
Rule-based system
A program that decides only by rules a person wrote
A rule-based system follows if-then rules written by people, such as "if the subject contains 'free money', mark it as spam". It does not learn from data, so it is easy to explain but breaks on cases nobody thought of. Machine learning instead finds the rules from examples. Both count as AI.
Learn more
Similarity
A number for how alike two vectors or texts are
Similarity measures how close two vectors are. Cosine similarity, the most common, compares their directions: 1 means the same direction, 0 unrelated, −1 opposite. Embedding search ranks documents by it. High similarity means related meaning, not that two sentences say the same true thing.
Learn more
scikit-learn fit
The scikit-learn method that trains a model on data
In scikit-learn every model is trained the same way: model.fit(X, y) learns from the inputs X and the answers y. After that, model.predict(X_new) gives predictions. Fit only on the training split and keep the test split for the end, otherwise the score looks better than it is.
Learn more
Stop condition
A checkable rule that says when an agent's job is done
A stop condition tells an agent when to stop, for example "all tests in tests/ pass and the page opens without errors". The agent checks it after each step and keeps working until it is true. A vague wish like "make it better" never ends, so a long-running loop also needs a limit on rounds or tokens.
Learn more
Stop token
A special token that means "the text ends here"
A stop token is a special token that marks the end of a text. During training the model sees it at the end of every example, so it learns where an answer finishes. While generating, the program stops as soon as this token is picked or a length limit is reached. Without it, the model would keep adding words.
Learn more
Learn more about GPT
Special token
A reserved token that marks a place instead of text
A special token is a token the tokenizer reserves for a job instead of a piece of text. [BOS] marks the beginning of a text and [EOS] the end — generation stops when the model produces it. [PAD] fills empty space and [UNK] stands for a symbol the tokenizer does not know. XSLM uses a newline the way real models use EOS.
Learn more
safetensors
A safe file format for storing model weights
safetensors is a file format for saving a model's weights. The file holds only numbers plus a small JSON header that lists each layer's name, data type and shape. Older formats based on Python pickle could run hidden code when the file was opened; safetensors cannot, which is why Hugging Face uses it by default.
Learn more
Learn more about weights and parameters
Pre-training
The first and largest stage of training a language model
Pre-training is the first stage of building a language model: the model learns to predict the next token over a huge amount of text. This gives it a general grasp of language, but it does not yet follow instructions well. Later stages such as instruction tuning shape it into an assistant.
Learn more
Instruction tuning (SFT)
Extra training on request-and-answer pairs to follow instructions
Instruction tuning, also called SFT (supervised fine-tuning), trains a pre-trained model further on pairs of a request and a good answer. This teaches the model to answer the way people ask, instead of just continuing the text. It is one part of post-training.
Learn more
Learn more about fine-tuning and LoRA
Llama
A family of open-weight language models from Meta
Llama is a family of language models made by Meta. Meta publishes the model weights, so anyone can download them and run them on their own computer under Meta's license terms. Tools such as llama.cpp and Ollama started out by running Llama models.
Learn more about weights and parameters
Qwen
A family of language models from Alibaba
Qwen is a family of language models made by Alibaba Cloud. Many Qwen models are released with open weights, from small sizes for laptops to large ones for servers, including coding models such as Qwen3-Coder. They can also be used through services such as OpenRouter.
Learn more about weights and parameters
Term frequency saturation
In keyword search, extra repeats of a word add less and less to the score
In BM25 keyword search, a word that appears often in a document raises the document's score, but each extra appearance adds less than the one before. The score levels off instead of growing without end. This stops a document from winning just by repeating one word many times.
Learn more
Threshold
The cut-off number above which a model's answer counts as "yes"
A classifier often gives a score, not a final answer. The threshold is the line you draw: a score above it means "yes", below it means "no". Moving the threshold changes which examples are called positive — raise it and the model says "yes" less often but more carefully. A score of 0.7 is not automatically a 70% probability.
Learn more
Token budget
The most tokens a task or agent loop may use
A token budget caps how many tokens an agent may spend on a task. Since AI services charge per token, the budget is also a cost limit. A loop that has no budget can run all night and keep spending; when the budget runs out, the agent stops even if the goal is not met.
Learn more
Validation loss
The loss measured on text the model did not train on
During training you watch two numbers. The training loss is measured on the text the model learns from; the validation loss on separate text it never trains on. If the training loss keeps falling while the validation loss starts to rise, the model is memorising instead of learning general patterns — that is the moment to stop or shrink the model.
Learn more
Antigravity CLI
Google's coding agent for the terminal, started with agy
Antigravity CLI is Google's coding agent that runs in a terminal. You install it with the script from the official page and start it with the agy command in a project folder; the first time, a browser opens to sign in with a Google account. It uses Google's Gemini models and, like Claude Code and Codex, reads, changes and runs files after you approve its plan. Google also publishes a guide for people moving from the older Gemini CLI.
Learn more
Claude Cowork
A Claude feature that carries out multi-step work across your files and apps
Claude Cowork is a feature of the Claude desktop app for multi-step work that is not programming: sorting files in a folder, turning a pile of receipts into a spreadsheet, or drafting a report from several documents. You give it a folder and describe the result; it plans the steps and works through them while you watch. Like every agent, what it produces is a draft you still have to check.
Learn more
Feature
One input value a model uses to decide, such as house size
A feature is one piece of input data a model looks at. A house-price model might use size, number of rooms and distance to the station; in a table, each column is usually one feature. Choosing useful features matters: features unrelated to the answer add noise and can make the model worse.
Learn more
Hugging Face
A website and company where people share AI models and datasets
Hugging Face is a website where people publish and download AI models, datasets and small demo apps — much as GitHub is for code. The company also makes open-source Python libraries, such as Transformers, that load and run those models. Many open models, from tiny to very large, are downloaded from there.
Image generation
An AI making a new picture from a written description
Image generation means an AI model turns a written description (a prompt) into a new picture, and many models can also edit a picture you give them. Good results come from describing the subject, style, size and what must not change, then fixing one thing at a time. Before using a picture, check it for mistakes such as wrong text or extra fingers, and for rights to people, brands and art styles.
Learn more
Image recognition
AI that tells what is in a picture
Image recognition is AI that looks at a picture and says what is in it: a cat, a stop sign, a handwritten digit. It is usually done with deep neural networks trained on many labelled photos. Phones use it to sort the photo album, and cars use it to read road signs.
LoRA
A cheap way to fine-tune: train a small add-on, keep the model frozen
LoRA (Low-Rank Adaptation) is a cheap way to fine-tune a large model. The original weights stay frozen; only small extra matrices, called an adapter, are trained and added to them. Because far fewer numbers are trained, it needs much less GPU memory and storage, and one base model can switch between several small adapters for different jobs.
Learn more
Learn more about weights and parameters
Full fine-tuning
Fine-tuning that updates every weight of the model
Full fine-tuning trains all the weights of a pre-trained model again on new data. For a 7B model, all 7 billion parameters change, and the result is a whole new model the same size as the original. It can change the model the most, but it needs the most GPU memory and time. When people just say "fine-tuning", they usually mean this.
Learn more
Learn more about weights and parameters
PEFT
Fine-tuning that trains only a tiny part of the model
PEFT (parameter-efficient fine-tuning) is the family of methods that fine-tune a model by training only a small number of parameters and leaving the rest frozen. LoRA is the best-known one. It needs far less memory and storage than full fine-tuning.
Learn more
Learn more about weights and parameters
QLoRA
LoRA on a quantized model, to use even less memory
QLoRA first quantizes the base model, storing its weights with fewer bits, and then trains a LoRA adapter on top of it. Because the frozen model takes up much less memory, a large model can be fine-tuned on a single ordinary GPU.
Learn more
Learn more about weights and parameters
Low-rank matrix
A big table of numbers that two small tables can rebuild
A low-rank matrix is a large table of numbers that carries little independent information, so it can be written as the product of two much smaller tables. The rank, r, says how small those tables can be. LoRA assumes the change that fine-tuning makes to the weights is low-rank, which is why it only has to train two small matrices.
Learn more
Learn more about weights and parameters
Catastrophic forgetting
A model losing what it knew while learning something new
When a model is trained further on new data, its weights move to fit the new data and can stop fitting what it learned before. The model then gets worse at things it used to do well. The risk is high in full fine-tuning, because every weight changes, and lower in LoRA, because the original weights stay frozen.
Learn more
Learn more about weights and parameters
Matrix multiplication
Multiplying two tables of numbers row by column
Matrix multiplication combines two matrices: each number in the result is a row of the first multiplied, item by item, with a column of the second, then added up. The inner sizes must match — a 4×3 matrix times a 3×2 matrix gives a 4×2 matrix. Almost all the work inside a neural network is matrix multiplication, which is why GPUs, built to do many of them at once, are used for AI.
Learn more
Neuron
The smallest calculating unit of a neural network
An artificial neuron takes several numbers, multiplies each by a weight, adds them together with a bias, and passes the sum through an activation function. Many neurons side by side form a layer, and layers stacked up form a neural network. The name comes from brain cells, but a neuron here is only arithmetic.
Learn more
Learn more about weights and parameters
Optimization
Adjusting a model's numbers so its loss becomes as small as possible
In machine learning, optimization means changing the model's weights step by step so that the loss — how wrong its predictions are — gets smaller. Gradient descent is the usual method: work out which direction lowers the loss and move a little that way. Optimization finds good numbers for the training data, which is not the same as being right on new data.
Learn more about weights and parameters
The shape you ask the answer to take: list, table, JSON, file
The output format is the part of a prompt that says what shape the answer should have: a bulleted list, a table with given columns, JSON with given keys, or a file. Stating it saves you rewriting the answer and lets a program read the result. It is one of the six parts of a good prompt, together with role, goal, context, examples and constraints.
Learn more
Positive class
In a yes/no classifier, the "yes" side the model is looking for
In a classifier with two answers, the positive class is the one you are trying to find — spam in a spam filter, illness in a medical test. "Positive" does not mean good; it only means "the thing we look for was detected". Precision, recall and the confusion matrix are all counted relative to this class.
scikit-learn
A Python library for classic machine learning
scikit-learn is a free Python library for classic machine learning: classification, regression, clustering and model evaluation. Every model is used the same way — fit() to learn from data, predict() to answer — so it is a common first tool for learners. It is not meant for training large neural networks; PyTorch is used for that.
Token ID
The number a tokenizer gives each token in its vocabulary
A tokenizer keeps a vocabulary — a numbered list of every token it knows. A token ID is a token's number in that list. Models never see letters: text is turned into a list of token IDs, the model works with those numbers, and its output IDs are turned back into text. The same word can get different IDs in different models.
Learn more about GPT
ChatGPT
OpenAI's AI chat service
ChatGPT is a chat service made by OpenAI, opened to the public in November 2022. You type a question and a large language model writes the answer; it can also read attached files and pictures and, when that feature is on, search the web. Its answers are drafts — fluent text can still be wrong, so check facts and numbers.
Learn more
Learn more about GPT
Gemini
Google's family of AI models and its chat app
Gemini is the name of Google's large language models and of the chat app built on them. The models are multimodal: they take text, pictures, sound and video. Google's coding agents, the Antigravity CLI and the earlier Gemini CLI, also run on Gemini models.
Grok
xAI's family of AI models and its chat app
Grok is the name of the large language models made by xAI, and also of the chat app built on them, on the web, on phones and inside X. Grok Build, also called the Grok CLI, is a different thing: xAI's coding agent that runs in the terminal. In short, Grok is the model, the Grok CLI is the tool around it.
SKILL.md
The file that defines an agent skill: name, description and steps
Every agent skill is a folder with a file named SKILL.md. At the top, between --- lines, it gives the skill's name and a description that tells the agent when to use it; below that come the instructions in plain Markdown. Agents first read only the name and description, and load the rest when a task needs it. Scripts and reference files can sit in the same folder.
Learn more
Gemini CLI
Google's earlier open-source AI agent for the terminal
Gemini CLI is an open-source AI agent from Google that runs in the terminal and uses Gemini models. Like Claude Code and Codex, it reads files, runs commands and follows instruction files, and it supports agent skills. BaeumNova's AI course now teaches Google's newer Antigravity CLI (agy); Google publishes a migration guide for Gemini CLI users, so check the official page before following old guides.
Resources
Data an MCP server offers for an app to read
An MCP server can offer three things: tools the model calls, prompts the user picks, and resources — data the app can read, such as a file, a record or a list of notes. Each resource has its own address (URI), for example notes://shopping. Tools do things; resources only hand over content.
Learn more
Claude
Anthropic's AI model family and chat service
Claude is the family of large language models made by Anthropic, and also the name of its chat app on the web, desktop and phone. Claude Code is a different thing: a harness program that uses a Claude model to work on your files. In short, Claude is the model, Claude Code is the tool around it.
GitHub Copilot
GitHub's AI assistant for writing code
GitHub Copilot is GitHub's AI coding assistant. In a code editor it suggests the next lines as you type and answers questions in a chat panel; in agent mode and in the Copilot CLI it can also change files and run commands. It reads agent skills too. Its suggestions are guesses, so you still read and test them.
OpenCode
An open-source AI coding agent for the terminal
OpenCode is an open-source coding agent that runs in the terminal. It works like Claude Code or Codex — it reads the project, changes files and runs commands — but you can connect it to models from many different providers. It also reads agent skills.
Learn more
Grok CLI
xAI's AI coding agent for the terminal (Grok Build)
Grok is the name of xAI's AI models. Grok Build, also called the Grok CLI, is xAI's coding agent that runs in the terminal, like Claude Code or Codex. It is installed with the official install script or as a global npm package, and you check the install with its version command.
Orca
An open-source desktop app for running several AI coding agents side by side
Orca is an open-source desktop app for running several AI coding agents at once, such as Claude Code, Codex and OpenCode. Each agent works in its own Git worktree, so their changes do not collide, and you compare and review the results in one window. You bring your own subscription for each agent.
AlphaGo
The Go-playing AI made by Google DeepMind
AlphaGo is a program for the board game Go, made by Google DeepMind. In March 2016 it beat Lee Sedol, one of the world's top players, 4 games to 1 — something many experts had expected to take another decade. It combined deep neural networks with search, and improved by playing against itself (reinforcement learning).
Learn more
Binary classification
Classification where the answer is one of exactly two choices
Binary classification is a classification problem with only two possible answers: spam or not spam, fraud or normal, sick or healthy. The model usually gives a score between 0 and 1, and a threshold turns it into "yes" or "no". The side you are trying to detect is called the positive class.
Learn more
Multiclass classification
Classification where the answer is one of three or more choices
In multiclass classification each input gets exactly one label out of three or more: cat, dog or rabbit; the digits 0 to 9. The model gives a score for every class, softmax turns them into probabilities, and the highest one is the answer. If one input can have several labels at once, that is a different task (multi-label).
Learn more
Chain-of-thought
A prompting method that asks the model to write out its steps
Chain-of-thought (CoT) prompting asks the model to write the steps of its working before the final answer, for example "solve step by step". Because a language model produces one token at a time, the written steps become context for the next ones, which tends to improve maths and logic answers. The written steps are text, not a guaranteed record of how the answer was really computed.
Chain rule
The rule for the slope of a function inside a function: multiply the slopes
The chain rule says that the slope (derivative) of a composite function equals the slope of the outer function times the slope of the inner one. A neural network is a long chain of functions, so backpropagation uses the chain rule again and again, from the output back to the first layer, to find how each weight affects the loss. Multiplying many small slopes is why gradients can vanish, and many large ones why they can explode.
Learn more about weights and parameters
Composite function
A function made by feeding one function's output into another
A composite function puts one function inside another: first compute g(x), then feed the result to f, written f(g(x)). A neural network is a big composite function — each layer takes the previous layer's output. The chain rule is how you find the slope of such a function.
Clustering
Grouping similar data together without given answers
Clustering puts similar items into groups when nobody has labelled them — for example, splitting shop customers into groups by what and how often they buy. It is a kind of unsupervised learning: the algorithm finds the groups, and people then decide what each group means. k-means, which you tell how many groups to make, is a common method.
Learn more
Confusion matrix
A table that counts right and wrong answers by kind
A confusion matrix compares a classifier's answers with the true answers. For a yes/no task it has four cells: true positives, false positives, false negatives and true negatives. Accuracy, precision and recall are all calculated from these four counts, so the table shows not just how often the model is wrong but which way.
False positive
The model says "yes" but the true answer is "no"
A false positive is a wrong alarm: the model says the thing is there when it is not — a normal email sent to the spam folder, a healthy person flagged as sick. Many false positives mean low precision. Whether they are worse than false negatives depends on the cost of each mistake in that job.
False negative
The model says "no" but the true answer is "yes"
A false negative is a miss: the thing was there but the model did not catch it — spam that reaches the inbox, an illness a test fails to find. Many false negatives mean low recall. In medical screening a miss can be far more costly than a false alarm, so recall matters more there.
Recall
Of all the real "yes" cases, the share the model found
Recall is the share of actual positive cases that the model correctly found: true positives ÷ (true positives + false negatives). Low recall means many misses — spam slipping into the inbox. Raising recall usually lowers precision, because catching more also lets more wrong alarms through.
Precision
Of everything the model called "yes", the share that really was
Precision is the share of the model's "yes" answers that were correct: true positives ÷ (true positives + false positives). Low precision means many false alarms — normal email ending up in the spam folder. Making the model stricter raises precision but usually lowers recall. (In computing, "precision" also means how many bits a number uses — a different idea.)
F1 score
One number that combines precision and recall
The F1 score is the harmonic mean of precision and recall: 2 × P × R ÷ (P + R). It is high only when both are high; if one is good and the other poor, F1 stays low. That makes it useful when the classes are uneven, where accuracy alone can look good while the model misses most positives.
Harmonic mean
An average pulled towards the smaller value
The harmonic mean of two numbers a and b is 2ab ÷ (a + b). Unlike the ordinary average, it is pulled strongly towards the smaller number, so one weak value cannot be hidden by a strong one. The F1 score uses it to combine precision and recall.
Cosine similarity
How closely two vectors point in the same direction, from −1 to 1
Cosine similarity measures the angle between two vectors, ignoring their length. 1 means the same direction, 0 means unrelated (at right angles) and −1 means opposite. Embedding search uses it to find texts whose meaning vectors point the same way as the question's. Close vectors mean similar topics, not that a sentence is true.
Data bias
Training data that leans one way and makes a model unfair
Data bias happens when the training data does not fairly represent the people or cases the model will meet. The model then learns the skew — for example, a face recognition model trained with few photos of one group makes more mistakes on that group. Check results separately for each group, and fix the data, not only the model.
Learn more
Deep research
An AI feature that searches many sources for minutes and writes a report
Deep research is a feature of AI chat services (such as ChatGPT, Gemini and Claude) where the AI searches and reads many web pages for several minutes, then writes a report with source links. It saves hours of searching, but the report can still misread or misquote a source, so open the links and check the important claims yourself.
Learn more
Learn more about GPT
Deepfake
A fake video, picture or voice made with AI to look real
A deepfake is a fake video, photo or voice recording made with deep learning so that it looks or sounds like a real person. The same technique is used legitimately in films and dubbing, but using it to put words in someone's mouth, to cheat, or to make sexual images of real people can be a crime. Check the source before believing a shocking clip.
Learn more
Dimension
How many numbers a vector holds
The dimension of a vector is how many numbers it contains: [1.2, 0.5, 3.0] is a 3-dimensional vector. Each number can be read as a position along one axis, so two numbers make a point on a flat map and three a point in space. Embeddings used by AI often have hundreds or thousands of dimensions — too many to draw, but computed the same way.
Learn more
Epoch
One full pass through all the training data
An epoch is one round in which the model has seen every example of the training data once. Training usually runs several epochs. After each one you check the loss on validation data: if it stops improving or starts rising while the training loss still falls, more epochs only make the model memorise.
Learn more
EU AI Act
The European Union's law that regulates AI by level of risk
The EU AI Act is the European Union's law on artificial intelligence, in force since 1 August 2024 and the first broad AI law of its kind. It sorts AI systems by risk: some uses are banned, high-risk uses (such as hiring or credit scoring) must meet strict rules, and others must be transparent — for example, people must be told when content is AI-generated. Its rules apply in stages over several years.
Learn more
Expert system
Early AI made of rules written by human experts
An expert system is an early kind of AI, popular in the 1970s and 1980s, built from many "if this, then that" rules written by human experts — for example, rules for diagnosing an illness. It could explain which rules it used, but it broke on cases nobody had written a rule for, and keeping thousands of rules up to date was hard. Machine learning, which learns rules from data, largely replaced it.
Learn more
Face recognition
AI that tells whose face is in a picture
Face recognition is AI that finds a face in a picture and matches it to a known person, as when a phone unlocks by looking at you. It works by turning a face into a vector and comparing vectors. It can be less accurate for groups that were under-represented in its training photos, and using it on people in public raises privacy and legal questions.
Learn more
Generative AI
AI that makes new text, pictures, sound or video
Generative AI is AI that produces new content — text, pictures, music, voices, video or code — instead of only sorting or predicting labels. Chat assistants and image generators are examples. Its output is built from patterns in the training data, so it can sound confident and still be wrong (hallucination).
Knowledge distillation
Training a small model to imitate a large one
Knowledge distillation trains a small "student" model to copy the outputs of a large "teacher" model, not just the right answers. The student learns from the teacher's full probability scores, which carry more information, so it often comes close to the teacher's quality while being much smaller and faster. It is a design trade-off: some accuracy for speed and cost.
Label
The correct answer attached to each training example
A label is the answer tag attached to a piece of training data: "cat" on a cat photo, "spam" on a spam email, the price on a house record. Supervised learning needs labels, and their quality limits the model — wrong or inconsistent labels teach wrong answers. Labelling is often slow work done by people.
Learn more
Supervised learning
Learning from examples that come with the right answers
In supervised learning the model learns from examples paired with correct answers (labels): photos marked "cat" or "dog", houses with their prices. It adjusts itself to match the answers, then predicts answers for new examples. Classification and regression are the two main kinds. It needs a lot of labelled data, which is often the expensive part.
Learn more
Unsupervised learning
Learning structure from data that has no answers attached
Unsupervised learning works on data without labels and looks for structure in it by itself — groups of similar items (clustering), unusual items, or a simpler way to describe the data. It is useful when labelling is too expensive. The groups it finds have no names; people decide what they mean.
Learn more
Reinforcement learning
Learning by trying actions and getting rewards or penalties
In reinforcement learning an agent tries actions in an environment and receives a reward or penalty for the result. Over many tries it learns which actions lead to the most reward in the long run. AlphaGo improved this way by playing against itself, and RLHF uses it to shape how chat models answer. A badly designed reward can teach the wrong behaviour.
Learn more
Reward model
A model that scores which of two answers people would prefer
A reward model is trained on comparisons where people picked the better of two answers. It learns to give a score to any answer, standing in for human judgement during reinforcement learning. Its weak spots matter: a language model can learn to please the reward model — for example with long, flattering answers — instead of really getting better.
RLHF
Reinforcement learning from human feedback: tuning a model with people's preferences
RLHF (Reinforcement Learning from Human Feedback) shapes a language model with human preferences. Typically: ① fine-tune the model on good example answers, ② have people compare pairs of answers and train a reward model on their choices, ③ use reinforcement learning so the model writes answers the reward model scores highly. It is a main reason chat assistants follow instructions and answer politely.
Learn more about fine-tuning and LoRA
Model card
A short document describing what a model is for and where it fails
A model card is a document published with a model. It says what the model is for and not for, what data it was trained on, how it was evaluated, its known limits and biases, and its licence. It helps others decide whether the model fits their job. Models on Hugging Face usually come with one.
n-gram
A model that predicts the next word from the previous n−1 words by counting
An n-gram model counts how often each word follows the previous n−1 words in a text, then turns the counts into probabilities. A bigram (n = 2) looks at one previous word, a trigram at two. It is fast and easy to understand, but it cannot use anything further back, and sequences it never saw get no probability — limits that neural language models overcome.
Learn more
Numerical differentiation
Estimating a slope by nudging the input a tiny bit
Numerical differentiation estimates a slope by adding a tiny amount h to the input and seeing how much the output changes: (f(x + h) − f(x)) ÷ h. It is simple and useful for checking, but it is only approximate and needs one extra calculation for every parameter — far too slow for a model with millions of weights. Deep learning frameworks use automatic differentiation instead.
Learn more about weights and parameters
On-device AI
AI that runs on the phone or laptop itself, without a server
On-device AI runs the model on your own phone, laptop or other device instead of sending data to a server. It keeps data private, works offline and answers without network delay. The cost is limited memory, speed and battery, so on-device models are small and often quantized — they cannot match the largest cloud models.
Learn more
Perplexity
A measure of how unsure a language model is about the next token
Perplexity measures how well a language model predicts a text; lower is better. A perplexity of 10 means the model is, on average, as unsure as if it had to choose evenly among 10 tokens at each step. It comes directly from the loss (it is e raised to the average loss). Low perplexity means good prediction, not that the text is true or useful.
Positional encoding
Numbers added to each token so the model knows its position
Attention on its own treats the input as a bag of tokens and cannot tell their order. Positional encoding fixes this by turning each position — 1st, 2nd, 3rd … — into numbers and adding them to the token's embedding. Without it, "dog bites man" and "man bites dog" would look the same to a transformer.
Prediction error
How far a prediction is from the right answer
The error of a prediction is the predicted value minus the correct value. Training usually squares the errors and averages them (mean squared error) to get the loss, so big misses count much more than small ones and positive and negative errors do not cancel out. The model then adjusts its weights to make that average smaller.
Learn more
Learn more about weights and parameters
Pruning
Cutting away a trained model's least important weights to make it smaller
Pruning removes weights or connections that matter little to a trained model's output, making it smaller and sometimes faster. The name comes from gardening: cutting branches the tree does not need. Too much pruning hurts accuracy, so a short round of extra training often follows. It is one of several ways to shrink a model, alongside quantization and distillation.
Learn more about weights and parameters
Reproducibility
Being able to rerun an experiment and get the same result
An experiment is reproducible when someone else can run it again with the same data, code and settings and get the same result. To make that possible, record the data version, code version, library and Python versions, the random seed and the exact commands. Even then, GPU calculations can differ slightly from run to run.
Rule-based chatbot
A chatbot that only replies with answers people wrote for set patterns
A rule-based chatbot matches what you type against rules written by people — keywords or patterns — and replies with a fixed answer. It is predictable and cheap, and you can build one with a few if statements. But it cannot handle anything outside its rules, unlike an LLM chatbot, which generates new replies.
Learn more
Speech model
An AI model that turns text into speech or speech into text
A speech model is an AI model that works with spoken sound: it turns text into a voice (speech synthesis) or a voice into text (speech recognition). Small speech models can run on your own device — BaeumNova's reading voice is downloaded once and then turns text into sound on your own device, while chat messages still go to a language model service.
Learn more
Transpose
A matrix with its rows and columns swapped
The transpose of a matrix swaps its rows and columns: row 1 becomes column 1, and so on. An m × n matrix becomes n × m, written Aᵀ. It is used all the time to make shapes match before matrix multiplication — for example, attention multiplies the query matrix by the transposed key matrix.
Turing test
A test of whether a machine's conversation can pass for a person's
In 1950 Alan Turing proposed the "imitation game": a judge chats in writing with a hidden person and a hidden machine, and if the judge cannot reliably tell which is the machine, it passes. It became known as the Turing test. Passing shows that a machine can imitate human conversation — not that it understands, feels or is conscious.
Learn more
Underfitting
When a model does poorly even on the data it trained on
A model underfits when it is too simple or trained too little to capture the pattern, so it gets many answers wrong even on its own training data. It is the opposite of overfitting, where the model fits the training data too closely. Fixes include a larger model, better features or more training.
Learn more
XAI
Explainable AI: methods that show why a model gave its answer
XAI (explainable AI) is the field of methods that let people see why a model produced an answer — for example, which words or pixels mattered most. It matters most where decisions affect people, such as loans, hiring and medicine, so mistakes and unfair reasons can be found and explained. An explanation is itself an approximation and should be checked.
Learn more
Amazon Bedrock
An AWS service for using AI models from several companies through one API
Amazon Bedrock is a cloud service from AWS that lets companies call AI models from several providers — including Anthropic's Claude — through AWS, with AWS billing and security settings. Claude Code can be set up to use Claude through Bedrock instead of an Anthropic account, but some features that need a claude.ai login are then unavailable.
Approval policy
The Codex setting for when the agent must ask you before acting
In Codex, the sandbox decides what the agent can technically do (write files, use the network), and the approval policy decides when it must stop and ask you. on-request asks when it needs to go outside the sandbox; never never asks, which suits scripts and isolated containers. Set the two together. It is chosen with -a or approval_policy in config.toml.
Learn more
workspace-write
Codex sandbox mode: edit and run commands only inside the project folder
workspace-write is a Codex sandbox mode. The agent may change files and run commands inside the working folder, but not elsewhere; the network is off by default and folders such as .git and .codex stay read-only. In a Git repository it is the usual choice together with the on-request approval policy. The other modes are read-only (look but change nothing) and danger-full-access.
Learn more
danger-full-access
Codex sandbox mode with no sandbox: the agent can do anything
danger-full-access turns the Codex sandbox off: the agent can read and write any file and use the network with your user's full rights. One wrong command can delete or leak your data, so use it only inside an isolated container or virtual machine that you can throw away.
Learn more
config.toml
Codex's settings file, ~/.codex/config.toml
config.toml is the file where Codex keeps its default settings, in your home folder at ~/.codex/config.toml. Options you would otherwise type every time — model, reasoning effort, approval policy, sandbox mode, MCP servers — are written here once. It uses the TOML format: key = value lines and [section] headings.
Learn more
Profile
A named set of Codex settings you pick with -p
A Codex profile is a named bundle of settings for one kind of work — for example a read-only "review" profile with a strict sandbox. Your defaults stay in config.toml, and codex -p <name> switches to the profile's settings for that run. It saves typing the same options again and again.
Learn more
codex cloud
Running Codex tasks in OpenAI's cloud and applying the results locally
codex cloud hands a task to Codex running in OpenAI's cloud environment instead of on your computer. Because your machine is free, you can run several tasks at once and pick only the finished results you want. codex apply copies a cloud task's changes into your local repository, where you review them like any other diff.
Learn more
Streamable HTTP
The way a remote MCP server talks over HTTP
MCP servers connect in two main ways. A local server runs on your computer and talks through standard input and output (stdio). A remote server runs elsewhere and talks over HTTP; the current standard way is called streamable HTTP, where the server can send a reply all at once or stream it in parts. You add one by giving its URL.
Learn more
Cross-review
Having a different AI or person check one agent's finished work
Cross-review means one agent does the work and a different one — another AI such as codex review, a sub-agent, or a person — checks it. A second checker with different blind spots catches mistakes the first one keeps repeating, including its own claim that "everything is done". It is a habit for agent work, not a statistics method like cross-validation.
Learn more
Pretrained weights
The numbers of a model someone else has already trained
Pretrained weights are the learned numbers of a model that someone has already trained, published as a file you can download. Starting from them saves the huge cost of training from scratch, and fine-tuning adapts them to your job. The BaeumNova XSLM course deliberately uses none: you write the code and train every number yourself, so you see where they come from.
Learn more
Learn more about weights and parameters
Learn more about fine-tuning and LoRA
Progressive disclosure
Reading a little first and more only when needed
Agents load skills by progressive disclosure, in three steps: ① at the start, only each skill's name and description; ② when a task matches, the whole SKILL.md; ③ only if needed, the extra scripts and reference files in its folder. This way you can install many skills without filling the context window.
Learn more
Research preview
An early, unfinished release of a feature, opened for trying out
A research preview is an early release of a new feature, opened before it is finished so people can try it and send feedback. It may change, have limits or be removed later, so do not rely on it for important work yet. Anthropic uses the label for features such as Claude Code channels.
Cursor
An AI code editor built on VS Code
Cursor is a code editor with AI built in, made by the company Anysphere and based on VS Code, so its look and extensions feel familiar. It suggests code as you type, answers questions about the project, and has an agent mode that changes several files at once. It also reads agent skills and AGENTS.md. (Not the blinking text cursor.)
Learn more
Goose
An open-source AI agent first released by Block
Goose is an open-source AI agent, first released by the payments company Block, that runs on your computer as a desktop app or in the terminal. It can use models from many providers and connects to tools through MCP. It also reads agent skills.
Learn more
Kiro
An AI coding editor from AWS that works from written specs
Kiro is an AI coding editor from AWS (Amazon Web Services). Besides chat and an agent that edits code, it encourages writing a spec — requirements, design and a task list — before the agent starts building. It also reads agent skills.
Learn more
Junie
JetBrains' AI coding agent
Junie is the AI coding agent from JetBrains, the company behind code editors such as IntelliJ IDEA, PyCharm and PhpStorm. It plans a task, changes files and runs tests inside those editors. It also reads agent skills.
Learn more
Kimi
Moonshot AI's AI models, also used in a terminal coding tool
Kimi is the name of the AI models and chat app from the Chinese company Moonshot AI. Some Kimi models are released as open weights. Moonshot AI also offers a command-line coding agent, so Kimi appears in lists of AI tools you can run in a terminal.
Learn more
Learn more about weights and parameters
Amp
An AI coding agent that started at Sourcegraph
Amp is an AI coding agent that started at Sourcegraph, the code search company. It runs in the terminal and in code editors, reads the project, changes files and runs commands like Claude Code or Codex. It also reads agent skills.
Learn more
OpenHands
An open-source AI agent for software development
OpenHands (formerly OpenDevin) is an open-source platform for AI software-development agents. Its agents write code, run commands and browse the web inside a sandbox, and you can run it on your own computer or server. It also reads agent skills.
Learn more
Roo Code
An open-source AI coding agent extension for VS Code
Roo Code is an open-source extension that adds an AI coding agent to VS Code. It reads and changes files and runs terminal commands after you approve them, and you choose which model provider it uses. It also reads agent skills.
Learn more
Laravel Boost
A Laravel package that helps AI agents work on Laravel projects
Laravel Boost is an official package from the Laravel team for AI coding agents. Installed in a Laravel project, it gives the agent an MCP server with tools to inspect the app (routes, database schema, logs), Laravel-specific guidelines, and searchable documentation for the package versions the project uses. It also works with agent skills.
Learn more
Automatic differentiation
The library works out the gradients for you
Training a neural network needs the gradient of the loss with respect to every weight. Automatic differentiation records each calculation as it runs, then walks that record backwards with the chain rule to compute all the gradients exactly. PyTorch calls this feature autograd, so you only write the forward calculation.
Learn more
Learn more about weights and parameters
Bias term
A number added to a neuron's sum, learned like a weight
A neuron multiplies each input by a weight, adds them up and then adds one more number, the bias. The bias lets the neuron give a non-zero answer even when every input is zero, and shifts how easily it reacts. Biases are parameters too, so training adjusts them together with the weights.
Learn more
Learn more about weights and parameters
Accuracy (model)
The share of all examples the model got right
Accuracy is the number of correct predictions divided by the total number of examples. It is easy to read but can mislead when one answer is far more common: if 99 of 100 emails are normal, a model that always says "normal" scores 99%. That is why precision and recall are checked as well.
Agent Skills
The open standard for packaging a skill as a SKILL.md folder
Agent Skills is the open format, kept at agentskills.io, that says how a skill is laid out: a folder with a SKILL.md file whose front matter gives a name and a description, plus optional scripts and reference files. Because it is shared, the same skill folder can be used by Claude Code, Codex and other coding agents. The standard only fixes the format; it does not check that a skill is safe.
Learn more
Marketplace
A list of plugins you can add and install in Claude Code
A plugin marketplace is a repository that lists plugins, bundles of skills, sub-agents, hooks and MCP servers. After you add a marketplace, you can install any plugin from it with one command, and a whole team can share and update the same tools. Only add marketplaces you trust, because plugins can run code on your computer.
Learn more
Stop hook
A script that runs when the agent is about to stop
A Stop hook is a hook that Claude Code runs each time the agent finishes its turn and is about to stop. The script can check the work and block the stop, sending the agent back to continue. Ralph Loop is built on this: the hook catches the stop and feeds the same prompt in again until the completion promise appears or the limit is reached.
Learn more
Claude Desktop
Anthropic's Claude app installed on a computer
Claude Desktop is the Claude app for macOS and Windows. Besides chatting, it can connect to MCP servers on your computer, so Claude can use local tools and files you allow, and it can run Claude Code and Cowork tasks without opening a terminal. In MCP terms, it is a host.
Learn more
Claude Agent SDK
A library for building your own agents on Claude Code's harness
The Claude Agent SDK is the harness that runs Claude Code — the agent loop, tools, permissions and context management — offered as a library for Python and TypeScript. With it you can build your own agent that reads files, runs commands and calls tools, without writing that loop yourself. Its old name was the Claude Code SDK.
Adaptive thinking
The model decides at each step whether to think first, and how much
With adaptive thinking, the model itself decides at each step whether to think before answering and for how long: quick for easy steps, longer for hard ones. Instead of setting a fixed number of thinking tokens, you steer the overall amount with the effort level — higher means more thinking and deeper checks, lower means faster and cheaper. Only some models support it.
Thinking tokens
Tokens a model spends reasoning before it writes the answer
Thinking tokens are the tokens a reasoning model writes while working through a problem before giving its answer. They are billed and take up context like any other tokens. The thinking budget is the most thinking tokens allowed; in Claude Code you raise or lower thinking with the effort level, and on models with a fixed budget the MAX_THINKING_TOKENS setting caps it.
bypassPermissions
The Claude Code permission mode that turns off every confirmation
bypassPermissions is a permission mode in which Claude Code edits files and runs commands without asking at all; it is what the --dangerously-skip-permissions flag starts. Because nothing stops a wrong or harmful command, use it only in a container or virtual machine with no internet access and nothing you would mind losing.
Learn more
Memory file
An instruction file Claude Code loads at the start of every session
In Claude Code, "memory" means instruction files it reads at the start of every session — CLAUDE.md files (or AGENTS.md), and the notes of auto memory. They are loaded into the context before you ask anything, so they take up part of the context window from the first message; keep them short. This memory is text files on disk, not the computer's RAM.
Learn more
ultrathink
A word you add to a request to make Claude think harder for that turn
Putting ultrathink in a request tells Claude Code to think more deeply for that one turn, without changing the model or the effort level for the rest of the session. It is handy for a single hard step, such as a tricky bug or a design decision. More thinking takes longer and uses more tokens.
Learn more
Claude in Chrome
A Chrome extension that lets Claude open, click and check pages in your browser
Claude in Chrome is a browser extension. Connected to Claude Code, it lets Claude open pages in your own Chrome, click, type and read console errors, so after changing code it can check the result on screen itself. It works in Chrome, Edge and other Chromium browsers, signed in with the same claude.ai account.
Learn more
The hook that runs just before a tool is used, and can block it
PreToolUse is a hook event that fires right before the agent uses a tool — before a file edit, before a shell command. Your hook command sees what is about to happen and can stop it: if it exits with code 2, the action is blocked and the message written to stderr is passed back to the agent. It is used to block dangerous commands or edits to secret files. Both Claude Code and Codex have it.
Learn more
PostToolUse
The hook that runs right after a tool has been used
PostToolUse is a hook event that fires right after the agent uses a tool, for example after it edits a file. It cannot undo the action, but it can follow it up — a common use is running a formatter such as Prettier on every file the agent changes. Both Claude Code and Codex have it.
Learn more
matcher
The part of a hook that says which tools it applies to
A hook's matcher chooses which tools the hook runs for. It is a tool name or a pattern: Bash for shell commands, Edit|Write for file changes. Without a matcher the hook would run for every tool, which is slow and can block things you did not mean to.
Learn more
/clear
Start a fresh conversation in Claude Code
Typing /clear empties the conversation so the next request starts with a clean context window. Use it when you move on to an unrelated task: old messages no longer fill the window or pull the work in the wrong direction. The previous session is saved, and /resume opens it again.
Learn more
Esc
The key that stops Claude Code in the middle of its work
Pressing Esc while Claude Code is working stops the current step at once, and the conversation stays, so you can say what was wrong and continue. Pressing Esc twice (or typing /rewind) opens the checkpoint list, where you can go back to an earlier point of the conversation and the code. Stopping early is cheaper than letting a wrong plan run to the end.
Learn more
claude mcp add
Register an MCP server with Claude Code from the terminal
claude mcp add gives Claude Code a new MCP server, and with it the tools that server provides. A remote server is added with --transport http and its address; a server that runs on your computer is added with -- and the command that starts it. claude mcp list shows the registered servers and claude mcp remove deletes one.
Learn more
--scope (MCP)
The option that decides where an added MCP server is saved and who can use it
claude mcp add saves a server in one of three scopes. local, the default, keeps it for you in this project only (in ~/.claude.json); project writes it to .mcp.json at the top of the project so the team gets it through git, with an approval on first use; user makes it available to you in every project. This scope is not the same as a variable's scope in the shell course.
Learn more
Shift+Tab
The shortcut that switches Claude Code's permission mode
In the Claude Code terminal, Shift+Tab cycles through the permission modes: default asks before each edit or command, acceptEdits edits files without asking, and plan only reads and shows a plan before changing anything. Switching to plan mode before a big change lets you read and fix the plan first. When a mode other than default is on, its name is shown under the input box. Since v2.1.283 a terminal session starts in auto mode, so the first press goes to default and the order is auto → default → acceptEdits → plan.
Learn more
Anthropic
The AI company that makes Claude
Anthropic is an AI research company founded in 2021. It builds the Claude family of language models and tools such as Claude Code, and publishes research on making AI safe.
OpenAI
The AI company that makes ChatGPT and Codex
OpenAI is an AI research company founded in 2015. It builds the GPT family of models, the ChatGPT chat service and the Codex coding agent.
Learn more about GPT
Hook event
The moment a hook runs, such as SessionStart
A hook event names the moment at which a hook (your script) runs: when a session starts, when you send a prompt, before or after a tool runs, when Claude stops. You choose the event in the settings file and the command that runs at that moment.
Learn more
Skill setting field
A Claude Code setting in a SKILL.md header, such as allowed-tools
Besides name and description, Claude Code reads extra fields in the frontmatter of SKILL.md: which tools the skill may use without asking, whether only you can start it, and whether it shows in the / menu. Other agents that read Agent Skills may ignore these fields.
Learn more
XChat
The chatbot you build in the XSLM course
XChat is the small chat program you write in the XSLM course. It sends your question to the Runner, which runs the GPT you trained yourself, and shows the answer — no ready-made model or outside AI API is used.
Learn more
Learn more about GPT
Choose how claude -p prints its result
When you run Claude Code without the chat screen (`claude -p`), this option decides what it prints. `text`, the default, prints only the answer; `json` prints one JSON object with the answer (`result`), the session ID and the estimated cost; `stream-json` prints one JSON line for each event while the run is going. Scripts usually pick `json` so another program can read the fields.
Learn more
AGENTS.override.md
Codex file that replaces AGENTS.md in the same folder
Codex looks for `AGENTS.override.md` before `AGENTS.md` in each folder and reads only one file per folder, so when the override file exists, the `AGENTS.md` next to it is skipped. It is handy for trying different instructions for a while without editing or deleting the shared file; remove it and the normal rules come back. Claude Code does not read this file.
Learn more
skill-creator
A skill that helps you make a new agent skill
skill-creator is a ready-made skill that walks you through building a new skill (a folder with a `SKILL.md`): it asks what the skill should do and when it should be used, then writes the files. Codex ships with it built in, and you start it by typing `$skill-creator`. Anthropic's public skills repository has a skill-creator too, which also helps test a skill with sample prompts.
Learn more
approval policy
Codex's rule for when to ask you before running a command
The approval policy decides when Codex stops and asks you before it runs a command. You set it with `--ask-for-approval` (`-a`): `on-request` runs what the sandbox allows without asking and asks about the rest, while `never` never asks but still stays inside the sandbox. Auto mode, which Codex suggests for version-controlled folders, is `workspace-write` plus `on-request`; the older value `untrusted` is no longer supported.
Learn more
evals.json
A file of test prompts and expected results for a skill
evals.json lists test cases for a skill: for each one, a request a user might type and what a good result should look like. You run the same requests with the skill and without it, then compare, to see whether the skill really helps. There is no single required format; Anthropic's skill-creator keeps it at `evals/evals.json` with the fields below.
Learn more
NVIDIA
The US company best known for graphics chips (GPUs) used in AI
NVIDIA is an American company that designs and sells graphics cards (GPUs). GPUs were first made to draw game graphics fast, but they also handle the huge amount of number crunching that training and running AI models needs, so most AI data centers use them. NVIDIA also makes CUDA, a set of tools for writing programs that run on its GPUs.
Learn more
Google DeepMind
Google's AI research lab, maker of AlphaGo and Gemini
Google DeepMind is the AI research organization inside Google. It built AlphaGo, the Go-playing AI that beat top player Lee Sedol in 2016, and it develops Google's Gemini models. It is a research lab, not a separate app — you meet its work inside Google products such as Gemini.
Learn more
NIST
The US standards agency that publishes AI risk guidelines
NIST (National Institute of Standards and Technology) is a US government agency that sets measurements and technical standards. For AI it publishes the AI Risk Management Framework (AI RMF), a guide to finding and reducing the risks of an AI system, and a Generative AI Profile that applies it to tools like chatbots. These are voluntary guidelines, not laws.
Learn more
Gartner
A US research and advisory company known for IT trend reports
Gartner is an American research firm that surveys companies and publishes reports and forecasts about information technology, such as its Hype Cycle charts. News articles often quote Gartner surveys of company leaders. Its figures describe the companies it surveyed, not everyone.
Learn more
Andrej Karpathy
An AI researcher known for nanoGPT and free courses on building language models
Andrej Karpathy is an AI researcher who was a founding member of OpenAI and led AI for Tesla Autopilot. He publishes small, readable projects such as nanoGPT, makemore and nanochat, and free video lectures that build a neural network from scratch. He also coined the phrase "vibe coding" in 2025.
Learn more
Learn more about GPT