Paste a 1,000-line file into a chat model and it reviews only the top half. The cause is the context window. The file plus the chat passed the limit, so part of your text never arrived. Every AI failure has a term like this, and this post maps 28 AI terms to the check that finds the cause.
Find your symptom in the list below, jump to its group, and run the check.
TLDR
Every AI failure traces back to one of six layers that feed the context window.
Start from the symptom, name the term, then run that term's check.
The 28 terms sit in six groups: misreads input, invents facts, wanders, must change, gets attacked, and costs too much. RAG and hallucination appear twice, because each fails in two places.
Commercial models invented a package name in at least 5.2% of 576,000 code samples in a 2025 USENIX study. Check every package name before you install it.
Try custom instructions first, then RAG, and fine-tune last.
One Term Turns a Mystery Into a Fix
Take one task and run it with two diagnoses. The task is to ask a chat model to review a 1,000-line file, and the review covers only the top of it.
Without the term, you rewrite the prompt three times and learn nothing. The same partial review comes back, and the next file fails the same way.
With the term, you check one thing. Did the file plus the chat exceed the context window? If yes, you attach only the file you need and ask again.
The model is the same in both runs. Only your diagnosis changed.
Every AI Failure Traces Back to One of 6 Layers
The context window is the text a model reads before it answers. Six layers put text into it, so each layer has its own failure.
My blog graph draws every post I have published and the links between them, so you can see which posts use which of these terms.
Start With the Symptom, Then Look up the Term
Read the symptom you see. The term after the arrow is where to look first.
The model reviews only part of my file. Look at the context window, and count tokens before you send.
It ignores a rule I wrote. Look at the system prompt, because a hidden instruction may conflict with yours.
It names a library that does not exist. Look at hallucination, and look the name up before you install.
It answers about my docs with confident errors. Look at grounding and RAG, and check which text it actually read.
The agent edits the wrong files or loops. Look at the agentic loop, and read its tool calls in order.
It forgets yesterday's decisions. Look at memory, because the context window starts empty each chat.
A web page or ticket changes its behavior. Look at prompt injection.
The bill is high for short questions. Look at prompt caching and the batch API.
Rule of thumb. Name the layer before you touch the prompt. A prompt rewrite fixes only the prompt layer.
When the Model Misreads Your Input, Check These 6 Terms
These six terms decide what text the model sees and how it reads it.
Token. A chunk of text the model reads, about 4 characters of English. OpenAI's tokenizer page gives the rule of thumb of 100 tokens to 75 words. A 1,000-line Python file often runs 3,000 to 5,000 tokens, so count tokens and never words when you ask whether a file fits.
Context window. The most tokens a model can hold at once. It covers your system prompt, every message, attached files, tool results and the reply. Anthropic's docs list up to 1M tokens for current Claude models and 200K for older ones. A chat app may drop older turns. An API call that fills the window stops with
stop_reason: model_context_window_exceeded. A new chat starts empty, so the window is working space and never memory.Prompt. The text you send. Compare "write a function" with a prompt that names the language, the input shape, the filter and the sort order. The second one gets a usable function because it states the spec.
System prompt. Instructions the tool injects before your message, usually hidden from you. When a rule you wrote gets ignored, ask what the system prompt says before you blame the model.
Few-shot prompting. You put 2 to 5 input and output examples inside the prompt, and the model copies the pattern. The examples last for that one context window only. The Appendix has a working example.
Chain-of-thought. You ask the model to reason step by step before it answers. It helps most on multi-step debugging, because a wrong early step shows up in the written reasoning.
When the Model Invents Facts, Check These 5 Terms
A model writes the text that looks most likely, and likely text is sometimes false. These five terms are the ways you feed it true text instead.
Hallucination. Confident output that is false, such as a function that does not exist. It comes with how language models work, so you manage it and never remove it. A 2025 USENIX Security study by Spracklen and colleagues, "We Have a Package for You!", generated 576,000 code samples from 16 models. At least 5.2% of the packages named by commercial models and 21.7% from open-source models did not exist.
Grounding. You anchor the answer to real text the model can read now. Ask what your
AuthServicedoes with no file attached and you get a guess. AttachAuthService.tsand the model reads the real code.RAG. Retrieval-augmented generation adds a search step before the model writes. The system finds the closest chunks of your documents and pastes them into the context window. Nothing is trained.
Embedding. A list of numbers that stores the meaning of a text. "Dog" and "puppy" land near each other, and "dog" and "database index" land far apart.
Vector store. A database built to find the stored embeddings closest to a query. Pinecone, Weaviate, Chroma and the Postgres extension pgvector are examples.
Watch out. When a RAG answer is wrong, read the fetched chunks first. A search that returns the wrong chunk produces a fluent answer from the wrong text.
Here is what a hallucinated package looks like to pip. I invented the name below to run the test, and no model produced it. I ran pip index versions first to confirm the name is unregistered, then installed it.
$ pip install pandas-csvfix-helper-zq
ERROR: Could not find a version that satisfies the requirement pandas-csvfix-helper-zq (from versions: none)
ERROR: No matching distribution found for pandas-csvfix-helper-zqA missing package fails loudly, and that is the good case. The USENIX paper calls package hallucination a package confusion attack, because someone can publish real code under the invented name.
When the Agent Wanders, Check These 6 Terms
An agent is a model that acts, and every action adds text to the context window. These six terms explain where it goes wrong.
Agent. A model that calls tools in a loop to finish a goal. A chatbot answers a question, and an agent goes and acts on the answer.
Agentic loop. The cycle of think, act and observe that repeats until the agent decides the work is done. It stops when it believes it is finished, whether or not the work is correct. A vague task gets a confident wrong result instead of a question back.
Tool call. One action the agent takes, such as reading a file or running a test. You define which tools exist, and the agent uses only those.
MCP. The Model Context Protocol, an open standard for connecting AI systems to data sources and tools. Anthropic announced it on November 25, 2024. One MCP server for your database serves any AI that speaks the protocol.
Memory. Facts saved outside the context window so a later session can load them. The window resets and memory survives. My own setup is in the 5 files that give Claude Code permanent memory.
Skills. A skill is a folder with a
SKILL.mdfile inside. The agent reads one line per skill at startup and opens the full file only when a job matches.
Alex Albert of Anthropic introduced MCP in this launch post.
I sell 39 skills in my skill pack, each one a folder like the one above.
When the Model Must Change, Compare 3 Options
Most people reach for fine-tuning first. Try these in order, because each one costs more setup than the one before.
Custom instructions. Saved rules the tool loads at the start of every chat, so you stop repeating yourself. They are the cheapest change, and the Appendix has a sample file.
Fine-tuning. You retrain a base model on your own examples, so its weights change for good. It needs a dataset and compute, and you then maintain the result. Most developers never need it, because instructions and RAG cover the cases that look like they do.
LoRA. Low-Rank Adaptation fine-tunes a model by training small added matrices and freezing the original weights. It needs far less compute than a full fine-tune, which is why open-source tooling uses it by default.
Tip. Move to the next option only after you can show the previous one failed on a real example.
When the Model Is Attacked, Check These 3 Terms
A model follows instructions in the text it reads, and an attacker can write that text.
Prompt injection. Hidden instructions inside input the model reads, such as a support ticket that says "ignore previous instructions". A jailbreak is the cousin that targets the model's own safety rules instead of your app.
Guardrails. Rules or filters built around the model that block unsafe output. A filter that rejects any reply containing an API key pattern is one example. A guardrail is a layer on top and never part of the model's judgment.
Hallucination, again. An invented package name is an attack surface, as the USENIX study above shows. Check every name against the registry before it reaches your build.
Riley Goodside demonstrated prompt injection against GPT-3 in this 2022 post.
6 New Terms for the Bill and the Build
These six terms decide what you pay and how much you can trust a run. Each links to the post where I use it.
Puneet Patwari posted the topics from an "AI engineering interview in 2026", starting with KV cache, prompt caching and semantic caching.
Prompt caching. The provider reuses the processed start of your prompt across requests. Anthropic's default cache lifetime is 5 minutes, and a cache read costs 10% of the base input price on most models, per its docs checked on 2026-10-13. Put static text first so every request hits the cache. The tactics are in Part 2 of the AI cost series.
KV cache. The key and value state a model keeps for tokens it already processed. Prompt caching saves that state across requests, which is why the cached prefix must match exactly.
Batch API. You submit non-urgent requests together at half price. Anthropic's batch docs say most batches finish within 1 hour and expire after 24. Part 3 of the cost series sorts this from the self-hosted techniques.
Structured output. You constrain the reply to a JSON schema so your code can parse it without a retry loop. Anthropic offers it as an API feature.
Hook. A script the tool runs on its own at a set event, such as after every file write. A hook can block a bad output, and a reminder in a prompt cannot.
Eval. A written test that grades a model's output, so you can compare two versions. One eval run on a Claude skill cut its token use by 17% with no quality loss, as I wrote up in Skills 2.0.
Five Look-Alike Pairs Fail in Different Places
Five pairs of terms get mixed up. Mixing them up sends you to the wrong layer.
A prompt and a system prompt differ in who writes them. You type the prompt, and the tool injects the system prompt.
Context window and memory differ in lifespan. The window resets each chat, and memory outlives it.
A skill and custom instructions differ in timing. Instructions load at every start, and a skill opens when a job matches.
A tool call and MCP differ in scope. A tool call is one action, and MCP is the standard that exposes tools to any AI.
Few-shot prompting and fine-tuning both teach by example. Few-shot lasts one window, and fine-tuning changes the weights.
Four Questions a Newcomer Asks About AI Terms
Which term do I learn first? Learn the context window, because it explains the most failures. Dropped files, forgotten rules and silent truncation all trace to it.
Does a bigger context window fix these failures? A bigger window raises the limit. It cannot fix a vague prompt, a wrong retrieved chunk or an agent that decides too early that it is done.
Can hallucination ever reach zero? No. Even the best commercial models in the USENIX study named at least 5.2% nonexistent packages on average. Verify any name before you install it.
Should I fine-tune my model? Try custom instructions first, then RAG. Fine-tune when you hold thousands of examples of the exact output and no prompt gets it right.
The Appendix Holds Every Command and Example
A few-shot prompt that teaches a format:
Convert these to title case:
Input: "hello world" -> Output: "Hello World"
Input: "this is a test" -> Output: "This Is A Test"
Input: "my variable name" -> Output:A custom instructions file for a TypeScript project:
- Use TypeScript with strict mode
- Prefer async/await over .then() chains
- Name component files in PascalCase
- Never use the any typeCheck that a package exists before you install it. The command prints the versions pip can see, or an error when the name is unregistered:
pip index versions <package-name>
npm view <package-name> version