How LLMs Work, and Why They're Not Consistent

An LLM is not a lookup table and it is not retrieving a stored recipe. At its core, an LLM is a next-word (technically, next-token) predictor. Given everything written so far — your question, and whatever the model has already written in its answer — it computes a probability for every possible next word it could produce, and then picks one.

For the prompt "How do I make a chocolate cake? Step 1: Pre—", the model isn't recalling a fact. It's computing something closer to:

It picks a token, adds it to the text, and repeats the process for the next token — over and over, one small piece at a time, until the response is complete. This happens because of how the model was trained: on an enormous amount of text, it learned the statistical patterns of how words follow other words in this kind of context. It didn't memorize one chocolate cake recipe — it absorbed the patterns across thousands of them, and can produce any recipe consistent with those patterns.

This is the first thing to internalize: an LLM response is a chain of probabilistic choices, not a retrieved fact. Consistency was never guaranteed by how the system works — it has to be deliberately engineered in.

1. Sampling — the model doesn't always pick the most likely word

Once the model computes the probability table, it has to choose one token. Always taking the single highest-probability token (greedy decoding) produces flat, repetitive, oddly robotic text. So most LLMs instead sample from the probability distribution: a high-probability token is likely to be picked, but a lower-probability one still has a real chance.

This is controlled by temperature:

• Temperature = 0 pushes toward always picking the top token (closest to deterministic).
• Higher temperature (e.g., 0.7–1.0) flattens the distribution, making less-likely tokens more likely to be picked — more variety, less repeatability.

This is why "cream butter and sugar" can beat out "mix flour, sugar, cocoa" on one run and lose on the next — both were plausible, reasonably high-probability openings.

2. Every choice compounds into every later choice

This is what turns a small amount of per-token randomness into two entirely different-looking recipes. Once the model samples "Cream" instead of "Mix" for step 2, everything downstream is now predicted conditioned on "Cream" — the model is far more likely to continue with "butter and sugar together" than "flour, cocoa powder." One early, small, random fork sends the rest of the response down a completely different (but equally valid) path.

3. The model itself changes over time

AI providers regularly ship updated model versions, adjust system prompts or default behavior, and update or swap the infrastructure serving the model. If the underlying model changed between your two questions, you're not even asking the same model the second time. This is a second, independent source of inconsistency layered on top of sampling — one that's easy to miss because nothing about the product visibly changed.

The exact same mechanism applies the moment you move from asking Claude Code for a recipe to asking it to analyze code. Say you hand it a real, 100-line JavaScript file and ask: "Analyze this code and tell me what issues you see."

You get back something like:

1. Line 23 — Possible null reference: user.profile.email is accessed without checking that user.profile exists.
2. Line 41 — The for loop iterates to <= items.length, which will read one element past the end of the array.
3. Line 58 — fetch() call has no .catch() — an unhandled promise rejection will fail silently.
4. Line 74 — Nested loop over the same array is O(n²) where a Map lookup would be O(n).
5. Line 90 — Inconsistent naming: getUserData alongside fetch_order_details.

Ask again — same file, same prompt, nothing changed in the code — and you might get back:

1. Line 23 — Possible null reference on user.profile.email.
2. Line 58 — Missing error handling on the fetch() call.
3. Line 74 — Nested loop could be optimized.
4. Line 88 — Variable temp is declared but never used.

Notice what happened: the off-by-one loop bound on line 41 — a real, fairly serious bug — was caught in Run 1 and silently missed in Run 2. Nothing about the code changed between the two runs — only which findings the model happened to sample into existence.

"Capital of the USA" is a retrieval task

During training, the model encountered the pairing USA → Washington, D.C. an enormous number of times, in an enormous number of phrasings. Answering the question is mostly a matter of retrieving an association that already exists inside its learned parameters. There's very little construction involved.

Your JavaScript file is not a retrieval task

The model almost certainly never saw your exact function. But during training it saw millions of examples of code containing the same building blocks:

• function introduces a reusable block of behavior
• if introduces a condition
• age > 65 is a comparison
• baseRate * 1.25 is arithmetic
• return produces an output

It learned the generalized pattern, not your specific instance of it. So when your code arrives, the model isn't looking up "this file" — it's recognizing familiar shapes inside a file it's never seen and reasoning about how those shapes relate to each other.

Your exact code (never seen before)
        │
        ▼
Break into tokens
        │
        ▼
Recognize familiar programming patterns
   (learned from millions of OTHER examples)
        │
        ▼
Understand relationships between
variables, conditions, and operations
        │
        ▼
Construct an interpretation
        │
        ▼
Answer your question

The deeper distinction: knowledge vs. context

Say the model was only ever trained on a trivial add(a, b) function, and you hand it something entirely new:

function calculateRisk(policy) {
    if (policy.driverAge < 25 && policy.vehicleValue > 50000) {
        return "HIGH";
    }
    return "NORMAL";
}

The model has no idea what your policy object actually contains, whether vehicleValue is in dollars, or whether this business rule is even correct. That's all missing context. But it can still read the structure: IF condition A AND condition B THEN return HIGH, ELSE return NORMAL.

Two very different resources are at work here: learned knowledge (baked into the model's parameters during training — general programming concepts) and temporary context (your specific code, placed in the prompt, reasoned over only while it's in the context window and then forgotten). A useful shorthand: training gives the model its general education; the prompt gives it the exam question.

Where it stops being recall and starts being reasoning

Suppose the requirement is "apply the surcharge if driver age is below 25 OR vehicle value exceeds $50,000," and the code says:

if (driverAge < 25 && vehicleValue > 50000) {
    return "HIGH";
}

To flag this as wrong, the model has to hold two new pieces of information — the plain-English requirement and your code — in its context at once, translate both into the same logical shape (A OR B vs. A AND B), and notice they disagree. That is not "look up the fact." That is comparing new information against new information — the same category of task as spotting that "over the moon" describes "flew," just applied to logic instead of grammar.

Different inputs activate different learned patterns

During training the model absorbed huge numbers of examples like:

Question: What is 2 + 2?
Answer: 4

Question: Why does this code fail?
Answer: Because the loop condition...

Requirement: The system should reject values below 18.
Code: if (age <= 18) reject();
Analysis: This incorrectly rejects 18.

The model isn't told "these are different modes" — it just learns, statistically, that text shaped like a code-and-requirement comparison tends to continue with step-by-step logical language, while text shaped like a trivia question tends to continue with a short, direct fact. Your prompt's shape is what steers it toward one learned pattern or the other — the same sampling-over-a-probability-distribution mechanism from Section 2, just conditioned on a different kind of input.

The subtle part: it isn't necessarily "thinking in steps" at all

A human reviewer explicitly runs through: understand requirement → parse code → compare → test the boundary. A base LLM, absent a specialized reasoning system, is still just predicting the next token, one at a time, as described in Section 2 — it's not guaranteed to execute those steps explicitly. But the learned representations can be sophisticated enough that the output looks like that reasoning happened:

Requirement: "age >= 65"        Code: "age > 65"
        │                                │
        ▼                                ▼
  comparison, threshold,           comparison, threshold,
  INCLUSIVE boundary               EXCLUSIVE boundary
        └───────────────┬────────────────┘
                         ▼
              Difference detected at age = 65

The model has learned that "inclusive" and "exclusive" boundary phrasing are semantically different — not because it executed a formal comparison algorithm, but because that distinction showed up often enough in training data that it became part of the learned representation.

Traditional generation vs. reasoning-oriented models

This is also where "reasoning models" genuinely differ from a standard model doing next-token prediction straight through:

Prompt
  ↓
Predict next token
  ↓
Predict next token
  ↓
Answer

Prompt
  ↓
Explore interpretations
  ↓
Perform logical steps
  ↓
Check result
  ↓
Answer

Pretraining gives a model the ability to recognize these patterns at all. A reasoning-oriented system adds more computation on top — more space to explore, check, and revise — before committing to an answer. Neither approach changes the fact from Section 2: this is still sampling from a learned distribution, not executing a guaranteed algorithm.