We’ve now covered quite a few of the building blocks that sit inside an LLM. We’ve talked about tokens, tensors, embedding spaces, attention heads, MLPs, loss functions, and our muddy-shoes local minima.

So now we can finally answer the bigger question: How does an LLM actually get trained?

And perhaps more importantly: How does it improve?

Because when people hear phrases like “training”, “back propagation”, and “machine learning”, it can all start to sound a bit mystical. It isn’t mystical. It is mostly just an unbelievably large amount of guessing, checking, blaming, adjusting, and trying again.

Imagine you run a gigantic call centre company. Your company’s only job is to finish sentences, when someone calls in with one.

Customers call in and say things like:

  • “The capital of France is…”
  • “The cat sat on the…”
  • “Two plus two equals…”

Your employee has to immediately say the next word. No hesitation, just a response instantly after the customer stops saying the first part of the sentence.

At the very start, your employees are terrible. Not “a bit rough around the edges” terrible. I mean “spectacularly useless”.

You say: “The capital of France is…” And they confidently reply “potato.” and hang up the phone with an accomplished smile.

So you stop them and say: “No. Wrong. The correct next token was Paris.”

Then you give them another one. And another. And another.

That, at a high level, is training.

You show the model enormous amounts of real text. You hide the next token. You make the model guess. Then you compare its guess to the real answer.

That comparison produces the loss.

And once you know how wrong it was, you can begin adjusting the internal weights so that next time it is slightly less wrong.

That adjustment process is what back propagation is all about. It’s small little adjustments that would have got the resulting token (“potato”) closer to the actual token that it should have been (“Paris”) so that if that prompt was given again, it is more likely to have got the right token in response.
Each time the weights are adjusted, they are adjusted whilst keeping in mind all the previous answers/tokens given/generated, so that it’s not chasing it’s tail, and constantly undoing the learning that has already been done.

We need to keep one thing clear, though. The model is not sitting there with thoughts in its head. It is not pondering language like a sleepy philosopher.

It is taking input tokens, running them through all the layers of the architecture, and producing scores for what token should come next.

So if the input is: The cat sat on the

The model might internally produce something like:

  • mat → very likely
  • rug → somewhat likely
  • roof → less likely
  • microwave → hopefully not very likely

The training data then reveals the true next token. Perhaps it was mat.

If the model gave mat a high probability, then good — the loss is relatively small. If it thought microwave was a serious contender, then it deserves to be told off. Quite firmly.

So where does back propagation come in? Let’s go back to our pretend company.

Inside the office, your sentence-finishing employee is not one person. It is a huge chain of departments.

One department looks at relationships between words. That is a bit like the attention mechanism.

Another department takes numbers in, multiplies them by lots of weights, and passes the numbers onward. That is a bit like the MLP layers.

By the time the answer comes out the far end, it has passed through many, many stages. Now suppose the final answer was wrong.

You know the company made a bad prediction. But which part of the company should be blamed?

Was Department 17 too aggressive? Was Department 42 not paying enough attention? Was one of the many little internal multiplications slightly off in a way that nudged the whole answer in the wrong direction?

Back propagation is the method used to send the blame backwards through the system.

You start with the final error, and then work backwards through all the layers to calculate how much each weight contributed to that error. Once you know that, you can adjust each weight a tiny bit in the direction that would have reduced the mistake.

Not a huge bit. A tiny bit. A microscopically small bit.

Then you do it again. Millions, billions, or trillions of times.

Back propagation is, in effect, the mathematics that lets the system say:

  • this step mattered a lot
  • this one mattered a little
  • this one pushed the result in the wrong direction
  • this one was actually helpful

So next time, each step is nudged slightly. A tiny bit more here. A tiny bit less there.

Without getting lost in the weeds, the training loop looks something like this:

  1. Feed in a chunk of training text
  2. Hide the next token(s)
  3. Let the model predict what comes next
  4. Compare its prediction to the real answer
  5. Calculate the loss
  6. Back-propagate that error through the network
  7. Adjust the weights slightly
  8. Repeat

That’s it.

Well — that’s “it” in the same way that saying “a plane flies by pushing air around” is technically true, while also skipping over an alarming amount of engineering. But at the conceptual level, that is the process.

You might reasonably ask: “Why not make big changes if the answer was very wrong?”

Because if you change too much, the whole system becomes unstable.

Think back to our person stumbling around on the muddy hills at night with the dreadful flashlight. If every time they found a downward slope they leapt 300 metres in that direction, they would not gracefully descend the landscape. They would launch themselves into a hedge, a ditch, or possibly a sheep.

Training works by taking many small corrective steps.

It is tempting to imagine the model memorising facts in neat little boxes. But that is not really what is happening. What it is learning is a vast pattern of weighted numerical relationships. These relationships make some token sequences more likely and others less likely.

So after seeing enormous quantities of text, the model becomes good at things like:

  • recognising that some words often go together
  • recognising that grammar has patterns
  • recognising that certain structures tend to follow certain prompts
  • recognising that in many contexts, some completions are much more plausible than others

This is why the final product can look so impressive. It has not been given a tiny lookup table of canned answers. It has had its weights adjusted, over and over, so that the architecture becomes very good at prediction across an enormous range of linguistic situations.

And this is the core concept of LLM training; An LLM is trained by making predictions on known text, measuring how wrong those predictions were, and sending that error backwards through the network so that all the internal weights can be adjusted slightly.

Then doing it again. And again. And again.

So what comes out at the end? After enough training, you do not get a machine that “knows” things in the way a person does, even thought that is what so many articles will tell you. You get a machine whose internal weights have been shaped so that it becomes extraordinarily good at predicting the next token in ways that often look intelligent.

There is no tiny librarian in the model. No little ghost in the GPU. No hidden homunculus reading your prompt and having a nice think.

Just layers, weights, error signals, gradient descent, and a frankly absurd amount of computation. Which, in a strange way, is even more interesting.

In the next post, we will look at a great use case for LLMs; Agentic AI.
We’ll talk about how building a good harness is key to getting around the downsides of LLM inference, and maximising the upsides.