In a coming post, I intend to talk about how the training process works; however there are two important concepts that you need to have some awareness of, so that you can get your head around the training process and how back propagation works.

These two things are:

• Loss function (discussed in this post)
• Local minimum / Global minimum (discussed in the next post)

Ok, let’s imagine that you are back in the world of apartments (a theme that this journey has returned to a few times!), and you are guessing the price of properties. Let’s assume that the last sale price (the value you are trying to guess) is known, just not by you. Once you’ve locked in your answer, you are then told “higher” or “lower” as to which direction you want to head in to guess the right price. “$550,000?” “Lower” Now at this point, you actually have no idea how much you are off by, and how much you need to change your guess by.

In Machine Learning concepts, you aren’t sure how much to punish or reward yourself, for how close you are. Could be 100,000 off, could be 20 bucks. Both would mean your guess wasn’t correct, but one is FAR closer than the other.

When computers are learning things, we like to (essentially) exaggerate their bad guesses so that they understand just how bad that guess was. This is the loss function: “How badly did you mess up by?”

A common approach when the model is guessing a number (like apartment price) is to work out the difference between the correct answer and your guess, then square that number.

For our two guess possibilities above: 100,000 off becomes a square error of 10,000,000,000 (so, you know, fairly shoddy) 20 off becomes a square error of 400

There are two main benefits to the squared error approach:

• The squared error is always a positive value, and;
• The further away the guess is, the more the error is effectively punished

When we are training machine learning systems on numbers, we often have the model guess a whole bunch of known examples, then calculate the mean squared error across all of them. That average score is a very common way to say “how good or bad is this predictive model?”

But LLMs are different. We are not trying to guess a number (like price). We are trying to teach the model to guess the correct next token out of tens of thousands of possibilities in its vocabulary. So here’s what actually happens: The model reads “The cat sat on the” and has to guess the next token. It outputs a probability score for every single token it knows (all adding up to 100 %). • ‘mat’ : 62 % • ‘roof’ : 12 % • ‘elephant’ : 0.0003 %
… and so on. The training data then reveals: the real next token was ‘mat’. The loss function (called cross-entropy loss) simply asks: “How surprised were you by the correct answer?” If you gave ‘mat’ 99.9 % probability, the loss is tiny. If you gave it only 62 %, the loss is bigger. If you gave it 0.001 %, the loss is huge. Even if ‘mat’ was your highest-probability guess, you still get punished until you push that probability closer to 100 %. The model then tweaks its internal weights (we’ll see exactly how in the training post) until this error is as low as it can get for the current training run. It’s important to realise that the lowest error rate for that training run isn’t necessarily the lowest error rate possible in the entire universe of possibilities. And how do we get to the lowest error rate for the training run? And why isn’t that always the absolute lowest possible?

Well, that will be in the next post, where we discuss the local and global minimums…