What’s a calendar for, anyway?

A colleague that I have been discussing LLMs and how they work, said something to me the other day that was technically incorrect, but seemed plausible in the anthropomorphising way that people talk about LLMs and “AI” (quote marks because far too often, people say AI, when they actually mean LLM).

They said that the model that they were using (model name withheld to protect the guilty) was putting a date range in an output, and then in parentheses, it put the days that the date range was meant to refer to. But there was an issue; the days didn’t actually line up to the dates. The immediate response from the colleague, when I spotted that, was to say that the model might have got confused with time zones. I pointed out that the 21st is the same calendar date everywhere in the world, so it can’t be that.

The reasoning we do

When we are working with a person that makes a mistake with a task that they are actioning, we can often find ourselves trying to figure out how they came to the incorrect output/answer. We do this because we understand that people will work through a task, puzzle, or problem, and apply logical frameworks (one would hope) to the thing(s) being faced, and therefore we can figure out where the logic flow went awry.

We do this in non-deterministic programs too (previously called “programs” before we had to specify things to that level), with the process of debugging; Which is, in essence, working through a person’s logic flow (the programmer’s) to figure out where an edge case wasn’t accounted for, or similar.

Because of the industry’s approach to anthropomorphising the LLM systems, we have carried this over to the outputs of generative AI as well. We’ll reason with each other about why a day doesn’t line up to the date, or why a number in a chart was wrong, or why something else happened that resulted in a “You’re absolutely right to call it out” kind of pat-on-the-back response for paying for token outputs only to find that the tokens weren’t connected to the reality of the situation you were trying to deal with.

And for my next trick…

I am now going to explain how I can make you perform the same trick an LLM does every time you use it. Finish this sentence:

“The cat sat on the ___”

You thought of the word mat, didn’t you?

Why do you think that you thought of that word? The obvious answer is that, well, that it’s obvious. But why is it obvious? It’s obvious because you’ve seen, heard, and said that very same sentence many, many times in your life. You didn’t think about what colour the cat is. Nor if it is a household cat, or a lion in the plains of Africa. You didn’t consider that the cat and mat combination was inside a house, or outside, or in a zoo, or on the moon.

Literally zero thought entered your mind, other than the word “mat”.

And that’s because of the amount of times that you’ve seen, heard, or said it. Just the repetition that you have had in your life alone, was sufficient for me to trigger you to think of a word, without saying the word. M-A-G-I-C

throws glitter in the air

What does that matter?

Consider how many times that sentence has been uttered or written in your presence during your life. When, even, was the last time you heard/read it? You probably can’t even remember, but it’s in your brain. Raring to burst out like a double-knock follows a knock, knock, double knock, knock. Even me writing that has likely made you think of the sound of that beat. Again, it’s a thing that you just react to without any thought at all.

This, is the essence of token prediction.

There is no thought required by you to spit out “mat” or to knock twice after a particular pattern.

Now, imagine that I was able to cram literally everything that humans have ever written, into your memory. Think of the “cat sat on the mat"s, the “knock knock"s, the “the quick fox jumped over the lazy dog"s that you would start to react to.

Consider, if you had everything from github jammed into your head, how quickly you would understand what is usually written after func or def. Think about the amount of rapid connections and sentence finishing you would be able to do, outside of our feline friend resting on a carpet.

So it’s not actually thinking at all?

No. LLMs aren’t thinking.

They are the annoying friend that jumps into finish the sentence that you’re saying, even though they get it wrong. They are spitting out the most likely thing that comes next, based on the preceding text.

Let’s visualise that.

If, before I said the sentence earlier, I had asked you to complete the sentence and gave you this as a starting point:

The

Nothing else, just that. Think of the possible words that you could have put. There’s so many. Not all of them even are physical objects. Now let’s add another word, and you pick the next word:

The cat

Now we are getting somewhere. But the next words are all just words to describe what the cat is doing. Ran, Sat, Ate, Slept, Woke or more about tense, Was, Will, Can, Can’t

Adding another:

The cat sat

At this point, you would be able to guess “on the mat” likely quite quickly. But you also wouldn’t be asking for details on the cat or the situation that it has found itself in.

But you are able to more accurately produce a sentence with the more words that precede the end of the given statement. You are, in that sense, behaving exactly like an LLM.

Only when you are using an LLM, you are essentially sending the entire conversation back and forth for it to guess what words (tokens, really) come next. There’s no thought about context (although we call it a context window), there’s no understanding of what you mean (although we call it an attention head), there’s no thinking (but we call it a neural network), and there’s no intention in the actions produced (although we call it agentic).

The use of anthropomorphic language has tainted the understanding of the technology underneath the flashy harness that you happen to be using.

So yes, the prediction of tokens can be incredibly impressive, but then, the cat also sat on the….