Writer’s note

This post is wrapping up the chain of posts for the LLM part of this blog. After this, I’ll likely be talking about LLMs less and actually getting on the topic of AI as a whole; I want to talk about how hidden Markov models can bring value, how a decision tree can be powerful, and expanding on the things that AI has, that people usually don’t mean when they say AI.

This post is going to assume that you understand the concepts of tokenisation (see https://glennwoodward.com/blog/2026-01-11-embedding-down-for-the-night/ if you need a refresher), and also how tensors are used to get to the token values in the embedding space (see https://glennwoodward.com/blog/2026-01-05-apartments-dollars-and-tensors-oh-my/ if you want to ensure you’re on the same page as me for that).

With that said, let’s get on with the post…

Secret squirrel business

Like probably all high school kids, I was obsessed with being able to communicate with classmates without teachers finding out what we were talking about. I also had an unhealthy interest in spying as an industry and how early cryptographers would encode their messages in a way that was difficult to break if you didn’t know the method of decryption, yet easy to decode if you knew how.

Everyone has looked at the most basic of encryption methods; namely, character substitution. Changing an A to a B, or an A to be a 1. Very simple, but also widely known as well.

I needed something that fit the modern (at that point) age, where I could communicate over the internal computer systems in the lab, and use the extremely basic BBS (bulletin board system) that the school operated for students to complain about homework (and then get immediately busted because keyword searches existed and the teachers weren’t idiots).

I was able to touch type early on, as were a few of my fellow students that were interested more in being able to chat in a cryptographic way, than they were in having information that needed to be encrypted. Therefore, the approach that I landed on, was a simple character substitution, but based on hardware.

To encode a message, we would simply move our hands to the right on a standard QWERTY keyboard by one key, and then touch type our thoughts. A became “S”, M became “,”, and L became “;”. Asking how someone was would turn into something resembling, at a glance, just random key smashes (jpe str upi fpomh).

What are you even talking about?

Before I answer that, let’s remember the flow for a single word in a prompt, that (in this hypothetical example) gives one word added on by an LLM.

You type “Hello” The LLM harness finds the token value number in the database for that word. Let’s say that it’s “3005” The process then looks at where 3005 is, within the embedding space, and gets the vector/tensor for the location of the value (let’s say that is [32,663,12,40]) The prediction system then processes that tensor and finds the most likely next token that comes after 3005, and manages to find 41934 as the most likely token The system then puts those two tokens together (3005 41934) and sends that to be processed in the database that holds what words (or punctuation, or whatever) are attached to each token value It then converts the two token values into words and you get on the screen: Hello there

That is, at a very basic level, what is happening when you use an LLM. The prediction process doesn’t know what the tokens represent in the database key/value system.

So, using my CIA grade encryption (haha) that I would run as a teenager, let’s imagine that we add in a function to the input side, and the output side.

def get_token_for_input(input_word): token = get_token_from_database(input_word)\n return int(token + 1)

And for the output side:

def get_token_for_output(output_token): return int(output_token -1)

Those are pretty simple python functions, but let’s look at what they are doing. For the input, we are getting the token number for the word being given, and we are simply adding 1 to the number. For the output, we are simply subtracting one.

The word you put in (“Hello”) in our example would be unchanged, and would be added one, and then subtracted one, and print to your screen as “Hello”. Exactly like you typed it. However, the second word wouldn’t be “there”.

We know this, because words aren’t tokenised in a numerical order. You can’t add 1 to “Hello”, but you can add 1 to “3005”. And 3005 is a completely different word.

Let’s pretend that the resulting token of 3006 is for the word “Good”. This is then processed in the LLM and it might process the next token as the one representing the word “dog”. Those two tokens can be imagined as flowing one to the other.

Let’s imagine that the token for “dog” is the number 883214. This is then passed through the output function that we put in the process, and we get the token ID of 883213. Let’s say this is the token ID that is then used to look up the key/value pair to get the word for it, and this word in our imaginary embedding space is “suitcase”.

The output on your screen now becomes:

Hello suitcase

Again, what are you talking about?

The token prediction worked flawlessly. The LLM took in a token embedding for the thing you typed (or what it thought you typed), predicted the next token (for the token it thought you typed) and returned it.

The statistical process executed in exactly the same way as it should. The difference in this case, was simply changing the integer for the token ID on the way in, and the way out.

This is a thought experiment to explain the fundamental core of LLMs - they are NOT looking at the words you type. They are NOT inferring meaning. They are just looking at numbers, and doing maths.

The ol’ switcheroo

If we could switch out the token/value databases (and I’m specifically referring to the vocabulary data, not the model weights) between Anthropic’s Fable 5, and OpenAI’s ChatGPT 5, the core processing underneath would work without (too much) of a problem, but the outputs would be utter gibberish. This is because the words you type and see from the model are NOT from the model, they are just the representations that we gave the token numbers that the model spat out.

The underlying LLM process doesn’t care if 3005 means “hello”, or “banana”, or “IPO failure”. It doesn’t matter at all, because the model was never, ever, trained on words.

Read that again.

The models that you are using, were never directly trained on words. They were trained to predict number sequences that we attributed words to.

Taking the Opus 4.8 algorithm and applying it to a completely different token embedding space would be like dropping me in the middle of Spain, and telling me to start a business (for the record, I don’t speak Spanish).

The language that you type, the code you paste in, the RAG-sourced PDFs, the internet search to “ground in the facts”… all of it is NOT passed into the LLM model as words and “rich, and complete” information… It’s just a bunch of tensors, that represent token IDs, for a very specific embedding space database. And if you just switch that database with another database for another model and training process (even if they are trained on the exact same input data) would result in utter nonsense for the output.

Yeah, but we aren’t switching out embedding spaces, are we?

Yeah, you are. Often. You just don’t realise it.

When an AI lab does a training run for a model (when training occurs for a ChatGPT upgrade, or Opus 4.7 to 4.8) the embedding space changes. It’s not “there’s a chance it changes”, I mean that a fresh training run means that the embedding vectors – at the very least – are completely different. A slight warm-start retrain will mean that the embedding space gets nudged around enough that the tokens aren’t where you left them.

Even the RLHF (reinforcement learning from human feedback) has a chance of nudging the locations of tokens in the space (assuming that they use a non-LoRA approach to the process).

The point I’m making here is that whenever someone tells you that the model is “looking for connections in the data” or “reviewing the documents for clues” or “completing an analysis from RAG inputs”, they are wrong. The model is looking at a bunch of numbers, that represent bits of words in a VERY specific lookup table, and retraining the model means basically throwing out the lookup table and starting again.

Final thought on LLMs for now

Large Language Models are amazing. They truly are. That we are able to create a model from a few hundred lines of Python code (not counting the PyTorch module imported) and have it be able to mimic human language use is incredible.

The problem is that some advocates don’t actually understand the process under the hood, and use anthropomorphic language (“it’s thinking”, “it’s given a recommendation”) to describe the output that they see.

This causes people to assume that a replication of language (that normally requires human intelligence to string together in a coherent way to transfer our thoughts to others), means that the computer must be intelligent.

Add to this the CEOs of these AI companies (and connected companies) to state that people can “buy intelligence on a meter” or that a model has “PhD level intelligence”, and you get a situation where companies have invested so many billions of dollars each, that to say that they might have made a mistake, would be impossible.