Welcome back! Thinking about our arrays that we looked at with the features of apartments, a few posts ago, you would be able to understand that the following tensor could describe an apartment in a building:
[59, 1, 0, 1, 630000, 0, 4.21, 27, 62]
You’ll notice that there are more features than last time, and you should also be both totally fine with extra features being present, and not really minding that you don’t know what they represent. It doesn’t really matter what the things are meaning, we only need to (in a machine learning sense) know that there are features that represent “something”).
Bearing that in mind, we could see the following stack of tensors:
[59, 1, 0, 0, 630000, 0, 4.21, 27, 62, 2] [61, 1, 1, 1, 675000, 1, 5.10, 23, 61, 2] [59, 1, 0, 0, 635000, 2, 4.25, 25, 62, 3] [61, 1, 1, 1, 670000, 0, 4.95, 31, 61, 3] [82, 3, 2, 1, 785000, -2, 9.42, 42, 62, 4] [71, 2, 1, 1, 715000, 5, 6.42, 0, 71, 4] [71, 2, 1, 1, 720000, 5, 6.39, 3, 71, 5]
There’s some things that should stand out to you in that collection:
- The tensors all have the same number of features
- The features are identically set, in that the types are the same (decimal number and integer types are respected through the different tensors)
- There feels like a similar range to each feature, even though you might not know what they are representing
Great. Let’s assume that this series of tensors are representing a block of apartments; If we want to ‘predict’ another apartment in the building, we might take the group of apartments (or in our case, the group of tensors) and pass them into the pretend model we have. (Note: We will be getting to the training approach, later. Just assume we already have our model.)
So, we have a bunch of tensors that actually represent the individual properties in the building, so we are going to take those and then allow the model to predict the most likely next tensor (direction) to the next token (remember, the tensors are directions in the mathematical space, to ‘navigate’ to the location in the space of a token, and the token represents - in our case - an apartment in the real world).
If we want to predict more than one apartment, to get the best responses, we wouldn’t just pass our input (the collection of tensors) a bunch of times and get the output and add the outputs together. The reason for that, is that the output from that, specific, collection of tensors is only the next apartment that is likely from that collection of tensors. Floor numbers are things that are likely in our features, and we can’t suddenly add 50 apartments on one floor.
So, how do we do this? Well, simply put, we pass in our input tensors, and get a predicted token (or specifically the tensor to that token), and then we append (add) that token to the input, and put the whole thing in again. This process repeats over and over again, until we have predicted the number of apartments that we want.
This is the basis of how LLMs process inputs and produce outputs.
You might think that you are asking the model a question when you use an LLM, and it’s certainly framed in that way in the user experience. However, what is ACTUALLY happening is that you are passing in words, that are converted to tokens. These tokens are converted into the tensor “addresses” of where those tokens reside in the multi-dimensional space, and then the next token (specifically the tensor) is predicted. Then this entire group of tensors are fed back into the model, over and over, until the most likely token is a signal to stop looping.
There isn’t thought, or consideration, in an LLM. It’s simply token prediction.
Now, that’s not to belittle the ability of these models to produce coherent and often accurate and useful information; but that is down to the size and quality of the training dataset, as well as the breakdown of tokens in the space, and the number of parameters (from all the attention heads and MLP layers). More data in the training dataset, more parameters, and better token embedding, and the outputs are greatly improved.