Imagine, if you will, that you are in the market for an apartment. Also imagine that the ONLY thing that determines the price of an apartment is the square metres of area (I’m in Australia, so I’m operating in a metric way, but don’t fear if you’re into imperial units, there’s no maths in this).
You see a few apartments:
- 2/12 Smith Avenue
- 72/204 Roberts Street
- 220/92 Thompson Boulevard
- 1/42 Universe Road
These apartments are listed on a website as for sale, and they have an ID number in the database for that site. So let’s now add those in as well.
- 2/12 Smith Avenue, 83627
- 72/204 Roberts Street, 99263
- 220/92 Thompson Boulevard, 13946
- 1/42 Universe Road, 62240
Fantastic. Let’s throw in the square metre values now too, while we are at it.
- 2/12 Smith Avenue, 83627, 59
- 72/204 Roberts Street, 99263, 57
- 220/92 Thompson Boulevard, 13946, 60
- 1/42 Universe Road, 62240, 72
And now the prices (we can say that the currency is dollars, but it’s not really important).
- 2/12 Smith Avenue, 83627, 59, 630000
- 72/204 Roberts Street, 99263, 57, 595000
- 220/92 Thompson Boulevard, 13946, 60, 650000
- 1/42 Universe Road, 62240, 72, 750000
Now, before we move any further into building out the features within each apartment line, let’s talk about what we have so far.
- Physical Address: This is the real world label or title we, as people, would give to the premises in question.
- ID: This is the digital label or title that the server, database, and/or software gives to the premises in question.
- Area: an integer (whole number) representing the floor area of the premises in question.
- Price: The amount of money required to purchase the premises.
So, we can say that “2/12 Smith Avenue” is the same as “83627”. They are BOTH the individual label/name for that location in the real world. A non-string representation tends to work better for handling large amounts of data in computers, so we can look at this in the same way an LLM has tokens and representations of those tokens.
In a simple way, a token in an LLM is either a word, or part of a word, or punctuation, or an “end of message” signal. Except these tokens are then represented as numerical values within the processing (the inference).
If an apartment in our list has a parking space in the building, we know that the parking space isn’t at the apartment location, but we also know that it’s connected (in an ownership sense) to the address. Therefore the ID for the property is representing the ENTIRE ownership proposition, whereas the address string (like ‘2/12 Smith Avenue’) is only representing the physical apartment itself.
The two “features” of each property (at this point) are the square metres and the price. For the 2/12 Smith Avenue (ID 83627) property, we can see that it has 59sqm, and costs 630000. Let’s put them into an array, or list, and see how that works for us… [59, 630000] Great.
Now, we can imagine that we have a simple two-dimensional chart, with sqm on the bottom, and price up the side. Using this chart concept, we can visualise that at 59 along the X (horizontal) axis, and at 630,000 along the Y axis (the vertical), we can imagine that there is a dot with the ID of 83627.
We can say that [59, 630000] is the way to arrive at the ID in the chart. 59 along, and 630000 up.
This array of [59, 630000] is path or direction to get to the apartment in that chart.
We can say that [59, 630000] is the vector to get to the point in the two-dimensional space of the chart. This vector is a Rank-1 Tensor and results in the location of the apartment, in the mathematical space.
That’s a lot for someone new to the topic, so let’s pause before we move into the next section; Multidimensional spaces (much more than the three dimensions of the real world) and how tensors can play into understanding how the LLMs that you use are generating their output.