What is an embedding?

How a computer can turn a word into a position, so that words with similar meanings end up near each other.

Press play, or drag a word.

Animals (circles)
Vehicles (squares)
Fruit (triangles)

Start

Ten words as points on a flat plane. Press play to watch them move, or drag any word.

Text description

Ten words (cat, kitten, dog, puppy, car, truck, bicycle, apple, banana, cherry) start scattered at random places on a plane. As the animation plays, they drift into three groups: animals, vehicles and fruit. Lines then connect a chosen word to its two nearest neighbours, which are always words from the same group. You can move any word with a mouse, a finger or the arrow keys, and the lines update to show its new nearest neighbours.

What you just saw

Placeholder. The real lesson will explain what the learner just watched: each word became a point, and words used in similar ways ended up close together.

Closeness as a number

Placeholder. Introduce distance only now that the learner has seen it.

The distance between two points on a plane is

d = \sqrt{(x_2 - x_1)^2 + (y_2 - y_1)^2}

where $d$ is the distance, $(x_1, y_1)$ is the position of the first word, and $(x_2, y_2)$ is the position of the second.

What this picture leaves out

Real embeddings have far more than two dimensions. Hundreds or thousands is typical. Two dimensions fit on a screen, but they can’t hold much meaning.
The positions here are placed by hand. Real embeddings are learned from large amounts of text, and nobody chooses where each word goes.
Models usually embed pieces of words, not whole words. These pieces are called tokens.

Sources

No sources listed yet.

How this lesson was made

Stub created to test the lesson structure. The text and visual are placeholders and have not been checked yet.

Tools used:

Claude (Anthropic): Scaffolding the lesson structure and placeholder visual.

Written by Alex Zhao. Not yet reviewed. Last updated 26 October 2026.