Dimensionality
Dimensionality is the number of values in a vector — for an embedding, a count fixed by the model that produced it and identical for every input that model is ever given.
A model that emits 768 numbers per input emits 768 for a single word and 768 for a full page. The count is a property of the model’s output layer, not of the text. This is what makes embeddings comparable at all: every vector in a collection has the same length, so the same distance calculation applies to all of them.
Each position in the vector is called a dimension. Individually the dimensions mean nothing readable. There is no dimension for topic and none for sentiment; meaning is spread across all of them jointly, which is why a vector cannot be interpreted by reading its components.
The word carries its ordinary geometric sense. A 768-dimension embedding is a point in a 768-dimensional space, and phrases like “near,” “far,” and “direction” apply there in the same formal way they apply in three dimensions — while human spatial intuition about them stops being reliable almost immediately.
Dimensionality is also used loosely as shorthand for the size of an embedding, since storage and search cost both scale with it.
In practice
Dimensionality determines three costs at once.
Storage. A vector’s raw size is its dimension count times the bytes per value. Halving the dimensions halves the memory before any other consideration, which is why the number appears in every index-sizing calculation.
Search work. Comparing two vectors touches every dimension, so the cost of a single similarity computation is linear in the count. Search algorithms reduce how many comparisons happen, not how expensive each one is.
Index overhead. Graph and cluster structures store their own bookkeeping on top of the vectors, and its relative weight depends on how large each vector is to begin with.
Commonly encountered counts run from a few hundred to a few thousand, with certain values recurring because particular model families settled on them. Nothing about the field requires a round number; it is a design choice inherited from the architecture.
Some models are trained so that a prefix of the vector remains usable on its own — the first few hundred values can be kept and the rest discarded, giving a shorter embedding at reduced accuracy from the same model pass. Where this is supported, dimensionality becomes a setting the system chooses rather than a fixed property of the model, and truncated vectors must be compared only against other vectors truncated the same way.
Commonly confused with
Model quality. More dimensions does not mean a better model. A compact model trained well can outperform a larger-dimension one on retrieval, and the count says nothing about training data, domain fit, or supported input length. The correlation is weak enough that dimensionality should be read as a cost figure, not a quality figure.
Context window. The context window is how much text a generative model can process; dimensionality is how many numbers an embedding model outputs. Both are limits attached to models and neither constrains the other. See context window.
Input length. The maximum text an embedding model accepts is a separate specification and a far more consequential one for chunking. Text beyond it is commonly truncated, sometimes without warning. A high-dimension model may still have a short input limit.
Quantisation. Quantisation reduces the bits per value while keeping the count of values the same. Truncation reduces the count while keeping the precision. Both shrink the vector; they are different operations and can be applied together. See quantisation.
Sparse vector length. A sparse representation may nominally have tens of thousands of dimensions while almost all are zero. Quoting that number alongside a dense model’s few hundred compares unlike things — dense dimensionality counts values that are all in use. See sparse retrieval.
Usage notes
“Dimension” and “dimensionality” are used interchangeably for the count, and “dims” is the usual abbreviation in code and configuration. A field named dimension almost always holds the count rather than an index into the vector.
The count is usually a hard constraint on the index. Most vector indexes are created with a fixed dimension and reject vectors of any other length, which means changing embedding model to one of a different size requires a new index rather than a migration.
High-dimensional geometry behaves counter-intuitively. As dimensions increase, distances between random points concentrate into a narrow band, so raw distance values become less discriminating. This is one reason absolute similarity scores travel badly between models. See cosine similarity.
“Dimensionality reduction” is a distinct technique from the truncation described above — a separate transformation fitted to a collection, of the kind used for visualisation or compression. Applying it to embeddings changes the space, so reduced vectors are comparable only with others reduced by the same fitted transformation.