Vector Normalisation

Vector normalisation is the operation of rescaling a vector so that its length is exactly one, dividing every component by the vector’s magnitude and leaving its direction unchanged.

The magnitude used is almost always the Euclidean length — the square root of the sum of the squared components — and the result is described as a unit vector, or as L2-normalised after the name of that length. Every normalised vector in a space lies on the surface of a sphere of radius one.

Direction is preserved and magnitude is discarded. Two vectors that point the same way and differ only in length become identical after normalisation. Whether that loses information depends on whether magnitude carried any, and in most embedding models it carries little that is useful for comparing meanings.

The operation’s practical consequence is an equivalence between two similarity functions. Cosine similarity is the dot product divided by the two magnitudes; when both magnitudes are one, that division has no effect, so the dot product and the cosine are the same number. Ranking by either produces the same order.

Euclidean distance is not equal to either, but on normalised vectors it becomes a strictly decreasing function of the dot product, so ranking by nearest Euclidean distance also yields the same order. On unit vectors the three common metrics agree about ordering while disagreeing about values.

In practice

Normalisation appears at two points, and which one applies is often unstated. Some embedding models emit unit vectors already, having normalised inside the model; some do not, leaving the caller to do it or to use a metric that compensates. Nothing in a vector’s appearance reveals which, short of computing its length.

Where an index is configured for a metric, that configuration interacts with normalisation. An index set to dot product over unnormalised vectors ranks partly by magnitude, so longer vectors are favoured regardless of direction. The same index over normalised vectors ranks by direction alone. The configuration and the data therefore have to be considered together; neither is meaningful in isolation. See cosine similarity.

The reason to prefer dot product over cosine on normalised data is cost. Dot product is a multiply-and-add over the dimensions with no square roots and no division, and it is the operation vector indexes are built to execute quickly. Normalising once at write time moves work out of every subsequent query.

Consistency between write and read is what the operation requires. Vectors normalised at ingestion and a query left unnormalised will still rank correctly under cosine, which normalises both anyway, but not under dot product, where the query’s magnitude scales every score identically and thus harmlessly — while a mixture of normalised and unnormalised stored vectors does corrupt the ranking, permanently and without any error. See upsert.

Quantisation is applied after normalisation where both are used, because the value range of unit-vector components is bounded and predictable, which is what makes a fixed quantisation scheme viable. See quantisation.

Commonly confused with

Cosine similarity. A similarity function, not an operation on a vector. Cosine similarity normalises implicitly as part of its formula; vector normalisation does it once and stores the result. They produce the same rankings and are not the same thing, and “we use cosine” and “we normalise” describe different implementation choices. See cosine similarity.

Text normalisation. An unrelated operation on strings — lowercasing, stripping accents, collapsing whitespace — performed before tokenisation. Both are called normalisation and they occur at different ends of the pipeline. Context distinguishes them; the word does not. See token.

Standardisation. Subtracting a mean and dividing by a standard deviation, per dimension across a dataset. It changes direction and does not produce unit length. The two are both called normalisation in general machine learning writing, where the ambiguity is long-established.

Quantisation. Reducing the precision of each component. Normalisation changes the values without changing their count or precision; quantisation changes precision without changing direction meaningfully. They compose and are frequently applied together. See quantisation.

Dimensionality reduction. Fewer components. Normalisation preserves the component count exactly. See dimensionality.

Usage notes

“Normalised” without qualification is ambiguous in any text that also discusses text processing, and in machine-learning writing generally, where it may mean standardisation. L2-normalised is the unambiguous term for the unit-length sense.

Whether a model’s outputs are already normalised is a documented property that is frequently not documented. Material describing an embedding model commonly omits it, and the answer changes between model versions.

Claims that one metric is better than another are usually claims about unnormalised data. On unit vectors, cosine, dot product, and Euclidean distance produce identical rankings, so a comparison between them is describing either a performance difference or data that is not normalised. See approximate nearest neighbour.

Normalising vectors that were trained to carry information in their magnitude discards it. Sparse and learned-sparse representations, where a component’s weight is the signal, are the usual case. Dense sentence embeddings generally are not. See sparse retrieval.

A zero vector cannot be normalised — its magnitude is zero and the division is undefined. Empty or unparseable text occasionally produces one, and implementations differ in whether they raise an error, return zeros, or store the vector unchanged.

See also

Cosine similarity · Embedding · Dimensionality · Quantisation · Vector index