Cross-Encoder
A cross-encoder is a model that takes a query and a document together as a single joined input and outputs one number scoring their relevance, rather than encoding each separately into a vector.
The word cross names what the arrangement permits: the model can relate every part of the query to every part of the document while it computes. A phrase in the query and the sentence that answers it are present in the same pass, so the score reflects their interaction rather than a comparison of two summaries made in ignorance of each other.
The output is a score and nothing else. A cross-encoder produces no reusable representation, so there is nothing to store and nothing to index. Its answer exists only for the pair it was given.
That is why the arrangement cannot search. Scoring a query against a million documents means a million model passes, which is not a slower version of nearest-neighbour search but a different order of cost entirely. A cross-encoder is therefore applied to a candidate list that something cheaper has already produced.
The term comes from the sentence-embedding literature, where it is defined by contrast with the bi-encoder. Product documentation usually calls the same component a reranker.
In practice
A cross-encoder occupies the second stage of a two-stage retrieval pipeline. A first-stage retriever returns candidates; the cross-encoder scores each one against the query; the list is reordered by those scores and truncated.
Its cost profile is the property that governs how it is used. Work scales linearly with the number of candidates, because each pair is an independent model pass, and none of it can be precomputed or cached across queries — the query changes every time, and the query is half the input. The candidate count is consequently the only substantial lever on its latency.
Input length matters more than for a bi-encoder, because the query and the document share one input budget. A long chunk may be truncated to fit, and what is cut is decided by the implementation rather than by relevance.
The scores it emits are not comparable with the similarity scores from the first stage. A cross-encoder’s output range depends on how it was trained, and some models emit values on an unbounded scale while others are trained to produce something resembling a probability. Ordering is what the score is for.
Because the model reads the pair, it can distinguish cases that a vector comparison cannot: a passage that discusses the query’s subject without answering it, and one that answers it. This is the accuracy the arrangement buys. See reranking.
Commonly confused with
Bi-encoder. The complementary arrangement, encoding query and document separately so that document vectors can be computed in advance. The bi-encoder searches; the cross-encoder rescores. Every distinction between them follows from whether the two texts meet inside the model. See bi-encoder.
Reranker. The role rather than the architecture. A cross-encoder is the usual implementation of a reranker, but reordering by metadata rules, by rank fusion, or by prompting a general-purpose language model is also reranking and involves no cross-encoder.
Embedding model. A cross-encoder is not one. It emits no vector, so its output cannot be stored, indexed, or compared with anything else. Asking a cross-encoder to embed a document is a category error the interface usually prevents.
Late-interaction models. An arrangement between the two: the document is encoded in advance, but as many vectors rather than one, and the interaction with the query happens at query time over those vectors. Precomputable like a bi-encoder, more expressive at query time, and not a cross-encoder, since the two texts are still never processed jointly.
Language-model scoring. Prompting a general instruction-following model to judge relevance also reads query and document together, and is sometimes called cross-encoding by analogy. The mechanism differs — a prompt and generated output rather than a model trained to emit a score — and so do the cost and the reliability.
Usage notes
The word is rare outside comparisons. It exists to mark the contrast with the bi-encoder, so a text using it is almost always explaining why a pipeline has two stages.
“Cross-encoder” and “reranker” are used interchangeably and should not be. The first is an architecture and the second is a position in a pipeline. Material describing “the cross-encoder step” may be describing a heuristic reordering.
Scores are not calibrated and do not transfer between models. A value from one cross-encoder means nothing relative to the same value from another, and a threshold set against one model has to be re-established for any replacement. See cosine similarity.
It cannot recover what retrieval missed. A cross-encoder only reorders the candidates it receives. Its accuracy advantage applies to the ordering of that list and to nothing outside it. See top-k.