Top-k

Top-k is the highest-scoring k items from a ranked result set, and by extension the setting that fixes how many results a retrieval step returns.

The letter k is a placeholder for that count, borrowed from the nearest-neighbour literature where the task is stated as “find the k nearest points.” In a retrieval system it appears as a parameter named k, top_k, limit, or n_results depending on the library.

The essential property is that top-k is a count, not a quality bar. A search asked for five results returns five whenever the collection holds at least five items, regardless of how well any of them match. Nothing in the operation can return three because only three were relevant, or zero because none were.

That makes it a fixed-size window onto an ordering. It answers “which are the best available” and never “which are good.”

The same request is often described as retrieving “the top five chunks” or “k equals five,” and both name the same setting.

In practice

A retrieval pipeline usually has two or three separate values of k, and confusing them is the most common problem the term causes.

Retrieval k — how many candidates come out of the index. Set high when a reranking pass follows, because the reranker can only reorder what it is given.

Rerank k — how many survive the second scoring pass and reach the prompt. Usually much smaller.

Index-internal k — some indexes are asked for more neighbours than requested and trim afterwards, particularly when metadata filtering may discard some.

The value interacts with the context window rather than existing independently. Each returned chunk consumes tokens, so k multiplied by chunk size is a substantial part of the prompt’s size, and raising k spends context and money per request. See context window.

Raising k has an asymmetric effect on retrieval quality. It can only increase the chance that a relevant chunk is present, since a larger window over the same ordering never drops items. It simultaneously increases the amount of irrelevant material placed in front of the model. Which effect dominates is a property of the corpus and cannot be inferred from the number.

For nearest-neighbour search specifically, k also affects cost inside the index: approximate structures do more work to return more neighbours, and their accuracy at large k is not the same as at small k.

Commonly confused with

Top-k sampling. An unrelated setting in text generation, restricting the model’s next-token choice to the k most probable tokens. Same name, same letter, different subsystem: retrieval top-k selects documents, sampling top-k selects tokens. Both may be configured in one application, and confusing them is easy because both are called “top-k.”

Top-p. Also a generation setting — nucleus sampling, keeping tokens until their cumulative probability reaches p. Named to parallel top-k sampling, and likewise nothing to do with retrieval.

Recall@k. A measurement rather than a setting: how often the relevant item appears within the top k results. The k is the same k; recall@k evaluates a choice of k rather than making one.

Similarity threshold. The alternative shape of the same decision — return everything scoring above a cut-off, however many that is. A threshold can return nothing, which top-k cannot. Thresholds require calibrated scores, which is why top-k is the more common default. See cosine similarity.

Page size. In a search interface, how many results are displayed per page — a presentation limit applied after retrieval, not the retrieval count itself.

Usage notes

Round default values are conventions, not findings. Values like 3, 5, and 10 appear in tutorials because they are legible, and get copied into systems as though tuned. The appropriate k depends on chunk size, corpus redundancy, and how much context the generation step can spend.

A low k hides retrieval problems and a high k hides them differently. With k too small, a system that retrieves the right chunk in fourth place looks broken. With k too large, the right chunk arrives buried among near-duplicates and the model may use a worse one. The number chosen shapes which failure is visible.

“Top-k” implies the ordering is trustworthy — that position two is better than position twenty. With approximate search the ordering is itself approximate, and with uncalibrated scores the differences near the tail can be noise. See approximate nearest neighbour.

Some APIs cap k below what a caller may request, silently returning fewer results. Others treat the parameter as a hint. Whether the returned count is guaranteed is implementation-specific.

See also

Reranking · Approximate nearest neighbour · Context window · Retriever