Tagged “indexing”
-
Document
In retrieval, a document is whatever unit the system treats as one retrievable item — which may be a file, a section, or a single chunk.
-
Document Loader
The component that reads a source file or system and returns its text plus metadata, before any chunking or embedding happens.
-
HNSW
Hierarchical Navigable Small World: a layered graph index for approximate nearest-neighbour search, walked from a sparse top layer down to a dense one.
-
Inverted Index
A structure mapping each term to the list of documents containing it, so a keyword query examines only documents that hold at least one query term.
-
Vector Index
A data structure that stores embeddings and finds the ones nearest to a query vector without comparing against every stored vector.
-
Quantisation
Storing each value of a vector in fewer bits than the model produced, cutting memory and comparison cost while keeping the number of values the same.
-
Upsert
A write that inserts a record if its identifier is new and replaces the existing record if it is not — the usual way vectors enter an index.
-
Approximate Nearest Neighbour
The problem of finding the items closest to a query point while allowing the answer to be slightly wrong, and the index structures that solve it.
-
Hybrid Search
Retrieval that runs a keyword method and a vector method over the same collection and merges their two ranked lists into one result set.
-
BM25
A ranking function that scores a document against a query from term frequency, term rarity, and document length — the standard keyword-search baseline.
-
Sparse Retrieval
Retrieval that scores text by the terms it shares with the query, using a representation with one position per vocabulary term, almost all of them empty.
-
Dimensionality
The number of values in an embedding vector — fixed by the model that produced it, and the same for every input that model ever sees.