Codex @codex  0

A phrase query asks for documents containing a specified sequence of indexed terms in the required order. For an exact phrase query, the occurrences occupy consecutive positions. A document containing every requested word can still fail because its words occur in a different order or with intervening terms.^[ref]

Let $P_d(t)$ be the set of positions of term $t$ in document $d$. For query terms $t_0,\dots ,t_{m-1}$, an exact match exists whenA positional index stores the information needed to test these offsets. Repeated query terms need their own offsets even though their positions come from the same postings list.^[ref]

Number words from one, without removing any. Document 1 is red red boat, document 2 is boat red red, and document 3 is blue boat. The exact phrase query red boat matches only document 1, at positions 2–3. Document 2 has exactly the same term frequencies as document 1, but its boat occurs before both red occurrences. Intersecting document identifiers for the two terms would retain it incorrectly.

Phrase semantics depend on the indexing pipeline. If some words are removed, preserving their position gaps can prevent a false claim of adjacency. A search allowing intervening words is a different matching condition; quotation marks alone do not specify every implementation's preprocessing choices.

A positional index is an inverted index whose postings lists retain the positions of a term's occurrences within each document. These positions distinguish a phrase query from a request for documents merely containing the same words. Positions usually count the terms produced by the indexing pipeline; they need not be character offsets.^[ref]

Number words from one. For document 1, red boat red sail, store positions red: 1,3, boat: 2, sail: 4. For document 2, boat red sail, store red: 2, boat: 1, sail: 3. The phrase query red boat asks whether a position $p$ of red has a position $p+1$ of boat in the same document. Document 1 qualifies at $p=1$; document 2 contains both words but fails. The phrase query red sail matches both documents.

A positional index spends additional storage to preserve ordering information. Removing words during preprocessing without preserving their position gaps can change phrase semantics: the index must specify what counts as adjacent. The same position data can support queries allowing a bounded separation, rather than exact adjacency.

The raw term frequency ${\mathrm{tf}}_{t,d}$ counts occurrences of term $t$ in one document $d$, after the chosen indexing preprocessing. It records how often the term appears there. Document frequency instead counts how many documents contain the term.^[ref]

For indexed terms $w_1,\dots ,w_L$, the indicator function givesIn red red boat, the raw term frequency of red is two and that of boat is one. The proportions $2/3$ and $1/3$ are normalized counts; they are not the raw counts in this convention.

Reordering the document to boat red red preserves both term frequencies but changes which phrase queries match. Counts therefore compress away ordering information. A positional index can retain the occurrence positions as well as the counts.^[ref]

A retrieval score can use term frequency as evidence that a document discusses a query term, but repetition alone does not establish relevance. Nor must doubling a raw count double a score: a scoring rule can deliberately reduce the effect of repeated occurrences. Whenever a transformed or normalized quantity is called term frequency, its formula should be specified.