An inverted index — also called a postings file or an inverted file — maps terms to the documents that contain them, enabling fast full-text search across large document sets. In OpenSearch Retrieval Engine Edition, inverted indexes are the core data structure powering document retrieval — they let the engine quickly find which documents contain a given term, where the term appears, and how frequently.
How an inverted index works in retrieval
When you submit a query, the engine performs the following steps:
Looks up the term in the dictionary to find the starting position of its entry in the postings file.
Parses the posting list to extract three components:
TermMeta — term-level statistics: document frequency (df), total term frequency (ttf), and term payload (termpayload).
DocList — per-document data: document ID (docid), term frequency in that document (tf), document payload (docpayload), and the fields where the term appears (fieldmap).
PositionList — position-level data: the exact offset of the term within each document (position) and position payload (positionpayload).
Index structures
An inverted index is built from the following structures:
| Structure | Full name | Description |
|---|---|---|
dictionary | — | Maps terms to their posting lists. The engine queries the dictionary to locate a term's starting position in the postings file. |
doclist | Document list | Stores per-document data for each term, including docid, tf, docpayload, and fieldmap. |
positionlist | Position list | Stores the position of each term within each document, along with positionpayload. |
truncatelist | Truncate list | A selective inverted index built only for high-quality documents based on the configuration that you specify, improving retrieval performance. |
bitmap | — | Stores certain inverted structures as bitmaps based on the configuration that you specify. Reduces index size and improves retrieval performance. |
Stored data items
Each entry in an inverted index can carry the following data items:
| Item | Full name | Scope | Description |
|---|---|---|---|
ttf | Total term frequency | Term | Total number of times the term appears across all documents |
df | Document frequency | Term | Number of documents that contain the term |
termpayload | Term payload | Term | Custom metadata attached to a term; available for downstream processing |
docid | Document ID | Document | Unique identifier of a document in the engine; used to retrieve other document information |
tf | Term frequency | Document | Number of times the term appears in a single document |
docpayload | Document payload | Document | Custom metadata attached to a document; available for downstream processing |
fieldmap | Field map | Document | Records which fields in a document contain the term |
| Section information | — | Document | Optional per-section metadata for documents divided into sections; available for downstream processing |
position | — | Position | Exact offset of the term within the document |
positionpayload | Position payload | Position | Custom metadata attached to a specific term position; available for downstream processing |