OpenSearch:Indexes

OpenSearch builds indexes to speed up data retrieval. Each document consists of multiple fields, and each field contains a series of terms. OpenSearch supports three index types based on how terms and documents are mapped:

• Inverted index — maps terms to the documents that contain them (term → DocID list). Use an inverted index for full-text search and keyword retrieval: given a query keyword, the index returns all matching documents without scanning the entire dataset.

  For example, for a dataset with two documents:

  • Document 1: "the quick brown fox"

  • Document 2: "the lazy brown dog"

  The inverted index maps each term to the documents it appears in:

  Term	Documents
  the	Doc1, Doc2
  quick	Doc1
  brown	Doc1, Doc2
  fox	Doc1
  lazy	Doc2
  dog	Doc2

  Given the query "brown", the inverted index immediately returns Doc1 and Doc2 without scanning all documents.

  In OpenSearch schema configuration, inverted index fields are referred to as index fields.

• Forward index — maps a DocID to the field values of that document (DocID → field values). After a query identifies matching documents via the inverted index, the forward index efficiently retrieves field values for those documents by DocID. Use a forward index for post-retrieval operations: statistics, sorting, and filtering.

  Forward index fields are referred to as attribute fields in OpenSearch schema configuration. Attributes come in two variants:

  • Single-value attribute: holds one value per document. Has a fixed storage length (except for the STRING type), which enables high lookup efficiency and supports updates.

  • Multi-value attribute: holds a variable number of values per document. Because its length is not fixed, lookup efficiency is lower than a single-value attribute and updates are not supported.

  The DPI engine supports the following primitive data types for forward index fields:

  Type	Description
  INT8	8-bit signed integer
  UINT8	8-bit unsigned integer
  INT16	16-bit signed integer
  UINT16	16-bit unsigned integer
  INTEGER	32-bit signed integer
  UINT32	32-bit unsigned integer
  INT64	64-bit signed integer
  UINT64	64-bit unsigned integer
  FLOAT	32-bit floating-point number
  DOUBLE	64-bit floating-point number
  STRING	String type

  Each single-value type has a corresponding multi-value variant. For example, multi_int8 is the multi-value variant of INT8, and multi_string is the multi-value variant of STRING.

• Summary — stores a snapshot of multiple fields from each document, mapped by DocID. A summary is similar in structure to a forward index, but is designed to hold multiple fields at once for display purposes rather than for filtering or sorting.

  Summaries are retrieved only for the final set of documents to be displayed — not during the main retrieval phase — because fetching many large summaries per query is inefficient.

  To reduce storage size, the DPI engine supports summary compression. When summary compression is configured in the schema, the engine compresses summary data using zlib before storing it and decompresses it on read before returning results.