Document write, read, and query operations - Elasticsearch

A Collection follows the main calling conventions of the standard Elasticsearch document and query APIs and uses _slice to specify the scope of business data. The most common operations in application development are writing and reading documents, controlling query visibility, and specifying the query scope. The access method table in the following section lists each supported access method and its scenarios.

Write and read documents

Access method	Scenarios
Single-document API	Real-time create, read, update, and delete operations when the document ID and its Slice are known.
Bulk	Write logs or knowledge base content in bulk, or write to multiple Slices in one batch.
MGet	Read documents from the same or different Slices in one request when multiple document IDs are known.
Reindex	Data migration between a regular index and a Collection, or between different Collections.

Single-document APIs:

Operation	API
Write or overwrite a document	`PUT /{collection}/_doc/{id}?_slice={slice}`
Automatically generate a document ID	`POST /{collection}/_doc?_slice={slice}`
Create a document only	`PUT /{collection}/_create/{id}?_slice={slice}`
Get a document	`GET /{collection}/_doc/{id}?_slice={slice}`
Check whether a document exists	`HEAD /{collection}/_doc/{id}?_slice={slice}`
Get `_source`	`GET /{collection}/_source/{id}?_slice={slice}`
Update a document	`POST /{collection}/_update/{id}?_slice={slice}`
Delete a document	`DELETE /{collection}/_doc/{id}?_slice={slice}`

Follow these rules:

A single-document request can specify only one Slice. You cannot use a comma-separated list or _all.
index, create, or update can automatically register a nonexistent Slice when auto_create_slice=true is set. Read and delete operations do not register Slices automatically, and specifying a nonexistent Slice returns HTTP 404.
A document is uniquely located by the combination of the Collection, the business Slice name, and _id. Different Slices can use the same _id. When an application reads, updates, or deletes a document, it must specify the corresponding Slice. Do not distinguish tenants by _id alone.
Bulk requests are subject to a separate Slice auto-registration quota, described in the Bulk section.
wait_for_active_shardsis not used as a condition for waiting for replicas in the Agent engine write path. Background refresh provides query visibility in most cases. If data must be searchable before a write request returns, use refresh=wait_for under the applicable conditions described later.
Vector values are not returned by default. Regular read and search operations return only the fields that your business requires. To verify or export vector values, see Read vector values on demand.
In a response, _index and _routing may return the underlying index name and the system-generated internal Slice identifier (slice_uid) instead of the Collection name and the business Slice name used in the request. Continue to use the business names in subsequent requests, and do not pass these internal values back. For more information, see Request targets and response identifiers.

Refresh and query visibility

For regular writes, do not passrefresh. Wait for the automatic background refresh. A successful write means that the data is persisted. Reading a document by ID and searching by condition have different visibility behavior. The following sections describe how to choose the read method and the refresh behavior that match your scenario.

Confirm a write by ID without a refresh

To confirm only the document that you just wrote, use GET or MGet. Both enable real-time reads by default (realtime=true), so you do not need to pass the parameter explicitly or trigger a manual refresh. For example, after a write request succeeds, read the document with the same Slice and document ID:

GET /knowledge-chunks/_doc/chunk-1001?_slice=tenant-a

Check the returned found value and the business fields. For MGet, check each result. Real-time reads are not a strong consistency guarantee during every failure window. If the primary shard is unavailable, or if migration or a restart temporarily makes the real-time read source unavailable, GET and MGet may fall back to the committed data view. They then return an earlier version, or return found=false for a document that has not yet entered that view. In this case, judge the result from the write result and the cluster status, and re-check after a limited number of retries. Do not conclude that a write was lost based on a single not-found response.

Search by condition and wait for the automatic background refresh

Search, Count, and KNN queries must wait until the data is searchable on query nodes. The Agent engine version has a default automatic refresh interval of5s, and you do not need to configure it when you create a regular index or a Collection. The default for traditional stateful Elasticsearch is 1s. The actual time until data becomes searchable is also affected by data publication, query node loading, and workload. 5s is not a guaranteed visibility deadline.

Use GET /{collection}/_settings to view only the settings that you explicitly configured, and add include_defaults=true to also return default values. Use PUT /{collection}/_settings to adjust the setting. Keep automatic refresh enabled for production workloads. Set the interval to at least5s. A shorter interval increases data publication on index nodes, object storage I/O, and refresh overhead on query nodes. To control the write workload, increase the interval as appropriate and evaluate the resulting data visibility latency.

Use refresh=wait_for only in limited scenarios

refresh=wait_for is intended only for one-off, low-frequency business synchronization points. Use it only when all of the following conditions are met: (1) the batch must become searchable before processing continues, (2) the parameter is used with a Bulk batch, (3) concurrent waits are limited, (4) automatic refresh remains enabled, and (5) you have evaluated the impact of the wait time on business latency and throughput. To confirm a write by ID, use GET or MGet. Do not pass this parameter for continuous imports or regular writes.

This parameter waits for a refresh and coordinates visibility on query replicas. Too many concurrent waits or exhausted refresh listeners may also trigger a forced refresh, which you can identify by forced_refresh: true in the response. Even when a forced refresh is triggered, the request may continue to wait for query-side visibility and is not guaranteed to return immediately. A client timeout does not mean that the write did not occur. Do not increase concurrent waits to speed up a refresh.

Use manual refresh only for isolated verification or an O&M window

Do not addrefresh=trueto an application code path or call_refresh. Do not refresh automatically after each import batch.refresh=true triggers an immediate distributed refresh, which increases data publication, object storage I/O, and refresh overhead on query nodes. Automatic background refresh is the standard alternative.

POST /{collection}/_refresh affects all Backings and all tenants of the entire Collection. You cannot estimate its overhead from the data volume of your own Slice, and the impact of the same request increases as the number of Backings and the data volume increase. You cannot narrow the scope with _slice. Passing _slice to this endpoint returns HTTP 400. A Collection does not provide a single-Backing refresh endpoint for application calls.

Use manual refresh only for a separately created verification Collection that contains a small amount of demonstration data, or during a low-frequency O&M window that an administrator schedules after evaluating the size of the entire Collection. The one-off offline import described in the following section is one such window. After the refresh, check _shards.failed. Regular online imports must wait for the automatic background refresh. If you need to prewarm data, do it after the data becomes searchable.

Disable automatic refresh only for a one-off offline import

Do not set index.refresh_interval=-1 for a production Collection. This setting disables automatic refresh, so searches do not see newly written data in time. When you combine it with refresh=wait_for, a request may wait for a long time or even time out.

The only exception is a clearly defined one-off offline import window for the entire target Collection, during which none of the associated tenants run queries. Within that window, complete the following steps in order: (1) Omit refresh during the import. (2) After the import completes, restore 5s or the previous interval of at least 5s by calling PUT /{collection}/_settings. (3) Still within the same window, have the import task run _refresh once, and then check _shards.failed in the response. (4) Verify that queries return the imported data. (5) If the workload is sensitive to cold query latency, prewarm the data before you restore production traffic. Do not use this process for a Collection that still serves other tenants.

A refresh addresses only query visibility. It does not mean that all query replicas are available, and it cannot replace query probes or high availability (HA) checks.

Bulk

Each Bulk item can provide its own _slice in the action metadata. For an example, see the bulk write example in Getting Started. If an entire batch is written to the same Slice, you can also provide a default value in the URL:

POST /knowledge-chunks/_bulk?_slice=tenant-a
{ "index": { "_id": "chunk-2001" } }
{ "tenant_id": "tenant-a", "document_id": "doc-account-security", "title": "Account security settings", "content": "An administrator can enable multi-factor authentication on the security settings page." }

The _slice value in an item can override the default value in the URL. Each Bulk request automatically registers up to 512 missing Slices. Existing Slices do not count toward this limit. Namespace mode does not derive a Slice from document fields. Even if the document body contains fields such as tenant_id, the actual write location is determined by _slice. Check the returned result for each item to confirm that the write succeeded. For error handling, see General API conventions.

MGet

Use MGet to read multiple documents with known IDs in one request. Each item can target a different Slice:

POST /knowledge-chunks/_mget
{
  "docs": [
    { "_id": "chunk-1001", "_slice": "tenant-a" },
    { "_id": "chunk-1001", "_slice": "tenant-b" }
  ]
}

These two items read documents with the same ID from different tenants. With the sample data from Getting Started, the returned titles are "Refund and return policy" and "Track shipment status". When you query your own data, base the result on found, the business fields, and the errors of each item. The _slice in the URL can serve as a default value that each item can override. Each item must resolve to exactly one Slice.

Reindex

Before you call Reindex, check the API support scope for Collection Reindex, especially the restrictions on scripts, destination pipelines, and remote sources.

Use Reindex when you migrate data between a regular index and a Collection or between Collections. To copy a Slice within the same Collection, use Copy Slice.

When the source is a Collection, you must explicitly specify source._slice in the request body. Otherwise, the request returns HTTP 400. When the destination is a Collection, you can specify only one destination Slice, and the destination value must use the ={slice} format.

Before you run the task, create the destination Collection, configure a Mapping that can accept the source documents, and follow Pre-create Backings to prepare capacity and register the destination Slice. Make sure that the source data to migrate is searchable. Reindex does not continuously synchronize subsequent writes, so you must also process incremental data before the production cutover.

By default, Reindex preserves the document _id and may overwrite a document with the same ID at the destination. Before you merge multiple source Slices into one destination Slice, make sure that document IDs are also unique across those Slices. Otherwise, migrate them to different destination Slices, or have the application generate new unique IDs before importing the documents.

The following example uses asynchronous execution, a single task, 500 documents per batch, and a target rate of 500 documents per second. Start with these settings to control the impact of migration on online workloads, and then adjust them based on the workload. requests_per_second throttles by document count between batches, not by the number of HTTP requests per second:

POST /_reindex?wait_for_completion=false&slices=1&requests_per_second=500
{
  "source": {
    "index": "source-knowledge-chunks",
    "_slice": "tenant-a",
    "size": 500
  },
  "dest": {
    "index": "target-knowledge-chunks",
    "_slice": "=tenant-archive"
  }
}

Save the task value from the response, then poll GET /_tasks/{task_id} at intervals until completed=true. Check error, failures, and version_conflicts in the result. Task completion does not mean that all destination data is searchable. Before the cutover, wait for a background refresh, verify the destination results, and prewarm the Slices that will be accessed.

Note the following rules:

source._slice supports one Slice, multiple Slices, or _all. The Reindex request body does not accept source.routing, and passing it returns HTTP 400.
The destination can use "_slice": "=slice-name" or "routing": "=slice-name", but not both. The leading = must be retained.
If the destination Slice does not exist, it is registered automatically only when auto_create_slice=true is set.
The top-level slices parameter specifies the parallelism of Elasticsearch Reindex and is unrelated to the number of business Slices.
When writing from a Collection to a regular index, if routing does not need to be retained, set "dest": { "routing": "discard" }.

Specify the query scope

This section describes the Slice query scope that the Agent engine version adds. Other endpoint parameters follow standard Elasticsearch API semantics. For known limitations, see the API support scope.

Query scope	Scenarios	Example
One Slice	Online queries for one tenant; searches within a specified namespace	`GET /knowledge-chunks/_search?_slice=tenant-a`
Multiple Slices	Aggregate queries for a small number of known tenants; queries across a small number of namespaces	`GET /knowledge-chunks/_search?_slice=tenant-a,tenant-b`

A single query can specify up to 1,024 Slices. The HTTP request line length limit also applies. With long Slice names, the request may return too_long_http_line_exception before it reaches the count limit. Shorten the Slice names or split the request into multiple queries.

Collection queries enable query and prewarming affinity by default. While this setting is enabled, the affinity rule overrides preference in the request, and the response gives no additional notice. To use preference to control node selection, an administrator must disable the setting. The setting is cluster-wide, so disabling it applies to all Collections.

For production queries, do not use_slice=_all. This parameter searches the entire Collection, and the overhead of shard requests, aggregations, and object storage reads on query nodes increases with the number of Backings and shards. Applications must specify one known Slice or a small number of known Slices. Use GET /knowledge-chunks/_search?_slice=_all only for dedicated analysis, audit, or O&M tasks that genuinely require all data, and only after you evaluate the load on the entire Collection, limit concurrency, and set a timeout.

If you use Branch Slice (Beta) during the Beta phase, multi-Slice queries and _slice=_all return HTTP 409 when the scope includes a Branch that has not yet been converted to a regular Slice. Workloads that do not use this feature are unaffected.

When a query list may include Slices that do not exist, use ignore_missing_slice:

GET /knowledge-chunks/_search?_slice=tenant-a,tenant-b&ignore_missing_slice=true

For an optional Slice that may not exist, set ignore_missing_slice=true to avoid retrying an entire batch because one Slice is missing. Without this parameter, the entire request returns HTTP 404 if any Slice does not exist. This parameter affects only named Slices. When you use it with _slice=_all, the parameter is ignored without an error.

The following query endpoints support the same Slice scope parameter:

Search and Count
Multi search
Search template
Async search
Validate query
Search shards
Update by query and Delete by query
Explain describes why a document matches a query. You can specify only one Slice. You cannot use a comma-separated list or _all:

POST /knowledge-chunks/_explain/chunk-1001?_slice=tenant-a
{
  "query": {
    "match": { "content": "refund" }
  }
}

For Update by query, Delete by query, and Reindex operations that target a Collection, you must explicitly specify the target Slice. Otherwise, the request is rejected with HTTP 400.

Default behavior when _slice is not specified

In namespace mode (exact), Search, Count, Search Template, Async Search, Validate Query, and Search Shards require _slice by default. The request fails when this parameter is missing. For the small number of dedicated tasks that meet the full-query conditions described earlier, pass _slice=_all explicitly in those requests instead of changing the global default behavior. Vector-cluster recall mode can automatically select candidate Slices based on the query vector. For more information, see Vector-cluster recall mode.

Keep the full-query fallback disabled by default. The cluster setting apack.slice_collection.search.default_to_all_slices controls this fallback, and the setting affects all Collections. In namespace mode, when _slice is not specified, the query falls back to all Slices. Vector-cluster mode falls back to a full query only when no scope is specified and the configuration path does not contain a query vector; when a valid query vector is provided, candidate Slices are still selected normally. A full-query fallback increases the overhead of shard requests, aggregations, and object storage reads.

A dedicated task that genuinely requires a full query must specify the scope explicitly in each request. Do not rely on this global switch to guarantee a full query. Enable this cluster-wide switch only when the application explicitly depends on full-query fallback, an administrator has evaluated the load on all affected Collections, and concurrency and timeout controls are in place:

PUT /_cluster/settings
{
  "persistent": {
    "apack.slice_collection.search.default_to_all_slices": true
  }
}

After the switch is enabled, GET /knowledge-chunks/_search in namespace mode is equivalent to explicitly passing _slice=_all. In vector-cluster mode, invalid paths, query_vector_builder, and incorrect vector dimensions are still rejected. For more information, see Automatically select candidate Slices.

This setting applies only to the query endpoints listed above that do not write data. It does not apply to GET, MGet, Explain, Reindex, Update by query, or Delete by query.

Routing compatibility parameter

For a Collection or Collection Alias, the document write, read, and query APIs accept the standard Elasticsearch routing parameter in place of _slice. Both parameters represent the same logical Slice and are semantically equivalent, so this compatibility syntax keeps your existing Elasticsearch clients working. For new applications, use_slice, which is semantically clearer. Note the following constraints:

Do not provide both _slice and routing in the same URL or in the same Bulk or MGet item.
This compatibility syntax does not apply to source.routing in a Reindex request body, Collection and Slice management endpoints, cache prewarming, Copy Slice, CAT, or an explicit refresh.
This compatibility syntax does not change the routing semantics of regular Elasticsearch indexes.

Regular indexes in mixed requests

When a Search-type request contains both regular indexes and Collections, the cluster setting apack.slice_collection.rewrite.mixed_collection_targets determines the behavior:

Value	Behavior of Search-type requests
`ignore` (default)	Skips Collection rewriting and returns a warning. When the request explicitly specifies existing regular indexes and Collections without `_slice`, only the regular indexes are queried. When the request includes `_slice`, a regular index that does not have Slice enabled causes the request to return HTTP 400 with the message `[_slice] is not allowed when [index.slice.enabled] is false`.
`reject`	Rejects the mixed request and returns HTTP 400.
`expand`	Reserved for operations that support expansion. Search-type requests do not currently support mixed expansion, so they return HTTP 400 with this value.

In this case, routing is still processed by the routing rules of regular indexes and does not represent a Slice in a Collection. Even if all names in a mixed request exist and the request succeeds, do not assume that the results contain Collection data.

A request that contains both a Collection and a remote target is also treated as a mixed request. If the instance contains a Collection, the preceding rules also apply to GET /_search without an index. By default, the setting skips only the rewriting of the logical Collection entry point. A full or overly broad wildcard query can still match visible Backings directly, and such a query cannot replace a query scoped by Collection and Slice.

To read both types of data, send separate requests and have the application merge the results.