Elasticsearch:Usage guide

Prerequisites

• You have activated the Alibaba Cloud Elasticsearch service and have created or plan to create an AI Engine Edition instance.

• You have planned the VPC, vSwitch, IP address whitelist, and client network for the instance.

• You have designed the mapping based on your business data. For vector search scenarios, you also need to determine the vector model, dimensions, similarity metric, and update method.

• Your account has the required permissions. Collection management typically requires manage, data reading requires read, and data writing requires write. For more information, see Collection permissions.

Create an instance

The purchase items and page layout may change based on the region, version, and product phase. The following steps are based on the actual console page. The regions and zones supported by AI Engine Edition are based on the actual purchasable items on the buy page.

1. Log on to the Alibaba Cloud Elasticsearch console and go to the instance creation page.

2. Select AI Engine Edition and select the region and zone provided by the console. The Elasticsearch version of this instance type is fixed. The creation page displays version 9.x, which cannot be changed. After the instance is created, you can view the specific version number on the instance list or instance details page.

3. Configure index nodes based on the write workload and configure search nodes based on the query workload. Other optional components are based on the console.

4. Configure the VPC, vSwitch, username, and password. The IP address whitelist is not configured on the creation page. You configure it on the security configuration page of the instance after the instance is created.

5. Confirm the configuration and cost, and then create the instance. After the instance status changes to Available, record the Elasticsearch and Kibana access addresses.

   The node specifications, minimum number of nodes, and dedicated master node configuration are based on the current purchasable items on the buy page.

The number and specifications of nodes should be determined based on write throughput, query concurrency, vector index construction, active datasets, and cache requirements. The specific operations for adjusting nodes and billing rules are based on the console.

Connect to an instance

You can use Kibana Dev Tools, Elasticsearch-compatible clients, or any HTTP client to access the instance. The instance address, network whitelist, certificate, and authentication method are based on the instance details page.

The access domain names of AI Engine Edition vary by network type. The domain name only determines which node role the request initially connects to and does not restrict the request type. The cluster forwards the request to the node role that actually handles it.

When you access the instance over a private network, both domain names support read and write operations. For production access, use the default private network domain name for query requests and the index node private network domain name for write requests to reduce network bandwidth consumption on search nodes. This is especially important for Bulk writes, queries that return large results, or high-concurrency scenarios. When you access the instance over the public network, use the default public network domain name for both read and write requests.

For example, if the instance ID is es-cn-xxx, the index node private network domain name is es-cn-xxx-index.elasticsearch.aliyuncs.com. The protocol, port, and network access method are based on the information displayed in the console.

The instance uses HTTP by default. If you enable HTTPS in the console, switch the client connection protocol to HTTPS.

Run the following request in Kibana Dev Tools to verify the connection:

GET /

AI Engine Edition 9.99.0 corresponds to Elasticsearch kernel version 9.5.0. Therefore, the version.number returned by this request is the kernel version number, which differs from the 9.99.0 displayed in the console. The actual Elasticsearch version is based on the cluster API response. When you install custom plugins, use the 9.99.0 version number that matches AI Engine Edition.

When you use cURL, replace the instance address and credentials. For production use, do not store passwords in plaintext in scripts or command history:

curl --user '{username}:{password}' 'http://{elasticsearch-endpoint}/'

When you use an official client, select the client version based on the API compatibility documentation for your instance version, and reuse the client connection pool. Do not create a new client for each request.

Access conventions

• Applications always access data through the Collection name or Collection Alias.

• In namespace mode, writes and queries specify the namespace through _slice. Queries that do not provide _slice fail. To query all Slices, explicitly use _slice=_all. Some document and query APIs also support routing as an alternative. For more information about the supported scope, see The routing compatibility parameter.

• In vector cluster recall mode, writes specify the vector cluster through _slice or routing_field. KNN queries can automatically select multiple candidate Slices based on the query vector. Queries that cannot automatically select candidate Slices still require an explicit _slice, or routing in compatible APIs.

• Slices are used to organize data and limit query scope. They are not account permission boundaries. To isolate tenant permissions, combine application-level authentication with security mechanisms supported by the product.

• Do not save or directly access .sc-* backing indexes (Backing Index). Their names and lifecycle are managed by the system. Some API responses (such as _index in write responses and backing_index in Slice registration responses) return these names for observation and troubleshooting only. They should not be used as targets for subsequent application requests. When you configure role permissions, use the .sc-<collection>-* wildcard pattern. For more information, see Collection permissions.

The routing compatibility parameter

When you use the document read/write and query APIs listed in this guide on a Collection or Collection Alias, you can use the standard Elasticsearch routing parameter instead of _slice. This is for compatibility with existing clients. For new applications, use the more semantically explicit _slice. The same URL or the same Bulk or MGet item cannot specify both parameters simultaneously.

This compatibility does not apply to Collection and Slice management, cache warming, Copy Slice, CAT, explicit Refresh, or Reindex. Reindex must use source._slice and dest._slice. routing_field is a configuration for deriving the Slice from a document field, not an alias for routing.

General API conventions

• JSON requests use Content-Type: application/json. Bulk and Multi Search use Content-Type: application/x-ndjson and require a trailing newline in the request body.

• Bulk, Multi Search, and batch management APIs may have item-level failures. Applications must not only check the HTTP status code. The single and batch Slice registration APIs also use item-level errors: the overall request returns HTTP 200, but the top-level errors field may be true. Check each items[].result individually.

• HTTP 429 indicates that the server currently rejects the request. The client should use exponential backoff with an upper limit and only retry operations that are suitable for retry.

• A connection timeout or HTTP 5xx does not mean the write definitely did not occur. Verify with a stable document ID or other idempotent mechanism before retrying.

• When you pass a large number of _slice values through the URL, note that the HTTP request line has a maximum length of 4096 bytes. When Slice names are long, too_long_http_line_exception may be triggered before the Slice count limit is reached.

  Collection management APIs commonly use the following query parameters. The specific supported scope is based on each API description.

A management request that returns acknowledged=false or times out does not mean the server-side operation has been rolled back. Check the current resource status before retrying.

Mode comparison

Collection provides namespace mode and vector cluster recall mode. You must choose one of the two modes. The mode is determined by the slice_strategy creation parameter and cannot be changed after creation. It affects the business meaning of Slices, the registration method, and write and query routing.

Vector cluster recall mode automatically selects multiple candidate Slices based on the vector cluster centroids and sends queries to these Slices in parallel. This does not mean that you must use this mode to store vectors. Namespace mode also supports dense_vector, KNN, and DiskBBQ. Only choose vector cluster recall mode when your business needs to automatically recall relevant vector clusters and you have completed cluster centroid maintenance and recall rate evaluation.

Decision guidance

Use the following guidance to choose between the two modes:

• Default recommendation — Choose namespace mode (exact) when the application can determine the query scope from a business identifier (tenant ID, knowledge base ID, code repository ID) before the query.

• Choose vector cluster recall mode only when all of the following are true:

  • Your business can generate and continuously maintain cluster centroids offline.

  • The data volume and number of Slices are large enough that narrowing the query scope provides tangible benefits.

  • Your business allows selecting candidate clusters before executing the KNN query, and you have determined the appropriate number of candidate Slices through recall rate testing.

• Do not use vector cluster recall mode when:

  • The data does not have a stable clustering structure, and reliable centroid vectors cannot be provided.

  • The data volume or number of Slices is small, and the cost of direct queries is already acceptable.

  • Queries must cover all vector data, and the recall scope changes introduced by candidate cluster selection are unacceptable. In this case, use namespace mode and explicitly specify the _slice query scope.

  After you select namespace mode, see Namespace mode: DiskBBQ vector search by Slice. After you select vector cluster recall mode, see Vector cluster recall mode: Automatic candidate Slice selection. The other Collection management, data APIs, cache warming, Alias, Copy Slice, and monitoring capabilities in this guide apply to both modes unless otherwise specified.

Query replicas and high availability

Queries in both modes are served by replica shards on search nodes. index.number_of_replicas must be set to at least 1. Otherwise, queries cannot be served. Setting it to 1 only meets the basic query requirement and does not provide query high availability. For production, set it to at least 2 and configure at least two search nodes so that queries can continue to be served when a single search node fails.

Namespace mode: DiskBBQ vector search by Slice

An enterprise knowledge base platform provides retrieval-augmented generation (RAG) services for multiple tenants. After the application completes tenant authentication, it can determine the current tenant ID. When a user submits a question, the application only needs to recall semantically relevant content segments from that tenant's knowledge base.

This scenario can determine the query scope before sending the request, making it suitable for namespace mode. This example creates a Collection named knowledge-chunks and uses each tenant as a Slice. The application specifies the tenant knowledge base through _slice and then uses DiskBBQ to perform KNN within the selected Slice. Slices are used to organize data and limit query scope. Tenant authentication is still handled by the application or supported Elasticsearch security mechanisms.

PUT /_slice_collection/knowledge-chunks
{
  "settings": {
    "index.number_of_shards": 2,
    "index.number_of_replicas": 2
  },
  "mappings": {
    "properties": {
      "tenant_id": { "type": "keyword" },
      "document_id": { "type": "keyword" },
      "title": { "type": "text" },
      "content": { "type": "text" },
      "category": { "type": "keyword" },
      "updated_at": { "type": "date" },
      "embedding": {
        "type": "dense_vector",
        "dims": 4,
        "index": true,
        "similarity": "cosine",
        "index_options": {
          "type": "bbq_disk"
        }
      }
    }
  }
}

{
  "acknowledged": true
}

PUT /knowledge-chunks/_doc/chunk-1001?_slice=tenant-a
{
  "tenant_id": "tenant-a",
  "document_id": "doc-refund-policy",
  "title": "Refund and return policy",
  "content": "You can request a refund for duplicate purchases within seven days after the order is completed.",
  "category": "after-sales",
  "updated_at": "2026-07-30T10:00:00Z",
  "embedding": [0.82, 0.10, 0.05, 0.03]
}

PUT /knowledge-chunks/_doc/chunk-1002?_slice=tenant-b
{
  "tenant_id": "tenant-b",
  "document_id": "doc-shipping-status",
  "title": "Check shipping status",
  "content": "Open the logistics information on the order details page to view the latest delivery status.",
  "category": "shipping",
  "updated_at": "2026-07-30T10:01:00Z",
  "embedding": [0.12, 0.78, 0.06, 0.04]
}

GET /knowledge-chunks/_doc/chunk-1001?_slice=tenant-a

POST /knowledge-chunks/_search?_slice=tenant-a
{
  "knn": {
    "field": "embedding",
    "query_vector": [0.80, 0.12, 0.05, 0.03],
    "k": 10,
    "num_candidates": 100
  },
  "_source": ["document_id", "title", "content"]
}

POST /_bulk
{ "index": { "_index": "knowledge-chunks", "_id": "chunk-1003", "_slice": "tenant-a" } }
{ "tenant_id": "tenant-a", "document_id": "doc-change-address", "title": "Change delivery address", "content": "Before an order is shipped, you can change the delivery address on the order details page.", "category": "orders", "updated_at": "2026-07-30T10:02:00Z", "embedding": [0.75, 0.16, 0.06, 0.03] }
{ "index": { "_index": "knowledge-chunks", "_id": "chunk-1004", "_slice": "tenant-b" } }
{ "tenant_id": "tenant-b", "document_id": "doc-invoice", "title": "Request an electronic invoice", "content": "After an order is completed, you can request an electronic invoice on the invoice management page.", "category": "billing", "updated_at": "2026-07-30T10:03:00Z", "embedding": [0.18, 0.70, 0.08, 0.04] }

GET /_cat/slice_collection/knowledge-chunks?v

GET /_cat/slice_collection/knowledge-chunks/slices?v

GET /_slice_collection/knowledge-chunks/slices?page_size=100

{
  "slices": [
    { "id": "tenant-a" },
    { "id": "tenant-b" }
  ]
}

GET /_slice_collection/knowledge-chunks/slices?page_size=100&cursor={next_cursor}

Vector cluster recall mode: Automatic candidate Slice selection

An e-commerce search platform maintains a large-scale product vector database and has divided products into thousands of vector clusters through offline clustering. When a user initiates a similar product query, the application only has the query vector and cannot determine the target clusters in advance. Querying all product data each time increases the query scope and resource overhead as the data scale grows.

This scenario is suitable for vector cluster recall mode: each Slice corresponds to a vector cluster, and the centroid vector of that cluster is registered. When a KNN query is executed, the system first calculates the similarity between the query vector and each cluster centroid, automatically selects the most relevant candidate Slices, and then performs DiskBBQ search in parallel within these Slices, thereby narrowing the query scope.

Collections do not train cluster centroids, nor do they automatically determine the target Slice based on document vectors. The application must compute and register centroid vectors in advance.

For when to use and when to avoid this mode, see Decision guidance.

PUT /_slice_collection/products
{
  "slice_strategy": "vector_cluster",
  "auto_create_slice": false,
  "settings": {
    "index.number_of_shards": 2,
    "index.number_of_replicas": 2
  },
  "mappings": {
    "properties": {
      "cluster_id": { "type": "keyword" },
      "name": { "type": "keyword" },
      "embedding": {
        "type": "dense_vector",
        "dims": 2,
        "index": true,
        "similarity": "cosine",
        "index_options": {
          "type": "bbq_disk"
        }
      }
    }
  },
  "config": {
    "routing_field": "cluster_id",
    "vector_dims": 2,
    "default_query_vector_path": "knn.query_vector",
    "default_query_slice_count": 128,
    "max_query_slice_count": 512
  }
}

PUT /_slice_collection/products/slices/cluster-a
{
  "vector": [0.9, 0.1]
}

PUT /_slice_collection/products/slices
{
  "slices": [
    {
      "slice_id": "cluster-b",
      "vector": [0.1, 0.9]
    },
    {
      "slice_id": "cluster-c",
      "vector": [0.6, 0.4]
    }
  ]
}

PUT /products/_doc/product-1
{
  "cluster_id": "cluster-a",
  "name": "example-product",
  "embedding": [0.92, 0.08]
}

POST /products/_search?query_slice_count=64
{
  "knn": {
    "field": "embedding",
    "query_vector": [0.91, 0.09],
    "k": 10,
    "num_candidates": 100
  }
}

Creation parameters

The API for creating a Collection is:

PUT /_slice_collection/{collection}

Query parameters:

The request body parameters are as follows.

If a Backing Index has N primary shards, the maximum number of Slices it can accept based on the count limit alone is N × max_slices_per_shard. max_slices_per_shard and max_storage_per_shard only determine whether subsequent new Slices continue to be allocated to this Backing:

• After either threshold is reached, the system selects or creates another Backing to host new Slices.

• Existing Slices can continue to receive writes.

• Changing the thresholds does not migrate existing Slices or rebalance existing data.

  In most scenarios, the default last is sufficient. Only use random when multiple Backings can accept new Slices simultaneously and you want to distribute newly registered Slices randomly.

View a Collection

View a single Collection:

GET /_slice_collection/knowledge-chunks

View all Collections:

GET /_slice_collection

This API supports the master_timeout query parameter.

{collection} supports comma-separated names and wildcards. An exact name that does not exist returns HTTP 404. A wildcard with no matches returns an empty object.

Response example:

{
  "slice_collections": {
    "knowledge-chunks": {
      "collection_uuid": "opaque-system-id",
      "lifecycle_state": "active",
      "max_slices_per_shard": 200,
      "max_storage_per_shard": "50gb",
      "backing_allocation_strategy": "last",
      "future_only_settings": {},
      "auto_create_slice": true,
      "slice_strategy": "exact",
      "max_managed_backing_generation": 0,
      "aliases": {}
    }
  }
}

The response may also contain system-managed identifiers and generation fields. Applications should treat them as opaque information and not use them for constructing Backing names or implementing business logic.

Update a Collection

This API is used to adjust the allocation strategy for subsequently registered Slices and does not move existing Slices.

Update the mutable configuration:

POST /_slice_collection/knowledge-chunks/_update
{
  "max_slices_per_shard": 300,
  "max_storage_per_shard": "80gb",
  "backing_allocation_strategy": "last",
  "auto_create_slice": false,
  "settings": {
    "apack.slice_collection.future.index.number_of_shards": 4
  }
}

Query parameters:

The updatable fields include:

• max_slices_per_shard

• max_storage_per_shard

• backing_allocation_strategy

• auto_create_slice

• Allowed future-only settings in settings

  Future-only settings only affect Backings that are created afterward and do not modify existing Backings. The allowed settings by default are:

• index.number_of_shards

• index.routing_partition_size

• index.number_of_routing_shards

  In _update requests, future-only settings must use the apack.slice_collection.future. prefix. For example, index.number_of_shards corresponds to apack.slice_collection.future.index.number_of_shards. When writing, the flat prefix is used. When reading back through GET /_slice_collection/{collection}, the settings are presented in a nested structure, for example, "future_only_settings": {"index": {"number_of_shards": "4"}}.

A successful response is {"acknowledged":true}. acknowledged=false indicates that the confirmation timed out, not that the update has been rolled back.

Update mappings and dynamic settings

Update the mapping by using the Collection name:

PUT /knowledge-chunks/_mapping
{
  "properties": {
    "channel": { "type": "keyword" }
  }
}

Update the dynamic index settings by using the Collection name:

PUT /knowledge-chunks/_settings
{
  "index.refresh_interval": "5s"
}

Mappings and dynamic index settings are applied to all current Backings and become the unified configuration for subsequent Backings. Do not modify individual Backing Indexes.

Register a Slice

When you choose a Slice registration method, see the following table.

When auto_create_slice=false, or when you want to complete resource preparation before writing, you can explicitly register a Slice:

PUT /_slice_collection/knowledge-chunks/slices/tenant-c

This API supports the master_timeout query parameter.

Response example:

{
  "acknowledged": true,
  "errors": false,
  "items": [
    {
      "slice_id": "tenant-c",
      "result": "created",
      "backing_index": ".sc-knowledge-chunks-...-00000"
    }
  ]
}

backing_index is for observation and troubleshooting only and should not be used as a target for subsequent application requests.

Register up to 20480 Slices with unique names at a time:

PUT /_slice_collection/knowledge-chunks/slices
{
  "slices": [
    "tenant-d",
    { "slice": "tenant-e" },
    { "slice_id": "tenant-f" }
  ]
}

This API supports the master_timeout query parameter. The three formats can be mixed in the same request.

Batch registration may partially succeed. The result of each item may be created, updated, noop, or failed. The top-level errors is true as long as any item has failed. When a Slice name is invalid (for example, contains :, starts with a non-alphanumeric character, uses the reserved value _all, or exceeds the length limit), the registration API also returns HTTP 200 and provides failed and the error reason in the corresponding item.

List Slices with pagination

GET /_slice_collection/knowledge-chunks/slices?prefix=tenant-&page_size=100

Results are returned in ascending order by Slice name. If Slices are concurrently added or deleted during pagination, the results are weakly consistent. Keep the same prefix when continuing to paginate.

Delete a Slice

DELETE /_slice_collection/knowledge-chunks/slices/tenant-c

Query parameters:

Successful response:

{
  "acknowledged": true
}

After the Slice is deleted, a Slice with the same name can be re-registered. New data is isolated from the old data pending cleanup in the background. Note:

• After a Slice is deleted, queries that specify the Slice name return HTTP 404 (resource_not_found_exception) instead of empty results.

• Full-scope queries using _slice=_all may briefly return old data before physical cleanup is complete.

• The deletion response does not indicate that storage space has been released.

Write and read data

Collections reuse Elasticsearch standard document APIs and specify the namespace or vector cluster through _slice. Common APIs are as follows.

Rules:

• A single-document request can specify only one Slice. Comma-separated values and _all are not allowed.

• index, create, and update can automatically register a nonexistent Slice when auto_create_slice=true. Read and delete operations do not automatically register Slices. Specifying a nonexistent Slice returns HTTP 404.

• Some document and query APIs can use routing to represent the logical Slice. For the supported scope and conflict rules, see The routing compatibility parameter.

• Each Bulk request can auto-register up to 512 different missing Slices. Existing Slices are not counted against this limit.

• wait_for_active_shards is not used as a condition for waiting for replicas in the AI Engine Edition write pipeline. To wait for a document to be searchable, use refresh=wait_for.

• The dense_vector field is not included in the returned _source by default. Neither GET /{collection}/_doc/{id} nor GET /{collection}/_source/{id} returns vector values. To return them, explicitly specify _source_includes.

Refresh and query visibility

A successful write request indicates that the data has been persisted, not that the data is queryable by search nodes. AI Engine Edition uses a stateless architecture. Refresh requires the index node to generate and publish a new Commit, and then the search node loads the new searchable version. This is a distributed cross-node operation with higher overhead than a local Refresh in traditional stateful Elasticsearch.

You can query and adjust index.refresh_interval through Collection settings. The current value can be viewed through GET /{collection}/_settings. A shorter Refresh interval increases resource overhead. For production, set it to 5s or higher. If you have low-latency visibility requirements, adjust the value based on your business workload. The actual effective value is based on the current Collection settings. Automatic Refresh can be disabled by setting the value to -1. Before shortening this interval, evaluate the Commit publishing frequency, write throughput, and object storage I/O.

Choose one of the following methods based on your business requirements for query visibility:

index.refresh_interval=-1 disables automatic Refresh. In this case, writes using refresh=wait_for will wait indefinitely until another request triggers an explicit Refresh. Do not combine these settings without an explicit Refresh process. For production, when read-after-write is required, choose between waiting for automatic Refresh or explicit Refresh based on your business latency requirements, and evaluate the distributed Refresh overhead. Refresh only addresses query visibility. It does not mean all query replicas are available and cannot replace query probes and high-availability checks.

Bulk

Each Bulk item can provide its own _slice in the action metadata, as shown in the quick start example. When all items in a batch are written to the same Slice, you can provide a default value in the URL:

POST /knowledge-chunks/_bulk?_slice=tenant-a
{ "index": { "_id": "chunk-2001" } }
{ "tenant_id": "tenant-a", "document_id": "doc-account-security", "title": "Account security settings", "content": "Administrators can enable multi-factor authentication on the security settings page." }

The _slice in the item can override the URL default value. Namespace mode does not derive the Slice from document fields. Even if the document body contains a tenant_id field, the actual write location is still determined by _slice. For error handling, see General API conventions.

Multi Get

MGet is suitable for reading multiple documents with known IDs at once. Each item can access a different Slice:

POST /knowledge-chunks/_mget
{
  "docs": [
    { "_id": "chunk-1001", "_slice": "tenant-a" },
    { "_id": "chunk-1002", "_slice": "tenant-b" }
  ]
}

The _slice in the URL can serve as a default value. Each item can override the default value. Each item must ultimately resolve to a single Slice.

Reindex

Reindex is suitable for migrating data between regular indexes and Collections, or between different Collections. To copy a Slice within the same Collection, use Copy Slice.

When the source is a Collection, you must explicitly specify source._slice in the request body. Otherwise, HTTP 400 is returned. When the destination is a Collection, only one destination Slice can be specified, and the value must use the ={slice} format:

POST /_reindex
{
  "source": {
    "index": "source-knowledge-chunks",
    "_slice": "tenant-a,tenant-b"
  },
  "dest": {
    "index": "target-knowledge-chunks",
    "_slice": "=tenant-archive"
  }
}

The destination Slice is automatically registered if it does not exist. The top-level slices parameter represents the Elasticsearch Reindex parallelism and is unrelated to the number of business Slices. When either side is a Collection, Reindex does not support scripts. When a Collection is the destination, explicit ingest pipelines are not supported.

Query data

Collection queries continue to use Elasticsearch Query DSL. This section covers only the Slice query scope added by AI Engine Edition. For Query DSL and query capabilities not listed in this guide, the actual supported scope of the 9.99.0 instance prevails.

Explicitly specifying _slice in queries applies to both modes. Namespace mode typically requires an explicit query scope. KNN queries in vector cluster recall mode can automatically select multiple candidate Slices. For more information, see Vector cluster recall mode: Automatic candidate Slice selection.

GET /knowledge-chunks/_search?_slice=tenant-a,tenant-b&ignore_missing_slice=true

PUT /_cluster/settings
{
  "persistent": {
    "apack.slice_collection.search.default_to_all_slices": true
  }
}

GET /knowledge-chunks/_search

PUT /_slice_collection/knowledge-chunks
{
  "settings": {
    "index.sliced_postings.enabled": true
  }
}

Warm Slice caches

The _warm_slice API prefetches data that subsequent queries to a specified Slice may access from object storage to the shared cache on search nodes, reducing the first-query latency for cold data. The warming scope includes DiskBBQ vectors, Slice-related inverted index data, doc values, and stored fields, and covers all searchable replicas of the shard where the Slice resides.

Warming is only for cache optimization. It only affects subsequent access speed, does not change query results, and does not guarantee how long data stays in the cache.

Use cases for warming:

• A user is about to open a knowledge base, code repository, or Agent memory space that has not been accessed for a long time. You can warm the corresponding Slice before the first query.

• After a batch import or data migration is completed, you can warm the Slices that are about to be activated before switching query traffic.

• Certain namespaces are known to experience access peaks during specific time periods. You can warm these Slices in advance.

  Active warming is not recommended when data access is unpredictable, when a large number of Slices need to be warmed at once, or when the target Slices are already hot data. Large-scale warming consumes object storage and search node network, cache, and compute resources. It may also evict existing hot data from the cache.

POST /knowledge-chunks/_warm_slice?_slice=tenant-a

{
  "status": "ACCEPTED",
  "message": "cache warm hint accepted",
  "task": "{nodeId}:{taskId}"
}

GET /_tasks/{nodeId}:{taskId}

POST /knowledge-chunks/_warm_slice?_slice=tenant-a&store_result=true

POST /knowledge-chunks/_warm_slice?_slice=tenant-a&wait_for_completion=true&timeout=60s

DiskBBQ vector indexing

DiskBBQ corresponds to the bbq_disk index type of dense_vector. It is a vector indexing capability used within shards, not a third Collection mode. Both namespace mode and vector cluster recall mode can use DiskBBQ. This section describes the configuration and query methods. For the performance reference of AI Engine Edition, see the Feature overview.

Use cases for DiskBBQ:

• The vector scale is large, and you want to reduce resident memory pressure through disk-native vector indexing.

• You can use a business evaluation dataset to tune the balance between recall rate, query latency, and data read overhead.

  DiskBBQ is not recommended as the default approach when the dataset is small, exact nearest-neighbor results are required, or recall rate evaluation has not been completed.

PUT /{collection}/_mapping
{
  "properties": {
    "embedding": {
      "type": "dense_vector",
      "dims": 4,
      "index": true,
      "similarity": "cosine",
      "index_options": {
        "type": "bbq_disk"
      }
    }
  }
}

{
  "index_options": {
    "type": "bbq_disk",
    "cluster_size": 384,
    "flat_index_threshold": -1,
    "default_visit_percentage": 0.0,
    "rescore_vector": { "oversample": 3.0 },
    "bits": 1
  }
}

POST /knowledge-chunks/_search?_slice=tenant-a
{
  "knn": {
    "field": "embedding",
    "query_vector": [0.80, 0.12, 0.05, 0.03],
    "k": 10,
    "visit_percentage": 10.0
  },
  "_source": ["document_id", "title", "content"]
}

Collection Alias

Collection Alias provides applications with a stable access name. It can be used to query multiple Collections uniformly or to switch write targets. You can add an Alias when creating a Collection through the aliases field, or manage Aliases by using the standard Elasticsearch _aliases API.

Use cases for Collection Alias:

• Version migration: The application always accesses a fixed Alias. After data preparation is completed, switch the write target from the old Collection to the new Collection.

• Multi-Collection queries: Query multiple Collections that use the same slice_strategy through a single Alias.

• Application decoupling: Business configurations only save the Alias and do not directly depend on Collection names that contain versions or dates.

  Collection Alias is not applicable if your business relies on Alias filter or routing, or if you want to include both regular indexes and Collections in the same Alias.

The following example assumes that both knowledge-chunks-v1 and knowledge-chunks-v2 are Collections using namespace mode (exact). Add them to the same Alias and set knowledge-chunks-v2 as the write target:

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "knowledge-chunks-v1",
        "alias": "knowledge-chunks-current"
      }
    },
    {
      "add": {
        "index": "knowledge-chunks-v2",
        "alias": "knowledge-chunks-current",
        "is_write_index": true
      }
    }
  ]
}

View the Alias:

GET /_alias/knowledge-chunks-current

Remove a member from the Alias:

POST /_aliases
{
  "actions": [
    {
      "remove": {
        "index": "knowledge-chunks-v1",
        "alias": "knowledge-chunks-current"
      }
    }
  ]
}

Note the following when you use Aliases:

• Members of the same Alias must use the same slice_strategy. Members that use vector cluster recall mode (vector_cluster) must also use the same vector dimensions.

• When a multi-member Alias is used for writes, exactly one member must have is_write_index=true.

• APIs such as Search, Count, and Bulk can access multi-member Aliases based on the Alias semantics.

• Single-document GET and each item in MGet must uniquely resolve to one Collection. Therefore, only single-member Aliases can be used.

• filter, routing, index_routing, search_routing, is_hidden, and remove_index are currently not supported for Aliases.

  When you query a multi-member Alias, the target Slice must exist in all member Collections. Otherwise, HTTP 404 is returned. During version migration, the Slice sets of the old and new Collections are typically different. In this case, explicitly include ignore_missing_slice=true, for example, GET /knowledge-chunks-current/_search?_slice=tenant-a&ignore_missing_slice=true, so that the request skips missing members and returns normally.

Copy Slice

Copy Slice copies the replicable documents from one Slice to another Slice within the same Collection online. It is suitable for data migration, not for point-in-time snapshots, backups, or atomic traffic switching.

Use cases for Copy Slice:

• Copy test or verification data for a tenant while maintaining the same Collection mapping as the source Slice.

• Prepare a new namespace within the same Collection. After the Copy is complete and verified, the application layer switches the access name.

• Perform online data migration for a Slice while keeping both the source Slice and destination Slice readable and writable.

  Writes may still occur during the Copy. Therefore, the result is not a point-in-time snapshot of the source. For backup, disaster recovery, strict consistent snapshots, or atomic traffic switching, do not use Copy Slice.

POST /_slice_collection/knowledge-chunks/slices/tenant-a/_copy/tenant-a-copy?wait_for_completion=false
{
  "workers": 8,
  "batch_size": 5000,
  "requests_per_second": -1
}

{
  "copy_id": "tenant-a-copy:1h",
  "completed": false,
  "timed_out": false,
  "state": "RUNNING",
  "source": "tenant-a",
  "target": "tenant-a-copy",
  "workers": 8,
  "progress": { "total": 0, "created": 0, "version_conflicts": 0 },
  "status_url": "/_slice_collection/knowledge-chunks/_copy/tenant-a-copy%3A1h",
  "cancel_url": "/_slice_collection/knowledge-chunks/_copy/tenant-a-copy%3A1h/_cancel"
}

GET /_slice_collection/knowledge-chunks/_copy/{copy_id}

PUT /_cluster/settings
{
  "persistent": {
    "apack.slice_collection.copy.reservation_retention": "1d"
  }
}

POST /_slice_collection/knowledge-chunks/_copy/{copy_id}/_cancel

Cluster monitoring

In the left-side navigation pane of the instance details page in the console, choose Monitoring and Logs > Cluster Monitoring to view the running metrics of the instance. When you configure alert thresholds, set them based on your business SLO and stress testing baselines. The specific metric names and alert entry points are based on the console.

The following table lists the mapping between common observation targets and console metrics.

The cluster monitoring page groups metrics by category and does not provide role-based grouping for index nodes and search nodes. To observe the two types of nodes separately, switch Resource Type to Specified Node and then filter by node name: index node names contain -index-, and search node names contain -search-. AI Engine Edition does not have warm nodes. You only need to monitor the index nodes, search nodes, and the monitoring metrics actually provided by AI Engine Edition.

CAT API

CAT API is suitable for manual observation and troubleshooting. For a daily overview, start with the Collection. When capacity or shard anomalies occur, check the Backing. To locate the data distribution and document count of a specific namespace, check the Slice. Applications should not rely on CAT output for business requests.

Default columns:

Common query parameters:

Slice CAT also supports backing={backing} to narrow the scope to a specific full Backing name. Sorting across all Backings by docs.count is a high-cost query and requires explicitly setting allow_expensive_search=true.

For programmatic processing, use format=json and explicitly specify the columns:

GET /_cat/slice_collection/knowledge-chunks?format=json&h=collection,state,backings,slices,docs.count,pri.store.size

CAT API is designed for manual troubleshooting and bounded operations queries:

• Slice CAT returns a maximum of 10000 rows. When the limit is exceeded, an HTTP Warning header indicates that the results are incomplete.

• When you need a complete and stable paginated Slice list, use GET /_slice_collection/{collection}/slices.

• docs.count is near-real-time data. Documents that have been written but not yet refreshed may not be counted.

• pri.store.size and store.size represent the indexed data size, which is not equivalent to the local disk usage on stateless nodes. They also do not include historical data in object storage that is pending cleanup.

Delete a Collection

This API is applicable to taking business data offline or cleaning up the examples in this guide. Deletion removes the Collection configuration and system-managed data. This operation is irreversible. To clean up only a specific Slice, use the delete Slice API.

DELETE /_slice_collection/knowledge-chunks

Query parameters:

Successful response:

{
  "acknowledged": true
}

{collection} supports comma-separated names. By default, wildcards * and _all are not allowed. The request returns HTTP 400. To use them, adjust the action.destructive_requires_name cluster setting.

Deletion returns HTTP 409 when a Copy Slice operation is running or being cancelled. Cancellation is asynchronous. Deletion will still be rejected until state changes to CANCELLED. Confirm that state=CANCELLED before executing the deletion.

acknowledged=true indicates that the logical resource and metadata deletion has been confirmed. Historical indexes and Translog files in object storage are asynchronously reclaimed by a background process. Physical space is not guaranteed to be released immediately when the response is returned.

Collection permissions

When you configure roles, the index name must include both the Collection name and the corresponding Backing wildcard:

<collection>
.sc-<collection>-*

For example, if the Collection name is knowledge-chunks, configure both knowledge-chunks and .sc-knowledge-chunks-* in the role. Configuring only the Collection name causes operations such as single-document read/write, MGet, explicit Refresh, and _update_by_query to return HTTP 403.

.sc-* is only for permission configuration. Applications should still access data through the Collection name and should not directly access or save specific Backing Index names.

Slices are not tenant permission isolation boundaries. To isolate permissions, implement this at the application layer or through other supported Elasticsearch security models.

Common differences from traditional stateful Elasticsearch

Settings marked as inapplicable in the following table may return HTTP 200 when modified, and the new value may be visible when read back. This does not indicate that the feature is enabled.

Additionally, replicas in the stateless architecture are primarily used for queries and caching. They are not equivalent to the persistent replicas in traditional stateful Elasticsearch that can be promoted to primary shards. The number of query replicas and search nodes together determine query capacity and availability. Persistent data is stored in object storage. Increasing the number of replicas cannot replace a backup strategy.

Limitations when using Collections and Slices

The following limitations apply only to the Collection and Slice usage in AI Engine Edition. They do not mean the same-named capabilities are also unavailable for regular indexes.

API quick reference

The following is a summary of the 9.99.0 public APIs required for the use cases in this guide. This is not a complete Elasticsearch API list, nor does it include internally maintained APIs.