Instance Operations - Elasticsearch - Alibaba Cloud Documentation Center

Daily operations for the Agent Engine edition include instance monitoring, Collection permissions, and Collection backup and recovery.

Instance monitoring

To view the running metrics of the instance, go to the instance product page in the console and choose Monitoring and Logging > Cluster Monitoring in the left-side navigation pane. Set alert thresholds based on your business service level objectives (SLOs) and stress test baselines. The metric names and alerting entry points that are displayed in the console prevail.

The Cluster Monitoring page groups metrics by category and does not group them by node role, such as index nodes or query nodes. To observe the two node types separately, set Resource Type to a specific node and then filter by node name. Index node names contain -index-. Query node names contain -search-, although the console metric names use both Query and Search. The Agent Engine edition has no warm nodes, so you need to monitor only index nodes and query nodes.

The following table maps common observation targets to console metrics, grouped by dimension. Start with the cluster-level metrics, drill down to the node-level metrics, and treat thread-pool rejections as capacity signals.

Observation target	Console metric	Dimension
Write throughput	Cluster Write QPS	Cluster level
Write latency	Average Cluster Write Latency	Cluster level
Query throughput	Cluster Query QPS	Cluster level
Query latency	Average Cluster Search Latency	Cluster level
CPU utilization	Node CPU Utilization_ES Business, Node CPU Utilization_Total	Node level
Heap memory utilization	Node Heap Memory Utilization_ES Business	Node level
Rejections	Number of Tasks Rejected by the Write Thread Pool, Number of Requests Rejected by the Query Thread Pool	Thread pool level

Collection permissions

Access to Collections, Slices, and Backings is controlled by the index permissions and cluster permissions that you configure in roles.

Collection and Backing authorization scope

When you configure a role, you usually need to grant access permissions on both the Collection name and the underlying Backing. Configure both of the following values in the role, and replace <collection> with the name of your Collection:

<collection>
.sc-<collection>-*

For example, if the Collection is named knowledge-chunks, you can configure knowledge-chunks and .sc-knowledge-chunks-* in the role. If you configure only the Collection name and omit the Backing pattern, single-document reads and writes, MGet, explicit refresh, _update_by_query, and _delete_by_query may return HTTP 403. To resolve this error, add the matching Backing pattern to the role. The permissions required by each API operation prevail, and online applications must not access the Backing directly.

Important

Check for prefix overlap before you grant permissions. .sc-knowledge-chunks-* also matches the Backing of knowledge-chunks-private, so this pattern alone cannot guarantee strict isolation between Collections. .sc-* has an even broader scope and must not be used as the default least-privilege configuration.

Permissions required for each operation

The following table lists the permissions that are required for each operation.

Operation	Required permissions
Create, update, or delete a Collection or Slice	The `manage` index permission on the Collection
Pre-create a Backing	The `manage` index permission on the Collection
Call `POST /{collection}/_warm_slice` to warm up a Slice	The `manage` index permission on the Collection
Start or cancel Copy Slice	The `manage` index permission on the Collection
Create a Branch Slice (Beta)	The `manage` index permission on the Collection
Create or modify a Collection alias	The `manage` index permission on both the Collection name and the alias name
Get a Collection, query or list Slices, use CAT, or query the Copy status	The `monitor` index permission on the Collection
Call `POST /{collection}/_refresh` to explicitly refresh a Collection	The `maintenance` index permission on the Collection. This permission also includes `flush` and `forcemerge`. Grant it only to O&M or data import task accounts, and never to online application accounts. An explicit refresh affects all tenants of the entire Collection, and the conditions for using it are described in Refresh and query visibility.
Read, write, and query documents	The corresponding `read` or `write` index permission
Use `GET /_tasks/{taskId}` to query an asynchronous warm-up task	The `monitor` cluster permission

Isolation and least-privilege practices

manage covers Collection management, warm-up, and deletion at the same time. Grant only read and write to online read and write accounts. Initiate warm-up, Backing pre-creation, Copy, Branch, and Collection or Slice management from a separate service account or O&M task, and limit the Collections that the account can access. If your business requires an online application to perform warm-up directly, accept that the account also has these management capabilities, and restrict calls to management APIs at the application layer.

In addition, apply the following principles:

Application access principle: Applications must always access data through the Collection name. Do not access or store Backing names directly. Direct access to a Backing bypasses the logical entry point of the Collection and the _slice scope rewriting, so a single request can read the data of all Slices in that Backing.
Strict isolation: To strictly isolate credentials between Collections, do not rely only on the current Backing name wildcard. Achieve isolation by avoiding prefix overlap between Collection names and by using independent isolation at the platform or application layer, and verify the index scope that is actually accessible after each permission change.
Tenant isolation and credentials: A Slice is not a tenant permission isolation boundary. The current permission model cannot issue an Elasticsearch account for a single tenant that can read only the Slices of that tenant. Perform tenant-level authentication and data scope restriction at the application layer, and never distribute Elasticsearch account credentials directly to end users.

Collection backup and recovery

Collection backup and recovery is full-namespace: all Collections in an instance are backed up and restored together. Restoring only one Collection or Slice is not supported, and you cannot rename a Collection during recovery. Technical Support performs these operations through tickets.

Before recovery, the target instance must not contain any Collection, and no underlying index with the same name as the backup may remain. After you determine your backup requirements and before you go live, submit a ticket with the source instance, the recovery target, and your backup requirements. Technical Support then verifies the version, the storage conditions, and the impact on global configuration, and schedules the backup window.

During a backup, normal document reads, queries, writes, updates, and deletions in existing Slices can continue. From the start to the end of the backup, suspend the following management operations:

Creating, updating, or deleting a Collection
Registering or deleting a Slice, including the automatic registration triggered by the first write
Rolling over or switching a Backing
Creating a Branch Slice (Beta)

Ask Technical Support to help coordinate these operations, including Slice registration and Backing rollover creation that may be triggered automatically. If any of these changes occur during a backup, create the backup again.