Elasticsearch:Upgrade cluster configuration

Before you upgrade

An upgrade can cause service latency, configuration conflicts, and billing changes. Review the following information before you proceed.

• Service stability — The following table describes how an upgrade affects service under different cluster load profiles, and the measures to take for each profile.

• Maintenance window — Perform the upgrade during off-peak business hours.

• Capacity planning

  Evaluate the required cluster capacity before you upgrade.

• Configuration constraints

  • Upgrading the cluster configuration does not support Elasticsearch version upgrades.

  • Each upgrade operation supports changing only one type of node.

• Cost impact

  After you submit an upgrade order, the system bills you according to the updated configuration. For billing rules, see Pay-as-you-go and Subscription.

Recommended specifications for common scenarios

Select a node specification family based on the current performance bottleneck of your cluster:

• Upgrading memory can indirectly improve write speed because a larger JVM heap reduces GC pauses and more off-heap memory reduces disk I/O. The actual effect varies by workload and data volume. Validate it in a test environment before you make a decision. For a complete evaluation of cluster capacity requirements, see Evaluate the required cluster capacity.

• Do not use nodes of 2 vCPU 4 GiB or lower in a production environment. For production environments, use nodes of at least 4 vCPU 8 GiB. For the causes of CPU spikes and node disconnections on small-specification nodes, and for the solutions, see the small-specification node question in the FAQ section of this topic.

Pre-upgrade checks

An upgrade that skips these checks can cause cluster crashes, data loss, or service unavailability. Verify each item before you proceed.

• Cluster health — Run GET _cluster/health and confirm that the cluster status is GREEN. If the cluster is unhealthy, resolve the issue as described in Cluster change error - Unhealthy cluster status.

• Load safety — Run GET _cat/nodes?v and confirm that CPU usage is ≤ 60%. If CPU usage exceeds 60%, apply the high-load response measures in the service stability table before you upgrade.

• Index readiness

  • Run GET /_cat/indices?v to check whether any index is in the CLOSE state. If an index is in the CLOSE state, run POST /<index_name>/_open to temporarily open it. Otherwise, the upgrade might fail for the following reasons:

    • The cluster cannot reach GREEN when an index is in the CLOSE state. ES requires the cluster to be GREEN before it performs certain sensitive changes, such as shard allocation rule adjustments.

    • The cluster reallocates shards during an upgrade:

      • Shards of closed indices cannot participate in reallocation.

      • Operations that depend on GREEN status fail.

      • The cluster cannot reach GREEN and can reach only YELLOW at best.

  • If a CLOSE index cannot be opened or deleted for business reasons, upgrade validation might still display the message "The cluster status is unhealthy. The current operation cannot be performed." CLOSE indices can trigger this message even when GET _cluster/health reports that the cluster is healthy. This is expected validation behavior. In this case, disable Intelligent Update on the upgrade page, set the update method to In-place Update, and then continue the upgrade. Because In-place Update rolls through nodes without copying data, keeps node IP addresses unchanged, and takes relatively less time, it suits a cluster that contains many CLOSE indices that cannot be handled.

    This method applies only to V2 architecture instances, on which you can disable Intelligent Update. V3 architecture instances (the new cloud-native control plane) do not support disabling Intelligent Update, so node specification and disk type upgrades on those instances always use a blue-green update. Carefully assess the impact before you choose In-place Update when cluster utilization is high, for example, CPU > 60%.

  • Run GET _cat/indices?v to check whether each index has at least 1 replica.

    For multi-zone instances, make sure that the replica count of any index in the cluster is less than the number of zones during the upgrade. (Recommended) Set the replica count to 1 and increase it manually after the upgrade is complete.

• Shard balance — Run GET _cat/shards?v to check whether shards are unbalanced. Confirming that shard distribution is balanced before an upgrade is critical to preventing performance degradation or cluster crashes during or after the upgrade.

  • prirep: Identifies whether a shard is a primary shard (p) or a replica shard (r).

  • state: Check whether a replica shard is UNASSIGNED, and whether a migration remains stuck in the RELOCATING state for an extended period.

    These issues prevent new nodes from receiving shards and cause the cluster to remain YELLOW or RED after the upgrade. If these issues occur, resolve them as described in Solutions for uneven cluster load.

Method 1: Upgrade through the console

Upgrade the cluster configuration in the Elasticsearch console. The rules and options that apply to each configuration item are described after the procedure.

1. Log on to the Elasticsearch console and go to the Instance List page.

2. Click Upgrade Configuration.

   Alternative entry point: On the Basic Information page, click Configuration UpdateUpgrade.

3. On the Change Specifications page, adjust configuration item values based on your business requirements.

   The configurable items vary based on cluster type and version. Refer to the options displayed on the upgrade page.

4. Review the Terms of Service and Service-Level Agreement (SLA) by clicking Terms of Service and Service Level Agreement. If you agree, click Buy Now. The payer is charged based on the payment method.

   After you click Buy Now, the upgrade starts. To track its progress and verify the result, see the Monitor progress and verify the upgrade section of this topic.

• Changes in the number of zones — If the required specification is out of stock in a zone, migrate the nodes in that zone before you upgrade.

  • Expansion: You can expand from 1 zone to 2 or 3 zones.

  • Reduction: Reducing a multi-zone instance to a single-zone instance is not supported. If you need to do this, purchase a new instance with the required zone configuration, migrate data to it, and release the original instance after the migration is complete.

• Supported node specification and node storage type upgrades, ordered from lowest to highest performance:

  • Previous-generation cloud disks: cloud disk (standard cloud disk), then ultra cloud disk, then standard SSD.

    These disk types are no longer available for purchase in some regions and zones. (Recommended) Choose ESSD cloud disks instead.

  • ESSD cloud disks: ESSD (Enterprise SSD) cloud disks use 25 GE networking and RDMA technologies to provide up to 1 million random read and write operations per disk and low-latency performance on a single connection.

  • Local disks.

    A local disk is a local disk device on the physical host of an ECS instance. It provides local storage access for ECS instances and is suitable for workloads with extremely high requirements for storage I/O performance and cost-effective mass storage.

Upgrade standard SSDs to ESSD cloud disks

If your cluster has high disk IOUtil utilization that frequently reaches its limit, upgrade the disk type from standard SSDs to ESSD cloud disks to improve I/O performance.

Storage upgrade constraints

• ESSD PL0 limitations: When you upgrade an existing standard SSD instance to an ESSD cloud disk, PL0 is unavailable. You can select only PL1 or a higher performance level (PL). PL0 remains available when you create a new ESSD instance.

• Forced Update: If a full disk causes an abnormal cluster status, first delete unnecessary indices or reduce the replica count to restore GREEN status. Otherwise, select this option on the upgrade page to force capacity expansion. For the impact of skipping the cluster health check, see the Force Change option described with the update methods below.

• Local disks: Local disk capacity is physically bound to the instance node specification and cannot be adjusted independently. To upgrade local disk capacity, upgrade the overall node specification. New instances currently support only the "new-generation cloud disk" specification family. Local disks apply only to existing instances.

• Automatic storage scaling: ES does not support an automatic storage scaling feature like that of the ApsaraDB RDS database service. Manually upgrade the cluster configuration when disk space is insufficient.

  Notes on combined changes: If you need to upgrade specifications, change the disk type, and expand capacity at the same time, note that a disk type change supports only a blue-green update and does not support an in-place update. To avoid multiple data migrations, perform the following steps separately:

1. First change the instance specification and expand capacity. You can use Intelligent Update or an in-place update. After the service recovers, perform the next step.

2. Change the disk type separately by using a blue-green update. A blue-green update can cause brief interruptions only when it switches nodes after data synchronization is complete, and has little impact on the business.

• Intelligent Update (enabled by default): The system automatically selects the optimal update method based on the configuration changes. You can disable it and specify an update method manually:

• Forced Update: This option skips the cluster health check and forces a cluster restart. It can make the service unstable during the restart and can cause a prolonged service interruption, and recovery time depends on the data volume. Use it only for emergency capacity expansion when the cluster is unavailable.

Method 2: Upgrade through the API

You can also upgrade the cluster configuration by calling an API operation. For the cluster upgrade API, see the API reference document: UpdateInstance.

Monitor progress and verify the upgrade

• To view progress after the upgrade starts, log on to the Elasticsearch console, choose Elasticsearch Clusters, open the Basic Information page of the instance, and click Show Details.

• After the upgrade is complete, verify that the configuration has taken effect on the cluster Basic Information page:

  • The upgrade is successful when the instance status returns to Normal.

  • Verify that the number of zones matches your expectations.

  • Node count and storage specifications: Verify that the new nodes have joined the cluster and that the storage specifications are correct.

  • Shard balance: Run GET _cat/allocation?v to check shard distribution. If shards are unbalanced, resolve the issue as described in Solutions for uneven cluster load.

  During the upgrade, the cluster status becomes Initializing. Cluster performance may fluctuate briefly, and transient request interruptions can occur. After the upgrade is complete, the cluster status becomes Normal.

PUT /_cluster/settings
{
  "transient" : {
    "cluster.routing.allocation.enable" : "all"
  }
}

FAQ