upgrade cluster configuration, scale up cluster, upgrade cluster, change cluster configuration - Elasticsearch

Upgrade your ES cluster by adding nodes, upgrading node specifications, expanding disk space, or adding node types when resource utilization stays high or performance falls short.

Before you begin

Important

Upgrading can cause service latency, configuration conflicts, and billing changes. Review the following items before you proceed.

Service stability

Service stability during configuration changes:

Cluster condition	Service impact	Recommended action
Normal load with replicas Normal load: CPU ≤ 60%, heap memory ≤ 50%, load < number of cores.	Service remains available. Minor performance decrease possible.	No action required.
High load without replicas High load: High concurrent writes or queries during the upgrade, with CPU > 60% or heap memory > 50%.	Occasional access timeouts may occur.	Enable a retry mechanism on the client. Increase the number of index replicas before the upgrade.
High load with an unhealthy status	Occasional access timeouts or service jitters may occur.	Restore cluster health before the change.

Maintenance window: Perform the operation during off-peak hours.

Capacity planning
evaluate the required cluster capacity.
Configuration constraints
- You cannot upgrade the Elasticsearch version while upgrading the cluster configuration.
- You can change only one type of node in a single upgrade operation.
- For V3 architecture clusters, you cannot disable Intelligent Update. If you call the API operation that updates the instance configuration and set the intelligent parameter to false, the system overrides the value to true. The upgrade page for V3 architecture clusters does not provide an option to disable Intelligent Update, and node specification or disk type upgrades always use a blue-green update.
Cost impact
After you submit the upgrade order, billing follows the updated configuration. Billing rules: pay-as-you-go, subscription.

Recommended specifications for common scenarios

When upgrading your Elasticsearch cluster, selecting the appropriate specification family based on your workload type helps you make efficient use of resources. Use the following table to identify the specification family that best fits your scenario:

Scenario	Recommended Spec Family	Example Spec	Use Case
CPU/Write-intensive	1:2 Compute-optimized	8 vCPU 16 GiB	High CPU utilization and write-intensive workloads
Memory-intensive	1:4 General-purpose	8 vCPU 32 GiB	Scenarios requiring larger off-heap memory
High memory demand	1:8 Memory-optimized	8 vCPU 64 GiB	Deep aggregations and large index caching

Note

Upgrading memory can improve write performance to some extent. However, the actual effect depends on your workload and data volume. Verify the effect in a test environment before applying changes to your production cluster.

Note

We do not recommend using node specifications of ≤2 vCPU 4 GiB in production environments. On such small-specification nodes, system processes and monitoring index maintenance consume a disproportionate share of the limited resources, which can cause CPU utilization to spike even when there is no business load. We recommend using a minimum specification of 4 vCPU 8 GiB for production environments.

For more information about how to evaluate the required specifications and storage capacity for your cluster, see Evaluate specifications and storage capacity.

Pre-upgrade checks

Important

Skipping these checks can cause cluster crashes, data loss, or service unavailability.

Cluster health
Run GET _cluster/health to verify the cluster status is GREEN. If unhealthy, resolve the issue using Cluster Change Error - Unhealthy Cluster State.
Load safety
Run GET _cat/nodes?v. CPU utilization should be ≤ 60%. If higher, enable client-side retries and increase index replicas.
Index readiness
- Run GET /_cat/indices?v to check for closed indices. Open any with POST /<index_name>/_open before proceeding. Closed indices cause configuration changes to fail because:
  - Closed indices prevent the cluster from reaching GREEN status, which is required for shard allocation changes.
  - During a configuration change, the cluster reallocates shards:
    - Shards of closed indices cannot participate in reallocation.
    - Operations requiring GREEN status fail.
    - The cluster can only reach YELLOW at best.
- If closed indices cannot be opened or deleted for business reasons, the console still blocks the upgrade with an unhealthy cluster status error, even if GET _cluster/health reports a normal status. This is expected validation behavior caused by the closed indices, not a conflicting signal. As a workaround, disable Intelligent Update on the upgrade page and manually select In-place Update to continue the upgrade. This update method performs a rolling restart of nodes without copying data, keeps node IP addresses unchanged, and takes less time, which makes it suitable for clusters that contain many closed indices that cannot be handled. This workaround applies only to V2 architecture clusters, where Intelligent Update can be disabled. For V3 architecture clusters, Intelligent Update cannot be disabled, the upgrade page does not provide this option, and node specification or disk type upgrades always use a Blue-green Update. To identify the architecture version of your cluster, see the Determine the architecture version of your cluster section in this topic. As with any in-place update, use caution if resource utilization is high (for example, CPU > 60%).
- Run GET _cat/indices?v to verify each index has at least 1 replica.
  For multi-zone deployments, keep the replica count below the number of zones during the change (recommended: 1). Increase replicas after the change completes.
Shard balance
Run GET _cat/shards?v to check for any unbalanced shards.
Important
Unbalanced shards can cause performance degradation or cluster crashes after the upgrade.
- prirep: Check if any replica shard (r) is UNASSIGNED.
- state: Check if any shard migration is stuck in the RELOCATING state for an extended period.
These issues prevent new nodes from receiving shards, leaving the cluster in YELLOW or RED. Resolve them using Solutions for uneven cluster load.

Determine the architecture version of your cluster

ES clusters run on one of two control plane architectures, v2 or v3. The available upgrade methods and the estimated duration differ by architecture (see the update method duration details in Method 1: Upgrade via console in this topic). Determine the architecture version of your cluster before you proceed with the upgrade.

Method 1: Check in the console

Log on to the ES console, go to the Basic Information page of your instance, and check the value of the Control Plane Deployment Mode field. This value indicates the architecture version of your cluster.

Method 2: Determine by Elasticsearch version number

Architecture version	Corresponding Elasticsearch version numbers
v2	5.5.3, 5.6.16, 6.3.2, some 6.7.0, 6.8.6, 7.4.0, 7.7.1, some 7.10.0, some 7.16.2
v3	some 6.7.0, 6.8.23, some 7.10.0, some 7.16.2, 8.x and later

Note

For version numbers 6.7.0, 7.10.0, and 7.16.2, both v2 and v3 instances exist. The version number alone cannot uniquely determine the architecture. Check the Control Plane Deployment Mode field on the Basic Information page of the console to confirm.

Method 1: Upgrade via console

On the Instances page, click Upgrade Configuration.
Alternative entry point: On the Basic Information page of your instance, click Configuration Update > Upgrade.

On the Upgrade/Downgrade page, adjust the configuration parameters based on your business requirements.

Important

Available parameters vary by cluster type and version. The upgrade page shows the applicable options.

Availability zone changes: If stock is insufficient for a specification in a zone, migrate nodes first.
You can expand from one zone to two or three. Reducing a multi-zone cluster to a single zone is not supported. If you need to reduce the number of zones, purchase a new instance with the required zone configuration, migrate your data to it, and then release the original instance.
Node specifications and storage types (lowest to highest performance):
1. Previous generation cloud disks: standard cloud disk -> ultra cloud disk -> SSD cloud disk.
  Note
  These disk types are being phased out in some regions. Use ESSDs instead.
2. ESSDs: ESSDs (Enterprise SSDs) use 25 GbE networking and RDMA for up to 1 million IOPS per disk with low latency.
3. Local disks.
  Note
  Local disks reside on the physical ECS host machine. They suit workloads requiring high I/O performance or cost-effective mass storage.
Note
When to upgrade from SSD to ESSD: If the IOUtil metric of your cluster remains consistently high and frequently reaches 100%, consider upgrading from SSD cloud disks to ESSD cloud disks to improve I/O performance.
Note
Storage upgrade constraints:
- ESSD PL0 limitation: If you upgrade an existing SSD cloud disk instance to an ESSD cloud disk, PL0 is not available. You can only select PL1 or a higher performance level. PL0 remains available when you create a new ESSD instance.
- Forced update: If a disk is full and the cluster status becomes abnormal, delete unnecessary indices or reduce the number of replicas to restore the cluster to the green status before you upgrade. Otherwise, you can select Forced Update on the upgrade page to force the capacity expansion, which may cause the service to become unstable during the restart. The upgrade page also provides an Intelligent Update option (enabled by default), which lets the system automatically select the update method that fits the type of change.
- Local disk dependency: If your cluster uses local disks, upgrading the storage capacity requires upgrading the overall node specification. Currently, only the New generation cloud disk node type is available when you create a new instance; local disks are only available for existing instances.
- Automatic storage scaling not supported: Elasticsearch does not support an automatic storage scaling feature similar to that of ApsaraDB RDS.
Important
Combined change considerations: If you need to upgrade the node specification, change the disk type, and expand capacity at the same time, note that changing the disk type does not support in-place updates — only blue-green updates are supported. To avoid multiple data migrations, perform the changes in separate steps:
1. Upgrade the node specification and expand capacity first. You can use Intelligent Update or in-place update for these changes.
2. After the cluster recovers, change the disk type separately using a blue-green update.
Blue-green updates may cause brief connection interruptions only during the node switchover phase after data synchronization is complete, minimizing the impact on your business.

Intelligent Update (enabled by default): The system selects the optimal update method automatically. You can disable it and choose manually:

Update method	Mechanism	Duration	Service impact and use cases
Blue-green Update	Add new nodes → Copy data → Seamless switchover	Long	Node IP addresses change. Temporary performance fluctuations possible. Best when availability matters more than update speed.
In-place Update	Rolling restart of nodes (no data copying required).	Short	Node IP addresses unchanged. Temporary performance fluctuations possible. Best for quick resolution of performance bottlenecks. Important Use caution if resource utilization is high (CPU > 60%).

Estimated duration for V2 architecture instances (blue-green update)

For a V2 architecture cluster, upgrading data node specifications or the ES version uses a blue-green update. Estimate the duration as follows:

Total duration = Control plane duration + Data migration duration

Control plane duration = Number of nodes × 10 minutes × 2. A blue-green update first adds new nodes and then removes the old ones, so each node is counted for two rounds of approximately 10 minutes each.
Data migration duration (in hours) = Maximum data volume on a single data node / indices.recovery.max_bytes_per_sec / 3600. The default value of indices.recovery.max_bytes_per_sec is 40 MB/s. You can call the GET /_cluster/settings operation to check the current value and adjust it as needed.

Example: For a V2 cluster with 3 data nodes, the control plane duration is approximately 3 × 10 × 2 = 60 minutes (about 1 hour). Add the data migration duration calculated with the preceding formula to obtain the estimated total duration.

Estimated duration for V3 architecture instances (blue-green update)

For a V3 architecture cluster, a Blue-green Update consists of a control plane phase and a data migration phase:

Total duration = Control plane duration + Data migration duration

Control plane duration: approximately 10 to 20 minutes. This covers starting new nodes and removing old nodes.
Data migration duration (in hours) = Maximum data volume on a single data node / indices.recovery.max_bytes_per_sec / 3600. The default value of indices.recovery.max_bytes_per_sec is 40 MB/s. You can call the GET /_cluster/settings operation to check the current value and adjust it as needed.

Example: For a 3-node cluster with 3 TB of total data and indices.recovery.max_bytes_per_sec set to 100 MB/s, the maximum data volume on a single data node is approximately 1 TB. The data migration duration is about 1 TB / 100 MB/s / 3600 ≈ 2.8 hours. The total blue-green update duration is approximately 10 minutes + 2.8 hours.

Note

Not all in-place update operations trigger a rolling restart. For V3 architecture clusters, the service impact varies by upgrade scenario:

Storage capacity upgrade: Performed as an online expansion. No node restarts, and running tasks are not affected during the change.
Horizontal scaling (adding data nodes): Existing data nodes are not restarted. After the change completes, the cluster automatically rebalances shards, and the data migration consumes some cluster resources.
Node specification or disk type upgrade: Performed by default as a Blue-green Update. New nodes are added, data is migrated to them, and the old nodes are then removed. Business requests may experience minor exceptions at the end of the change, during the node switchover.

For V2 architecture clusters, node specification changes still restart nodes one at a time to complete the update.

Forced Update: Skips health checks and forces a cluster restart. Recovery time depends on data volume. Use only for emergency scaling when the cluster is already unavailable.

Read the Terms of Service and Service Level Agreement. If you agree, click Buy Now. Billing follows the selected method.
During the change, cluster status changes to Initializing with possible performance fluctuations and transient request failures. After completion, status returns to Normal.

Method 2: Upgrade via API

Call the UpdateInstance API operation.

Monitor and verify

After the upgrade starts, check progress on the Basic Information page in the Elasticsearch Clusters console:
Click Show Details:
After the upgrade, verify the new configuration on the Basic Information page:
- The cluster status returns to Active.
- Availability zone
- Node count and storage: Confirm new nodes joined the cluster and storage specifications are correct.
- Shard balance: Run GET _cat/allocation?v to check shard distribution. If unbalanced, use Solutions for uneven cluster load.

Troubleshoot an upgrade stuck in the Initializing state

Expected duration: Expanding ESSD cloud disks usually takes 30 minutes to 1 hour. If the upgrade stays in the Initializing state for more than 2 hours, investigate further.
Check shard allocation on V2 architecture clusters: During expansion, shard allocation may be automatically disabled (cluster.routing.allocation.enable set to none), which can make the upgrade appear stuck. Run the following command in Kibana Dev Tools to re-enable shard allocation:
```
PUT /_cluster/settings
{
  "transient" : {
    "cluster.routing.allocation.enable" : "all"
  }
}
```
Speed up data migration: If data migration is slow, increase the node data transfer bandwidth in the console to 80 to accelerate the migration.
Connection timeouts are expected: Instance restarts during an upgrade or disk expansion can cause transient connection timeouts on the business side. This is expected behavior. Configure a retry mechanism on your client to handle these timeouts.

FAQ

Upgrades and performance metrics

Q: Does upgrading always fix performance?

A: Not necessarily. Upgrading CPU, memory, or disk does not fix all issues. Identify the specific bottleneck first:

CPU bottleneck: Sustained utilization above 80% (not brief spikes).
Memory bottleneck: Frequent long garbage collection (GC) times, memory swapping, or OutOfMemory (OOM) errors.
Disk bottleneck: Check actual I/O throughput, not just %util (see below).
Network bottleneck: Network bandwidth is consistently saturated, and latency is noticeably high.

Confirm the bottleneck before upgrading. Optimizing indices, queries, or configurations is often more effective than adding resources.

Q: Why is my system normal if %util is 100%?

A: The %util metric in iostat measures the percentage of time a device was busy with I/O, not the I/O volume. Modern disks process multiple I/O requests in parallel, so 100% %util does not mean the device is saturated.

For example, a disk takes 0.1 seconds to process a single I/O request and can handle 10 requests simultaneously.
- If 10 I/O requests are submitted sequentially, it takes 1 second to complete them, and %util is 100%.
- If 10 I/O requests are submitted at once, they are processed in parallel and complete in 0.1 seconds. Measured over a one-second interval, this results in a %util of 10%.

The %util metric is largely irrelevant for modern storage and should not drive upgrade decisions alone.

Q: What metrics indicate a disk bottleneck?

A: Focus on these metrics instead of %util:

I/O throughput: Actual read/write rate (MB/s) versus your requirements.
Response time: The latency of I/O requests, which directly affects query performance.
IOPS: I/O operations per second for your workload pattern.
Queue depth: The number of pending I/O requests.

A %util of 100% that is not sustained for hours is usually not concerning. Use tools like fio to benchmark actual maximum bandwidth and IOPS.

Q: How to assess a disk performance bottleneck?

A: Evaluate disk performance as follows:

Analyze your workload: Determine the read/write ratio, I/O size, and access patterns.
Compare actual throughput against requirements, not %util.
Benchmark with fio: Simulate your actual workload to test real-world disk performance.
Evaluate query latency: Verify ES query response times meet your requirements.
Monitor ES metrics: Track indexing latency and search latency.

Base upgrade decisions on a complete set of metrics — CPU utilization, memory usage, I/O throughput, and query latency — not a single metric.

Q: Why does CPU utilization spike above 90% and cause node disconnection on a small-specification (≤2 vCPU 4 GiB) ES node with no business load? How do I resolve this?

A: This is usually caused by one or more of the following:

Insufficient resource specification: On a ≤2 vCPU 4 GiB node, system processes consume a disproportionate share of the already limited resources. We do not recommend using ≤2 vCPU 4 GiB specifications in production environments.
High heap memory watermark: If heap memory usage stays at or above 85%, even a minor fluctuation can fill the heap, triggering frequent Full GC cycles that cause CPU spikes.
Amplified monitoring overhead: On low-specification nodes, maintaining monitoring indices consumes a disproportionately large share of resources.

To resolve this issue:

Emergency mitigation: Restart the affected node to release resources. This is a temporary measure.
Permanent fix: Upgrade the node specification to 4 vCPU 8 GiB or higher. For upgrade steps, see the Method 1: Upgrade via console section in this topic.

Upgrade operations

Q: Can I schedule when an upgrade or disk expansion takes effect, or does it run immediately?

A: No. Neither upgrading a cluster (for example, changing specifications or adding nodes) nor expanding disk capacity supports scheduled execution. Both operations take effect immediately after you submit and pay: the system immediately starts the change and, for an upgrade, the corresponding restart process. You cannot schedule a specific execution time for either operation.

If you want the change to take effect at a specific time (for example, during off-peak hours at 3:00 AM), submit and pay for the upgrade or disk expansion at that time.

Q: Can I read and write normally during an upgrade or disk expansion? How long does it take?

A: Under normal conditions — when the cluster is healthy (GREEN), indices have replicas, and resource utilization is within safe limits — read and write operations continue to work during an upgrade or disk expansion. However, the process triggers a rolling restart of nodes, which may cause brief service jitters or increased latency. We recommend that you perform these operations during off-peak hours and configure a retry mechanism on your client.

Estimated duration for V2 architecture instances:

Update method	Estimated duration	Notes
In-place restart (data nodes)	Approximately 60–90 minutes	Duration depends on the number of nodes and data volume.
Blue-green update	Approximately 15 hours	Default data migration rate is 40 MB/s. The maximum recommended rate is 100 MB/s.

Rebalancing runs concurrently with the restart and does not add extra time.

Important

If any index does not have replicas, forced changes or restarts may cause occasional access timeouts. We recommend that you add replicas to all indices before proceeding.

Q: Do I need to rebuild indexes after adding data nodes?

A: No. Adding data nodes is an online operation that does not require you to rebuild indexes. The cluster automatically rebalances and redistributes data across all nodes, including the new ones. Your existing indexes, data, and business queries are not interrupted during this process.

Q: How do I ensure business continuity during a cluster upgrade?

A: Before the upgrade, make sure that each index has at least one replica and that resource utilization stays at normal levels. Transient request interruptions may occur during the upgrade. Configure a retry mechanism on the application side, and switch traffic over promptly when a failure occurs.

Elasticsearch:Upgrade cluster configuration

Before you begin

Recommended specifications for common scenarios

Pre-upgrade checks

Determine the architecture version of your cluster

Method 1: Upgrade via console

Method 2: Upgrade via API

Monitor and verify

Troubleshoot an upgrade stuck in the Initializing state

FAQ

Upgrades and performance metrics

Q: Does upgrading always fix performance?

Q: Why is my system normal if %util is 100%?

Q: What metrics indicate a disk bottleneck?

Q: How to assess a disk performance bottleneck?

Q: Why does CPU utilization spike above 90% and cause node disconnection on a small-specification (≤2 vCPU 4 GiB) ES node with no business load? How do I resolve this?

Upgrade operations

Q: Can I schedule when an upgrade or disk expansion takes effect, or does it run immediately?

Q: Can I read and write normally during an upgrade or disk expansion? How long does it take?

Q: Do I need to rebuild indexes after adding data nodes?

Q: How do I ensure business continuity during a cluster upgrade?

Other questions