Alibaba Cloud Elasticsearch supports four migration tools: OSS snapshots, Logstash, the reindex API, and elasticsearch-dump. Each suits different data volumes, network topologies, and version requirements. Use the tables below to match your scenario to the right solution.
The network architecture of Alibaba Cloud Elasticsearch changed in October 2020. Clusters created before October 2020 run on the original architecture; clusters created in October 2020 or later run on the new architecture. Cross-cluster operations — reindex, searches, and replication — are not supported between clusters on different architectures. Make sure both clusters use the same architecture before attempting these operations. The adjustment date for the China (Zhangjiakou) region and regions outside China is uncertain. If you need to run cross-cluster operations between a cluster created before October 2020 and one created in October 2020 or later in these regions, submit a ticket to verify whether the clusters can connect.
We recommend that you do not migrate system indices whose names start with a period (.), such as.monitoring,.kibana, and.security. Migrating these indices may cause Kibana to fail.
Choose a solution
Each tool has a distinct role:
Tool | What it does | Best for |
OSS snapshots | Streams index data via the Elasticsearch snapshot mechanism | GB, TB, or PB-scale data; same or cross-account migration |
Logstash | Streams data between clusters with filtering support | Incremental or filtered migration without strict real-time requirements |
reindex API | Migrates data between clusters using the reindex API | Small datasets; query-result migration from Kibana |
elasticsearch-dump | Exports and imports index data via a standalone tool | Small datasets with a small number of indices |
OSS snapshots + reindex API | Chains snapshot restore and reindex through an intermediate version cluster | Cross-major-version migration where snapshots alone are incompatible |
Migration between Alibaba Cloud Elasticsearch clusters
Solution | Best when | Constraints | Examples |
OSS snapshots | The source stores GB, TB, or PB of data. | For incremental migration, you may need to stop the Elasticsearch service and must disable the destination index before migration. For a shared OSS repository, clusters must be in the same region, under the same Alibaba Cloud account or RAM user, and the source version must be ≤ the destination version. For more information about version compatibility, see Version compatibility of data restoration from snapshots. For cross-account migration, both clusters must use the AccessKey pair of the account that owns the OSS bucket. | Configure a shared OSS repository / Create manual snapshots and restore data |
Logstash | Full or incremental migration where real-time performance is not critical. Migrate only query results or filtered data. Migrate between clusters of different major versions, such as V5.X to V6.X or V7.X. | The source cluster, Logstash cluster, and destination cluster must be in the same Virtual Private Cloud (VPC). If they are in different VPCs, configure NAT gateways for the Logstash cluster to route traffic over the Internet. Version compatibility must be met across all three clusters. For incremental migration, the source index ID must match the destination index ID, and a scheduled migration task is required. Cross-account, cross-region migration with different VPCs also requires NAT gateways. | Migrate data between Alibaba Cloud Elasticsearch clusters / Use the reindex API to migrate data in a multi-type index of an earlier version |
reindex API | Small data volumes with no strict speed requirements. Migrate only query results from the Kibana console. | Both clusters must use the same network architecture. See Use the reindex API to migrate data in a multi-type index of an earlier version. | |
elasticsearch-dump | Small data volumes with a small number of indices. | Network connectivity must exist among the source Elasticsearch cluster, destination cluster, and the server running the elasticsearch-dump tool. | |
OSS snapshots + reindex API | Full and incremental migration across major versions where snapshots alone are incompatible — for example, V6.X to V8.X. | The intermediate cluster must be compatible with both the source and destination versions. For version compatibility details, see Elastic version changes and compatibility. Example: To migrate from V6.X to V8.X: restore a V6.X snapshot into a V7.X cluster, reindex within V7.X, then restore a V7.X snapshot into V8.X. | Create manual snapshots and restore data / Use the reindex API to migrate data |
To migrate data between clusters in different regions, use Elasticsearch snapshot and restore commands together with the cross-region replication (CRR) feature provided by Object Storage Service (OSS).
Migration from a self-managed Elasticsearch cluster
Solution | Best when | Constraints | Examples |
OSS snapshots | The source stores GB, TB, or PB of data. Migrate snapshots between the self-managed cluster and an Alibaba Cloud Elasticsearch cluster, whether in the same or different regions and accounts. | The elasticsearch-repository-oss plugin must be installed on each node of the Alibaba Cloud Elasticsearch cluster, at the same version as the plugin on the self-managed cluster. For incremental migration, you may need to stop the Elasticsearch service and must disable the destination index before migration. | |
Logstash | Full or incremental migration where real-time performance is not critical. Migrate only query results or filtered data. Migrate between clusters of different major versions, such as a 5.x cluster to V6.X or V7.X. | The source cluster, Logstash cluster, and destination cluster must be in the same VPC. If they are in different VPCs, configure NAT gateways for the Logstash cluster to route traffic over the Internet. Version compatibility must be met across all three clusters. For incremental migration, the source index ID must match the destination index ID, and a scheduled migration task is required. | Use Alibaba Cloud Logstash to migrate data from a self-managed Elasticsearch cluster to an Alibaba Cloud Elasticsearch cluster / Use the reindex API to migrate data in a multi-type index of an earlier version |
reindex API | Small data volumes with no strict speed requirements. Migrate only query results from the Kibana console. When migrating from an older self-managed version (such as 6.x) to a newer Alibaba Cloud Elasticsearch version (such as V8.X), establish private connections between VPCs using PrivateLink before running the reindex operation. | Both clusters must use the same network architecture. See Use the reindex API to migrate data in a multi-type index of an earlier version. | |
elasticsearch-dump | Small data volumes with a small number of indices. | Network connectivity must exist among the source Elasticsearch cluster, destination cluster, and the server running the elasticsearch-dump tool. |
Migration from a third-party Elasticsearch source
Solution | Best when | Constraints | Examples |
OSS snapshots | The source stores GB, TB, or PB of data. This solution is not limited to cloud service providers but depends on the Elasticsearch snapshot mechanism — for example, you can use it to migrate from Amazon OpenSearch Service or other providers. | For incremental migration, you may need to stop the Elasticsearch service and must disable the destination index before migration. The destination cluster version must be the same as or later than the source. For version compatibility, see Version compatibility of data restoration from snapshots. | Migrate Elasticsearch index data from Amazon OpenSearch Service to Alibaba Cloud Elasticsearch |
elasticsearch-dump | Small data volumes with a small number of indices. | Network connectivity must exist among the source Elasticsearch cluster, destination cluster, and the server running the elasticsearch-dump tool. |
To migrate Elasticsearch data from a Tencent Cloud Object Storage (COS) bucket to Alibaba Cloud Elasticsearch, use the Data Online Migration service to move the snapshots from the COS bucket to an Alibaba Cloud OSS bucket, then restore the snapshots to the destination cluster. For migration steps, see Migrate data. For snapshot restoration, see Create manual snapshots and restore data from manual snapshots.