This topic describes how to use the cross-cluster replication (CCR) feature to migrate
data between a local Alibaba Cloud Elasticsearch cluster and a remote Alibaba Cloud
Elasticsearch cluster.
Background information
CCR is a commercial feature released in open source Elasticsearch Platinum. After you purchase an Alibaba Cloud Elasticsearch cluster, you can use this feature free of charge based on a few simple configurations. Only
single-zone Elasticsearch clusters of V6.7.0 or later support this feature. CCR is
used in the following scenarios:
- Disaster recovery and high availability
You can use CCR to back up data among Elasticsearch clusters that reside in different
regions. If a cluster fails, you can retrieve its index data from other clusters.
This prevents data loss.
- Data access from a nearby cluster
For example, Company A has multiple subsidiaries that are located in different regions.
To speed up business processing, you can plan the business of the subsidiaries based
on their geographical locations. Then, use CCR to distribute business data to Elasticsearch
clusters in different regions. Each subsidiary can directly use the cluster in the
region where the subsidiary is located to process business.
- Centralized reporting
You can use CCR to replicate data from multiple small clusters to one cluster. Then,
you can perform visualized analytics and reporting for the data in a centralized manner.
To use CCR, you must prepare two types of clusters: local clusters and remote clusters.
Remote clusters provide source data, which is stored in leader indexes. Local clusters
replicate the data and store it in follower indexes. You can also use CCR to migrate
large volumes of data at a time in real time. For more information, see Cross-cluster replication.
Preparations
- Prepare a local cluster and a remote cluster.
For more information, see
Create an Alibaba Cloud Elasticsearch cluster. The two clusters must be single-zone clusters and belong to the same virtual private
cloud (VPC) and VSwitch. The versions of the two clusters must be the same (V6.7.0
or later).
- Log on to the Kibana console of the remote cluster and create a leader index.
Notice
- When you create an index in an Elasticsearch cluster of V7.0 or earlier, you must
enable the soft_deletes attribute. Otherwise, an error is reported.
- If you want to migrate data in an existing index, you can call the reindex operation
to enable the soft_deletes attribute.
PUT myindex
{
"settings": {
"index.soft_deletes.retention.operations": 1024,
"index.soft_deletes.enabled": true
}
}
- Disable the physical replication feature for the leader index.
The
physical replication feature is automatically enabled for indexes in Elasticsearch V6.7.0 clusters. Before
you use CCR, you must disable the physical replication feature.
- Disable the index.
- Update the settings configuration of the index to disable the physical replication
feature.
PUT myindex/_settings
{
"index.replication.type" : null
}
- Enable the index.
Step 1: Connect clusters
Configure the remote cluster to connect it to the local cluster. For more information,
see
Connect two Elasticsearch clusters. If the two clusters are connected, the information shown in the following figure
appears.

Step 2: Add the remote cluster
- Log on to the Kibana console of the local cluster.
- In the left-side navigation pane, click Management.
- In the Elasticsearch section, click Remote Clusters.
- Click Add a remote cluster.
- In the Add remote cluster section, configure the following parameters.

- Name: the name of the remote cluster. The name must be unique.
- Seed nodes: the nodes in the remote cluster. Specify each node in the format of Node IP address:9300. To obtain the IP addresses of nodes, log on to the Kibana console of the remote
cluster and run the
GET /_cat/nodes?v
command on the Console tab of the Dev Tools page. The nodes you specify must include
a dedicated master node of the remote cluster. We recommend that you specify multiple
nodes. This way, if the dedicated master node fails, you can still use CCR.
Notice During CCR, Kibana uses the IP addresses of data nodes to access clusters over TCP
port 9300. HTTP port 9200 is not supported.
- Click Save.
The system then automatically connects to the remote cluster. If the connection is
established,
Connected appears.

Step 3: Configure CCR
- Log on to the Kibana console of the local cluster. In the left-side navigation pane,
click Management. In the Elasticsearch section of the page that appears, click Cross Cluster Replication.
- On the Follower indices tab, click Create a follower index.
- In the Add follower index section, configure the following parameters.

Parameter |
Description |
Remote cluster |
Select the cluster you added in Step 2: Add the remote cluster.
|
Leader index |
The index whose data you want to migrate. In this example, the myindex index that is created in Preparations is used.
|
Follower index |
The index to which data is migrated. You must specify a unique name. |
- Click Create.
After the follower index is created, the index is in the
Active state.

Step 4: View migration results
- Log on to the Kibana console of the remote cluster and insert data into the cluster.
POST myindex/_doc/
{
"name":"Jack",
"age":40
}
- Run the following command in the Kibana console of the local cluster to check whether
the inserted data is migrated to the local cluster:
GET myindex_follow/_search
If the command is successfully executed, the result shown in the following figure
is returned.

The preceding figure shows that data in the leader index myindex of the remote cluster
is migrated to the follower index myindex_follow of the local cluster.
- Insert a data record into the remote cluster and check whether the data record is
migrated to the local cluster in real time.
POST myindex/_doc/
{
"name":"Pony",
"age":50
}
Query the inserted data record in the local cluster immediately. The following figure
shows the data record.

The preceding figure shows that the CCR feature can implement real-time migration
of incremental data.
FAQ
Q: I can use port 9300 to add a remote cluster. Why is only port 9200 accessible when I use a domain name to access an Elasticsearch
cluster?
A: Port 9300 is an open port. However, when you access a cluster over the Internet,
Server Load Balancer (SLB) enables only port 9200 during port verification for security
purposes. This will be adjusted in the future.