Alibaba Cloud Elasticsearch instances created in or after October 2020 use the cloud-native v3 architecture, which restricts features such as X-Pack Watcher, reindex, LDAP, and AD authentication. To enable these features, use PrivateLink and SLB to establish private connections between your Elasticsearch instance and other cloud resources (ECS, ENI, or ECI) or between multiple Elasticsearch instances. This is the only way to enable these features in the v3 architecture.
Instances created before October 2020, including those on Alibaba Finance Cloud and Alibaba Gov Cloud, use the v2 architecture and do not require PrivateLink.
PrivateLink use cases
Access external network resources for Watcher, LDAP, and AD features.
Configure a private connection between an Elasticsearch instance and an ECS instance that serves as a business server or network proxy.
Applicable features and scenarios:
Feature
Description
Related documentation
Watcher
Monitors system metrics based on query conditions and sends alerts to external services.
Security features
Provides cluster-level security features such as LDAP and user authentication.
Cross-cluster reindex
Writes data from a self-managed Elasticsearch cluster to an Alibaba Cloud Elasticsearch instance.
Migrate data between two Elasticsearch instances by using cross-cluster reindex. In this example, ES_1 in VPC_1 connects to ES_2 in VPC_2 through PrivateLink, linking the endpoint in VPC_1 to the endpoint service in VPC_2.
Connect to an ECS instance with CLB and PrivateLink
Prerequisites
An Alibaba Cloud Elasticsearch v3 instance (created in or after October 2020) exists in VPC_1. Create an Alibaba Cloud Elasticsearch instance.
An ECS instance with the required applications is deployed in VPC_2. Create an instance by using the wizard.
The ECS instance serves as a backend server for the load balancer. Select a PrivateLink-supported region and zone. Regions and zones that support PrivateLink.
The Elasticsearch instance, ECS instance, and load balancer must be in the same zone of the same region.
Step 1: Create and configure a load balancer
Log on to the Classic Load Balancer (CLB) console.
Create a load balancer instance that supports PrivateLink.
On the Manage Cluster page, click Create CLB.
On the Classic Load Balancer (Pay-As-You-Go) page, select the same region as your Elasticsearch instance, set Cluster Type to Internal-facing, and click Buy Now.
Configure the load balancer instance.
On the Manage Cluster page, find the target instance and click Configure Listener in the Actions column.
On the Configure Server Load Balancer page, configure the listener, backend servers, and health check, then review the configuration.
Step 2: Create an endpoint service
PrivateLink automatically creates an endpoint in the Elasticsearch instance's network. You only need to create an endpoint service in the VPC where the ECS instance is deployed.
In the top navigation bar, select the region where you want to create the endpoint service.
The endpoint service must be in the same region as the load balancer.
Click Create Endpoint Service.
On the Create Endpoint Service page, select the load balancer instance you created as the service resource, configure other parameters, and click Confirm and Create.
Step 3: Configure PrivateLink
Log on to the Alibaba Cloud Elasticsearch console.
In the left-side navigation pane, choose Configuration and Management > Security Settings.
In the Network Settings section, click Configure Private Connection.
Click Add Private Connection and select the endpoint service that you created in the previous step.
Wait for the connection status to change to Connected. This indicates that the Elasticsearch cluster in VPC_1 is now privately connected to VPC_2.
Obtain the PrivateLink domain name.
Return to the PrivateLink console. On the Endpoint Connection Status tab of your endpoint service, find the automatically created endpoint connection. Use this domain name when you configure remote clusters.
Connect Elasticsearch instances with NLB and PrivateLink
You can use NLB and PrivateLink to enable private communication between Elasticsearch clusters. The following example uses a same-region disaster recovery scenario with cross-cluster replication (CCR) for near-real-time data synchronization. Perform the following steps:
Connect the VPCs of the Leader cluster and the Follower cluster by using Cloud Enterprise Network (CEN).
Obtain the private IP addresses of the nodes in the Leader cluster.
In the VPC of the Follower cluster, create a Network Load Balancer (NLB) instance and configure a listener that forwards traffic to the Leader cluster nodes.
Create a PrivateLink endpoint service and associate it with the NLB instance.
In the Elasticsearch console of the Follower cluster, add a private connection and select the endpoint service that you created.
After the connection status changes to Connected, obtain the PrivateLink domain name and use it to configure CCR between the two clusters.
For detailed steps, see Replicate data for Elasticsearch instances in the same region.
Glossary
PrivateLink has two components: an endpoint service and an endpoint.
Term | Description |
endpoint service | Enables other VPCs to connect through endpoints. You must manually create the service resources. |
endpoint | Connects to an endpoint service to establish a private connection within a VPC. When you configure PrivateLink for an Elasticsearch instance, an endpoint is automatically created. |
FAQ
Q: The Enable HTTPS button is grayed out and cannot be clicked. What should I do?
In some regions (for example, China (Shenzhen)), a new rule prevents you from enabling HTTPS directly while Public network access is already enabled for the instance. This conflict between public network access and HTTPS causes the Enable HTTPS button to be grayed out (unclickable).
To resolve this issue, perform the following steps on the instance details page in the Alibaba Cloud Elasticsearch console:
Disable Public network access for the instance.
Enable HTTPS.
After HTTPS is enabled, re-enable Public network access.
Q: An InternalServerError occurs when I enable Kibana private access. What should I do?
This error usually occurs because PrivateLink is not enabled in the target region, such as the Malaysia (Kuala Lumpur) region.
To resolve this issue, go to the PrivateLink console, enable PrivateLink in the target region, and then try enabling Kibana private access again.