Alibaba Cloud Elasticsearch (ES) instances run on one of two architectures: a basic control architecture (v2) or a cloud-native control architecture (v3). Instances created in or after October 2020 use the v3 architecture. Due to network limitations in this architecture, some features such as X-Pack Watcher, reindex, and LDAP and AD authentication cannot be used directly. To resolve this issue, you must use PrivateLink and a load balancer to establish a private connection. This connection can link your ES instance to other cloud resources (such as ECS, ENI, or ECI) or to another ES instance. This is the only solution to overcome these feature limitations for v3 instances.
Instances created before October 2020, including those in Alibaba Finance Cloud and Alibaba Gov Cloud, use the v2 architecture. These instances do not have these network limitations and do not require a private connection.
Use cases for Alibaba Cloud ES PrivateLink
Accessing external services from an ES instance for features like Watcher alerts, LDAP, and Active Directory.
You can configure a private connection between an Alibaba Cloud ES instance and an Alibaba Cloud ECS instance. The ECS instance acts as a business or proxy server to forward network traffic. The following figure shows the architecture.
The following table describes the specific use cases and features.
Feature
Description
References
Watcher
The X-Pack feature of Alibaba Cloud ES monitors system information based on query conditions and sends alerts to external services.
Security features
The X-Pack feature of Alibaba Cloud ES supports various cluster-level security settings, such as LDAP and user authentication.
Cross-cluster reindex
Write data from a self-managed ES cluster to an Alibaba Cloud ES instance by using cross-cluster reindex.
Migrate data between two Alibaba Cloud ES instances by using cross-cluster reindex. If two ES instances are deployed in VPC_1 and VPC_2 respectively, you can configure a private connection for ES_1 in VPC_1. This establishes a private connection between the endpoint in VPC_1 and the endpoint service in VPC_2. The following figure shows the architecture.
Private connection between ES and ECS
Prerequisites
An Alibaba Cloud ES instance must exist in VPC_1, created in or after October 2020. For more information, see Create an Alibaba Cloud Elasticsearch instance.
An ECS instance with the required applications must exist in a user-owned VPC_2. For more information, see Create an instance by using the wizard.
The ECS instance serves as a backend server that receives requests forwarded by the load balancer instance. When you create the ECS instance, you must select a region and zone that support PrivateLink. For more information about supported regions, see Regions and zones that support PrivateLink.
The Alibaba Cloud ES, ECS, and load balancer instances must be in the same region and zone.
Step 1: Create and configure a load balancer
Log on to the Classic Load Balancer (CLB) console.
Create a load balancer instance that supports PrivateLink.
On the Instances page, click Create CLB.
On the Classic Load Balancer (CLB) (Pay-As-You-Go) purchase page, configure the instance. Select the same region as your Alibaba Cloud ES instance, set Instance Type to Intranet, and then click Buy Now.
Configure the load balancer instance.
On the Instances page, find the target instance and click Configure Listener in the Actions column.
On the Configure Server Load Balancer page, configure the listener and backend servers as needed, and then perform a health check and review the configuration.
Step 2: Create an endpoint service
The private connection feature automatically creates an endpoint in the ES instance's VPC. You only need to manually create an endpoint service in the ECS instance's VPC.
In the top menu bar, select the region where you want to create the endpoint service.
The endpoint service must be in the same region as the load balancer instance.
Click Create Endpoint Service.
On the Create Endpoint Service page, for Service Resource, select the load balancer instance that you created. Configure the other parameters and then click OK.
Step 3: Configure the private connection
Log on to the Alibaba Cloud Elasticsearch console.
In the left-side navigation pane, choose Configuration and Management>Security Settings.
In the Network Settings area, click Configure Private Connection.
Click Add Private Connection and select the endpoint service that you created in the previous step.
Wait for the connection status to become Connected.
Obtain the PrivateLink domain name.
After the connection is successful, return to the PrivateLink console. On the Endpoint Connection Status tab of the endpoint service, find the domain name of the automatically created endpoint. You will use this domain name for remote cluster configurations.
Private connection between ES instances
This topic describes how to use Network Load Balancer (NLB) and PrivateLink to establish a private connection between Alibaba Cloud Elasticsearch instances in a same-region disaster recovery scenario. By configuring a secure private link and cross-cluster replication (CCR), you can achieve near-real-time data synchronization for high availability and disaster recovery.
For detailed steps, see Same-region Elasticsearch data replication.
Key concepts
To implement private access by using PrivateLink, you need to create an endpoint service and an endpoint.
Term | Description |
endpoint service | An endpoint service is a service that can be accessed from other VPCs by creating an endpoint to establish a private connection. You need to manually create the related service resources. |
endpoint | An endpoint connects to an endpoint service, enabling private access to external services from within your VPC. When you configure a private connection for an Alibaba Cloud ES instance, an endpoint is created automatically in the instance's VPC. |