You can configure private connections to enable communication between virtual private clouds (VPCs) and prevent security risks that are caused by access over the Internet. This topic describes how to configure a private connection for an Alibaba Cloud Elasticsearch cluster.

Background information

In October 2020, the network architecture of Alibaba Cloud Elasticsearch was adjusted. After this adjustment, some features of Elasticsearch are limited. You can use the PrivateLink service to establish private connections between the exclusive VPC for Elasticsearch and your VPC to resolve some communication issues.

To use PrivateLink to establish private connections, you must create endpoint services and endpoints.
  • Endpoint services

    Endpoint services within a VPC can be accessed by other VPCs over private connections. You must create endpoints for these VPCs to establish private connections. Endpoint services are created and managed by service providers.

  • Endpoints

    You can associate an endpoint with an endpoint service to establish private connections. These connections allow a VPC to access external services. For Elasticsearch, endpoints are automatically created and managed by the service account of Elasticsearch.

The following table describes the Elasticsearch features that are limited due to the network architecture adjustment and can be implemented by using PrivateLink.
Feature Description
Watcher X-Pack Watcher can monitor system information based on query criteria and report alerts.
Security features X-Pack supports a variety of cluster-level security features, such as single sign-on, Lightweight Directory Access Protocol (LDAP) authentication, and user authentication.
External dictionary access of custom plug-ins Custom plug-ins can dynamically access external dictionaries.

Prerequisites

  • An Alibaba Cloud Elasticsearch cluster is created. For more information, see Create an Alibaba Cloud Elasticsearch cluster.
  • Elastic Compute Service (ECS) instances are created in your virtual private cloud (VPC), and the required applications are deployed on the ECS instances. For more information, see Create an instance by using the wizard.
    Note
    • The ECS instances are used as backend servers to receive requests that are forwarded by a Server Load Balancer (SLB) instance. The ECS instances can be deployed in zones that are different from the SLB instance but must be deployed in the same VPC and region as the SLB instance.
    • If a PrivateLink endpoint service is created in your VPC, SLB service resources are configured for the endpoint service, and the health check states of backend servers are Normal, you can directly configure a private connection for your Elasticsearch cluster and obtain the domain name of the related endpoint. For more information, see Step 4: Configure a private connection for the Elasticsearch cluster and View the domain name of an endpoint.

Limits

Only some regions support PrivateLink. For more information, see Regions and zones that support PrivateLink. The following table lists the regions and zones that support both Elasticsearch and PrivateLink.
Region Zone
Germany (Frankfurt) Zone A and Zone B
China (Beijing) Zone H and Zone G
China (Shenzhen) Zone D and Zone E
China (Hong Kong) Zone B and Zone C
China (Zhangjiakou) Zone A and Zone B
Singapore (Singapore) Zone B and Zone C
China (Shanghai) Zone E and Zone G
UK (London) Zone A and Zone B
China (Heyuan) Zone A and Zone B
China (Hangzhou) Zone H and Zone I
Malaysia (Kuala Lumpur) Zone A and Zone B

Precautions

You can configure private connections only for clusters that are deployed in the new network architecture. Only clusters created before October are deployed in the original network architecture (including Alibaba Gov Cloud and Alibaba Finance Cloud). Clusters created in October or later are deployed in the new network architecture.

Procedure

  1. Step 1: Create a CLB instance that supports PrivateLink
    Only Classic Load Balancer (CLB) instances that support PrivateLink can serve as service resources for endpoint services. Before you use PrivateLink to establish private connections to access services across VPCs, you must create a CLB instance that supports PrivateLink.
  2. Step 2: Configure the CLB instance
    After you create a CLB instance, you must add at least one listener and one group of backend servers to the CLB instance. This way, connection requests can be directed to the CLB instance.
  3. Step 3: Create an endpoint service
    Endpoint services within a VPC can be accessed by other VPCs over private connections. After you configure the CLB instance, you must create an endpoint service.
  4. Step 4: Configure a private connection for the Elasticsearch cluster
    An endpoint is automatically created. You can use the endpoint to connect the Elasticsearch cluster to the endpoint service.

Step 1: Create a CLB instance that supports PrivateLink

  1. Log on to the CLB console.
  2. In the left-side navigation pane, choose CLB (FKA SLB) > Instances.
  3. On the Instances page, click Create CLB.
  4. On the buy page, configure the parameters.
    Buy page
    Key parameter Description Example value
    Region, Primary Zone, and Backup Zone Select the region and zones where you want to deploy the CLB instance. Make sure that the CLB instance and the ECS instances you want to add to the CLB instance are deployed in the same region. China (Hangzhou), China East 1 Zone I, and China East 1 Zone H
    Instance Type Valid values: Internet and Intranet.
    • Internet: If you select Internet, a public IP address is allocated to the CLB instance. You can access the CLB instance over the Internet.
    • Intranet: If you select Intranet, a private IP address is allocated to the CLB instance. You can access the CLB instance only over an internal network.
    Note Only CLB instances that are deployed in internal networks support PrivateLink.
    Intranet
    VPC and Virtual switch Select a VPC and a vSwitch based on your business requirements.
    • elasticsearch-vpc-test
    • elasticsearch-vswitch-test
    Feature Select the feature or service that the CLB instance supports. Valid values: Standard and Support PrivateLink. Support PrivateLink

    For information about other parameters, see Create a CLB instance that supports PrivateLink.

  5. Click Buy Now and complete the payment.

Step 2: Configure the CLB instance

  1. On the Instances page, find the CLB instance that is created in Step 1 and click Configure in the Port/Health Check/Backend Server column.
  2. In the Protocol and Listener step of the Configure Server Load Balancer wizard, configure the parameters based on your business requirements.
    Protocol and Listener step
    Key parameter Description Example value
    Select Listener Protocol TCP, UDP, HTTP, and HTTPS are supported. TCP
    Listening Port The frontend port that is used to receive requests and forward the requests to backend servers. 8080
    Listener Name Enter a name based on your requirements. If you do not specify a name, a name in the Protocol_Port number format is used by default. test
    Advanced You can configure the parameters in this section based on your business requirements or retain the default values of the parameters. Default values
  3. Click Next.
  4. Configure backend servers.
    1. In the Backend Servers step, select Default Server Group for Forward Requests To and click Add More in the Servers Added section.
    2. In the My Servers panel, select the ECS instances that you created and click Next in the Select Servers step.
    3. In the Configure Ports and Weights step, specify weights for the ECS instances based on your business requirements. In this example, the default value 100 is used.
      Note The greater the weight of an ECS instance, the more requests the instance receives.
    4. Click Add.
    5. In the Backend Servers step, specify the ports that are enabled on the backend servers to receive requests. You can specify the same port for all backend servers that belong to the same CLB instance. In this example, port 8080 is used.
      Specify ports for backend servers
  5. Click Next. In the Health Check step, configure the parameters based on your business requirements. In this example, the default values of the parameters are used.
  6. Click Next to go to the Confirm step. After you confirm the configuration information, click Submit.
  7. In the Configure Server Load Balancer message, click OK. The Instances page appears.

    If the health check state of an ECS instance is Normal, the ECS instance is ready to process requests.

Step 3: Create an endpoint service

  1. Log on to the VPC console.
  2. In the left-side navigation pane, click Endpoints Service.
  3. In the top navigation bar, select the region in which you want to create an endpoint service. In this example, the China (Hangzhou) region is selected.
  4. On the Endpoints Service page, click Create Endpoint Service.
  5. On the Create Endpoint Service page, configure the parameters based on your business requirements.
    Create Endpoint Service
    Parameter Description
    Select Service Resource Select a zone to which you want to distribute network traffic. Then, select the CLB instance that you want to associate with the endpoint service. The zone where an endpoint service is deployed must be the same as the primary zone where the CLB instance you want to associate with the endpoint service is deployed. CLB instances serve as service resources and can be associated with endpoint services. The CLB instances that are associated with endpoint services receive requests from clients.
    CLB instances can serve as service resources only if they meet the following requirements:
    • Network Type is set to VPC.
    • Feature is set to Support PrivateLink.
    Automatically Accept Endpoint Connections Specifies whether to automatically accept connection requests from endpoints. Valid values:
    • Yes: The endpoint service accepts all connection requests from an endpoint that is associated with the endpoint service. In this case, you can use the associated endpoint to access the endpoint service. We recommend that you set this parameter to Yes.
    • No: The endpoint connection of the endpoint service is in the Disconnected state. In this case, endpoint connection requests to the endpoint service must be manually accepted or denied by the service administrator.
      • If the service administrator accepts endpoint connection requests from the associated endpoint, you can use the associated endpoint to access the endpoint service.
      • If the service administrator denies endpoint connection requests from the associated endpoint, you cannot use the associated endpoint to access the endpoint service.
    Note
    • If you set Automatically Accept Endpoint Connections to Yes, the value of Endpoint Connection Status in the Configure Private Connection panel of the Elasticsearch console is Connected. In this case, you can click Deny Connection in the Actions column.
    • If you set Automatically Accept Endpoint Connections to No, the value of Endpoint Connection Status in the Configure Private Connection panel of the Elasticsearch console is Disconnected. In this case, you can click Allow Connection in the Actions column.
    Whether to Enable Zone Affinity We recommend that you set this parameter to Yes.
    Description Enter a description for the endpoint service. The description must be 2 to 256 characters in length and cannot start with http:// or https://.
  6. Click OK.

Step 4: Configure a private connection for the Elasticsearch cluster

  1. Log on to the Elasticsearch console.
  2. In the left-side navigation pane, click Elasticsearch Clusters.
  3. Navigate to the desired cluster.
    1. In the top navigation bar, select a resource group and a region.
    2. In the left-side navigation pane, click Elasticsearch Clusters. On the Elasticsearch Clusters page, find the desired cluster and click its ID.
  4. In the left-side navigation pane of the Basic Information page, click Security.
  5. In the Network Settings section, click Edit on the right side of Configure Private Connection.
  6. In the Configure Private Connection panel, click Add Private Connection. In the Create Private Connection dialog box, select the endpoint service that is created in Step 3 and select a zone. Then, select the check box.
    Select a zone
  7. Click OK. Then, the endpoint service attempts to connect to the associated endpoint. If the value of Endpoint Connection Status is Connected, the endpoint service is connected to the associated endpoint.
    Connected
    Note
    • If you set Automatically Accept Endpoint Connections to Yes, the value of Endpoint Connection Status in the Configure Private Connection panel of the Elasticsearch console is Connected. In this case, you can click Deny Connection in the Actions column.
    • If you set Automatically Accept Endpoint Connections to No, the value of Endpoint Connection Status in the Configure Private Connection panel of the Elasticsearch console is Disconnected. In this case, you can click Allow Connection in the Actions column.
    If you want to obtain the domain name of the endpoint, perform operations based on the instructions in View the domain name of an endpoint.

View the domain name of an endpoint

You can use the domain names of endpoints to access the endpoint services with which the endpoints are associated. To view the domain name of an endpoint, perform the following steps:

  1. In the Configure Private Connection panel, click the ID of the endpoint in the Endpoint ID column.
    Click the endpoint ID
  2. On the Endpoint Connections tab of the page that appears, click the Expand icon next to the ID of the endpoint. Then, you can view the domain name of the endpoint.
    View the domain name