DataWorks provides Elasticsearch Reader and Elasticsearch Writer for you to read data from and write data to Elasticsearch data sources. You can use the codeless user interface (UI) or code editor to configure synchronization nodes for Elasticsearch data sources.

Procedure

  1. Go to the Data Source page.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. After you select the region where the required workspace resides, find the workspace and click Data Integration in the Actions column.
    4. In the left-side navigation pane of the Data Integration page, choose Data Source > Data Sources to go to the Data Source page.
  2. On the Data Source page, click Add data source in the upper-right corner.
  3. In the Add data source dialog box, click Elasticsearch in the NoSQL section.
  4. In the Add Elasticsearch data source dialog box, configure the parameters.
    You can use one of the following modes to add an Elasticsearch data source: Alibaba Cloud instance mode and Connection string mode.
    • The following table describes the parameters you must configure if you add an Elasticsearch data source by using the Alibaba Cloud instance mode. ES
      Parameter Description
      Data source type The type of the data source. Set this parameter to Alibaba Cloud instance mode.
      Data Source Name The name of the data source. The name can contain letters, digits, and underscores (_) and must start with a letter.
      Data source description The description of the data source. The description can be a maximum of 80 characters in length.
      Environment The environment in which the data source is used. Valid values: Development and Production.
      Note This parameter is displayed only when the workspace is in standard mode.
      Region The region where the data source resides.
      Username The username that is used to connect to the database.
      Password The password that is used to connect to the database.
    • The following table describes the parameters you must configure if you add an Elasticsearch data source by using the connection string mode. ES
      Parameter Description
      Data Source Name The name of the data source. The name can contain letters, digits, and underscores (_) and must start with a letter.
      Data source description The description of the data source. The description can be a maximum of 80 characters in length.
      Environment The environment in which the data source is used. Valid values: Development and Production.
      Note This parameter is displayed only when the workspace is in standard mode.
      Endpoint The endpoint of Elasticsearch. Specify this parameter in the format of http://esxxxx.elasticsearch.aliyuncs.com:9200.
      Authentication The authentication mode. Valid values: Login with username and password and Anonymous.
      Username The username that is used to connect to the database.
      Password The password that is used to connect to the database.
  5. Set Resource Group connectivity to Data Integration.
  6. Find the desired resource group in the resource group list in the lower part of the dialog box and click Test connectivity in the Actions column.
    A synchronization node can use only one type of resource group. To ensure that your synchronization nodes can be normally run, you must test the connectivity of all the resource groups for Data Integration on which your synchronization nodes will be run. If you want to test the connectivity of multiple resource groups for Data Integration at a time, select the resource groups and click Batch test connectivity. For more information, see Select a network connectivity solution.
    Note
    • By default, the resource group list displays only exclusive resource groups for Data Integration. To ensure the stability and performance of data synchronization, we recommend that you use exclusive resource groups for Data Integration.
    • If you want to test the network connectivity between the shared resource group or a custom resource group and the data source, click Advanced below the resource group list. In the Warning message, click Confirm. Then, all available shared and custom resource groups appear in the resource group list.
  7. After the data source passes the connectivity test, click Complete.

What to do next

You have learned how to add an Elasticsearch data source. You can proceed to subsequent tutorials. In subsequent tutorials, you will learn how to configure Elasticsearch Reader and Elasticsearch Writer. For more information, see Elasticsearch Reader or Elasticsearch Writer.