Elasticsearch getting started - - Alibaba Cloud Documentation Center

This topic demonstrates how to create an Elasticsearch cluster and perform operations such as index creation and data search using the scenario of a financial service enterprise that uses Alibaba Cloud Elasticsearch (ES) to provide search functionality for wealth management products.

Operation notes

Data description: The example wealth management product information is as follows.

Data example

{
    "products": [
        {
            "productName": "Wealth Management Product A",
            "annual_rate": "3.2200%",
            "describe": "180-day wealth management product that requires the minimum investment of USD 20,000, enables low-risk investment, and lets you select whether to receive push messages for returns"
        },
        {
            "productName": "Wealth Management Product B",
            "annual_rate": "3.1100%",
            "describe": "90-day wealth management product that requires the minimum investment of USD 10,000 and daily pushes messages for returns credited to your account"
        },
        {
            "productName": "Wealth Management Product C",
            "annual_rate": "3.3500%",
            "describe": "270-day wealth management product that requires the minimum investment of USD 40,000 and daily pushes messages for returns immediately credited to your account"
        },
        {
            "productName": "Wealth Management Product D",
            "annual_rate": "3.1200%",
            "describe": "90-day wealth management product that requires the minimum investment of USD 12,000 and daily pushes messages for returns credited to your account"
        },
        {
            "productName": "Wealth Management Product E",
            "annual_rate": "3.0100%",
            "describe": "30-day wealth management product that requires the minimum investment of USD 8,000 and daily pushes messages for returns"
        },
        {
            "productName": "Wealth Management Product F",
            "annual_rate": "2.7500%",
            "describe": "3-day popular wealth management product that does not require service fees, requires the minimum investment of USD 500, and pushes messages for returns"
        }
    ]
}

Version information: Alibaba Cloud ES supports multiple versions with different features. This topic demonstrates using version 8.17.
Cost information: This topic demonstrates creating a pay-as-you-go instance, which will incur fees based on actual resource usage. Completing this example is estimated to cost less than 20 CNY.
Kibana: A flexible data analytics and visualization tool that lets you search and view data in ES indexes and interact with it.
RESTful API: Allows interaction with ES clusters through HTTP protocol. This topic uses RESTful API for operations such as index creation, data insertion, searching, and index deletion.

Prerequisites

You have created a virtual private cloud (VPC) and a virtual switch (vSwitch). You can use VPC and vSwitch to ensure network security for your ES instance, enable communication with other resources, and achieve high availability.

Note

When creating an ES instance, you need to select appropriate VPC and vSwitch. The VPC and vSwitch must be in the same region and zone as the ES instance, and cannot be changed after the instance is created.

The core parameter configurations for the VPC and vSwitch in this example are as follows.

Category	Configuration example
VPC	Region: China (Hangzhou) IPv4 CIDR Block: Select manual input, with CIDR block `172.16.0.0/16`.
vSwitch	Zone: Hangzhou Zone I.

Procedure

The procedure for using Alibaba Cloud ES to build a cluster and complete data search operations is as follows.

Create an instance: An Alibaba Cloud ES instance is a managed ES service that provides an easy-to-use, scalable, and stable search engine platform. All operations for data storage, search, and analysis need to be performed within the instance.
Access the instance: Access the instance through Kibana, which will be used for subsequent operations such as index creation, document creation, data insertion, and searching.
Create an index: Used to store the wealth management product information in this example, and define the search type and tokenization method for each field in the information.
Note
An index is a logical container used by ES to store related documents, helping you organize and manage data, and significantly improving query efficiency.
Create documents and insert data: Insert the wealth management product information from this example as documents into the index created in the previous step for subsequent queries.
Search for data: Query the required products through methods such as full-text search and search by condition.
(Optional) Next steps: You can perform operations such as deleting test indexes, migrating data, scaling clusters, or releasing instances as needed.

Step 1: Create an instance

Go to the Elasticsearch buy page.

Configure the instance information.

Configure the instance information according to the following table.

Note

Keep the default configuration for parameters not mentioned. For more information about instance creation parameters, see Create an Alibaba Cloud Elasticsearch cluster.

Parameter	Description
Product Type	Select Pay-as-you-go.
Region And Zone	Select China (Hangzhou) and Zone I.
Number Of Zones	Select Single Zone.
VPC And VSwitch	Select the VPC and vSwitch you created in the Prerequisites.
Instance Type And Version	Select Vector Enhanced Edition with version `8.17.0`.
Instance Name	Configure as ES_test.
Username And Password	The username is elastic by default and cannot be changed; customize the password.

Click Buy Now and follow the instructions to complete the payment. After payment is complete, the instance creation begins.
Note
The creation process takes approximately 20 minutes. Please be patient.
Check the instance status.
1. Go to the Alibaba Cloud Elasticsearch console.
2. In the top menu bar, select the China (Hangzhou) region, click Elasticsearch Clusters in the navigation pane on the left to view the status of the created instance in the instance list.
  When the instance status shows Normal, it indicates that the instance has been created successfully, and you can proceed with the following steps.

Step 2: Access the instance

Access the visualization control.
1. In the Elasticsearch Clusters list, click the target instance ID.
2. In the navigation pane on the left, click Configuration And Management > Data Visualization.
Configure the Kibana whitelist.
This example accesses Kibana through the Internet. To avoid access restrictions, you need to add your client's IP address to the Kibana public network access whitelist.
Note
For more information about Kibana network configuration and FAQ, see Configure access to Kibana over the Internet or a VPC.
1. In the Kibana section, click Modify Configuration.
2. Click Modify next to Kibana Public Network Access Whitelist, and follow the instructions to add your client's IP address to the whitelist.
Access the Kibana Dev Tools.
You can access the ES instance through Kibana's Dev Tools to perform operations such as index creation, document creation, data insertion, and searching.
1. Return to the Data Visualization page, and click Access Over Internet in the Kibana section.
2. Enter the username and password, then click Log In.
  Note
  The username is elastic, and the password is the one you customized in Step 1.
3. On the welcome page, click Explore On My Own.
4. Click the icon in the upper-left corner and select Management > Dev Tools to access the Dev Tools console.
Test the access result.
Run the GET / command in the console tab to access the ES instance. The following result indicates successful access.

Step 3: Create an index

Create an index named product_info, and configure full-text search and coarse-grained tokenization for the productName and describe fields, and exact matching for the annual_rate field. The example code is as follows.

PUT /product_info
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
      "properties": {
        "productName": {
          "type": "text",
          "analyzer": "ik_smart"
        },
        "annual_rate":{
          "type":"keyword"
        },
        "describe": {
          "type": "text",
          "analyzer": "ik_smart"
        }
    }
  }
}

The core parameters are described as follows.

Category

Parameters and description

settings (Configure index)

Define the number of primary shards and replicas.

number_of_shards: Number of primary shards, used for horizontal scaling of data storage and query load.
number_of_replicas: Number of replicas for each primary shard, used to improve fault tolerance and read performance.

mappings (Define field mappings)

Define the search type and tokenization method for the productName, annual_rate, and describe fields in the wealth management product information.

type: Define the data type and search method for fields.
- Configured as text: Indicates that the relevant fields will be full-text searchable, supporting tokenization and fuzzy matching.
- Configured as keyword: Indicates that the relevant fields will be exactly matched without tokenization.
analyzer: Define the tokenization method for fields. This example uses the Chinese tokenizer plugin analysis-ik for coarse-grained tokenization.

The following result indicates that the index has been created successfully.

Step 4: Create documents and insert data

Use the _bulk API to batch insert multiple documents into the product_info index. Each document contains wealth management product information (including product name, annual rate, and description fields). The example code is as follows.

Note

{"index":{}} indicates inserting a new document, and ES will automatically generate a unique ID for the document.

POST /product_info/_bulk
{"index":{}}
{"productName":"Wealth Management Product A","annual_rate":"3.2200%","describe":"180-day wealth management product that requires the minimum investment of USD 20,000, enables low-risk investment, and lets you select whether to receive push messages for returns"}
{"index":{}}
{"productName":"Wealth Management Product B","annual_rate":"3.1100%","describe":"90-day wealth management product that requires the minimum investment of USD 10,000 and daily pushes messages for returns credited to your account"}
{"index":{}}
{"productName":"Wealth Management Product C","annual_rate":"3.3500%","describe":"270-day wealth management product that requires the minimum investment of USD 40,000 and daily pushes messages for returns immediately credited to your account"}
{"index":{}}
{"productName":"Wealth Management Product D","annual_rate":"3.1200%","describe":"90-day wealth management product that requires the minimum investment of USD 12,000 and daily pushes messages for returns credited to your account"}
{"index":{}}
{"productName":"Wealth Management Product E","annual_rate":"3.0100%","describe":"30-day wealth management product that requires the minimum investment of USD 8,000 and daily pushes messages for returns"}
{"index":{}}
{"productName":"Wealth Management Product F","annual_rate":"2.7500%","describe":"3-day popular wealth management product that does not require service fees, requires the minimum investment of USD 500, and pushes messages for returns"}

The return result contains "errors": false, indicating that the data has been inserted successfully.

Step 5: Search for data

Scenario 1: Full-text search

This example uses the match query to perform keyword matching on the describe field, searching for all products in the product_info index whose descriptions contain daily pushes messages for returns credited to your account. The code is as follows.

GET /product_info/_search
{
  "query": {
    "match": {
      "describe": "daily pushes messages for returns credited to your account"
    }
  }
}

The core parameters are described as follows.

Parameter	Description
query	Define the query conditions.
match	The input content will be tokenized.
describe	The field to query, which is the `description content`. In the `product_info` index, the `describe` field is defined as `text` type, using the `analysis-ik` tokenizer for coarse-grained tokenization, meaning that queries on this field will use full-text matching and tokenization.

The return result is as follows.

Note

Alibaba Cloud ES supports searching data through tokenizers and scoring by relevance. In the return results, the higher-ranked search results have higher matching degrees and scores.
This query will return all documents whose description field contains content related to the keyword text. Due to the full-text search mechanism, ES can still match based on relevance even if some words in the description are in a different order or omitted.

{
  "took": 27,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 6,
      "relation": "eq"
    },
    "max_score": 1.7260926,
    "hits": [
      {
        "_index": "product_info",
        "_id": "MXS855cBfq0GQy5BxZ0m",
        "_score": 1.7260926,
        "_source": {
          "productName": "Wealth Management Product C",
          "annual_rate": "3.3500%",
          "describe": "270-day wealth management product that requires the minimum investment of USD 40,000 and daily pushes messages for returns immediately credited to your account"
        }
      },
      {
        "_index": "product_info",
        "_id": "MnS855cBfq0GQy5BxZ0m",
        "_score": 1.7260926,
        "_source": {
          "productName": "Wealth Management Product D",
          "annual_rate": "3.1200%",
          "describe": "90-day wealth management product that requires the minimum investment of USD 12,000 and daily pushes messages for returns credited to your account"
        }
      },
      {
        "_index": "product_info",
        "_id": "MHS855cBfq0GQy5BxZ0m",
        "_score": 1.7260926,
        "_source": {
          "productName": "Wealth Management Product B",
          "annual_rate": "3.1100%",
          "describe": "90-day wealth management product that requires the minimum investment of USD 10,000 and daily pushes messages for returns credited to your account"
        }
      },
      {
        "_index": "product_info",
        "_id": "M3S855cBfq0GQy5BxZ0m",
        "_score": 1.1507283,
        "_source": {
          "productName": "Wealth Management Product E",
          "annual_rate": "3.0100%",
          "describe": "30-day wealth management product that requires the minimum investment of USD 8,000 and daily pushes messages for returns"
        }
      },
      {
        "_index": "product_info",
        "_id": "L3S855cBfq0GQy5BxZ0m",
        "_score": 1.1325164,
        "_source": {
          "productName": "Wealth Management Product A",
          "annual_rate": "3.2200%",
          "describe": "180-day wealth management product that requires the minimum investment of USD 20,000, enables low-risk investment, and lets you select whether to receive push messages for returns"
        }
      },
      {
        "_index": "product_info",
        "_id": "NHS855cBfq0GQy5BxZ0m",
        "_score": 0.3420724,
        "_source": {
          "productName": "Wealth Management Product F",
          "annual_rate": "2.7500%",
          "describe": "3-day popular wealth management product that does not require service fees, requires the minimum investment of USD 500, and pushes messages for returns"
        }
      }
    ]
  }
}

Scenario 2: Search by condition

This example uses the range query to search for products in the product_info index with an annual rate between 3.0000% and 3.1300%. The code is as follows.

GET /product_info/_search
{
  "query": {
    "range": {
      "annual_rate": {
        "gte": "3.0000%",
        "lte": "3.1300%"
      }
    }
  }
}

The core parameters are described as follows.

Parameter	Description
query	Define the query conditions.
range	Indicates a range query.
annual_rate	The field to query, which is the `annual rate`. In the `product_info` index, the `annual_rate` field is defined as `keyword` type, meaning that queries on this field will use exact matching.
gte	Value range, greater than or equal to.
lte	Value range, less than or equal to.

The return result is as follows.

{
  "took": 27,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 3,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "product_info",
        "_id": "M3S855cBfq0GQy5BxZ0m",
        "_score": 1,
        "_source": {
          "productName": "Wealth Management Product E",
          "annual_rate": "3.0100%",
          "describe": "30-day wealth management product that requires the minimum investment of USD 8,000 and daily pushes messages for returns"
        }
      },
      {
        "_index": "product_info",
        "_id": "MnS855cBfq0GQy5BxZ0m",
        "_score": 1,
        "_source": {
          "productName": "Wealth Management Product D",
          "annual_rate": "3.1200%",
          "describe": "90-day wealth management product that requires the minimum investment of USD 12,000 and daily pushes messages for returns credited to your account"
        }
      },
      {
        "_index": "product_info",
        "_id": "MHS855cBfq0GQy5BxZ0m",
        "_score": 1,
        "_source": {
          "productName": "Wealth Management Product B",
          "annual_rate": "3.1100%",
          "describe": "90-day wealth management product that requires the minimum investment of USD 10,000 and daily pushes messages for returns credited to your account"
        }
      }
    ]
  }
}

(Optional) Next steps

Delete the index

Run the following code to delete the test index created in this topic.

DELETE /product_info

The return result is as follows.

Data migration and cluster scaling

Data migration: You can migrate your business data to Alibaba Cloud ES for subsequent query and analysis operations.
Cluster scaling: If the current cluster configuration cannot meet your business load and performance requirements, you can scale it as needed.

Release the instance

If the ES instance is no longer needed, please release it promptly to avoid wasting resources and incurring additional costs.

Warning

After an instance is released, the data cannot be recovered. Please proceed with caution. We recommend that you back up data before releasing the instance. After the instance is released, billing will stop.

References

For more information about accessing and configuring Alibaba Cloud ES, see Access and configure an Elasticsearch cluster.
For operations related to synchronizing RDS MySQL to Alibaba Cloud ES, see Select a synchronization method.