This topic describes how to create an Alibaba Cloud Elasticsearch cluster and access the cluster. It also describes how to call Elasticsearch RESTful APIs to create an index, create a document and insert data into the document, search for data, and delete the index.

Prerequisites

Scenarios

A finance service enterprise uses an online platform to manage its wealth management products and uses conventional databases to provide the search feature for customers. The wealth management products provided by the enterprise offer satisfactory returns, and the customer base of the enterprise grows rapidly. The expansion of its business system and the increase of customer information cause the inherent issues of conventional databases to become noticeable. These issues include slow search responses, low accuracy, and the low performance of data service devices. To resolve these issues and improve customer satisfaction, the enterprise purchases the Alibaba Cloud Elasticsearch service. This topic uses the preceding scenario to describe how to use Alibaba Cloud Elasticsearch to create a cluster and search for data.

For example, the enterprise provides the following wealth management products: 
{
  "products":[
    {"productName":"Wealth Management Product A","annual_rate":"3.2200%","describe":"180-day wealth management product that requires the minimum investment of USD 20,000, enables low-risk investment, and allows you to select whether to receive push messages for returns."}
    {"productName":"Wealth Management Product B","annual_rate":"3.1100%","describe":"90-day wealth management product that requires the minimum investment of USD 10,000 and daily pushes messages for returns credited to your account."}
    {"productName":"Wealth Management Product C","annual_rate":"3.3500%","describe":"270-day wealth management product that requires the minimum investment of USD 40,000 and daily pushes messages for returns immediately credited to your account."}
    {"productName":"Wealth Management Product D","annual_rate":"3.1200%","describe":"90-day wealth management product that requires the minimum investment of USD 12,000 and daily pushes messages for returns credited to your account."}
    {"productName":"Wealth Management Product E","annual_rate":"3.0100%","describe":"30-day wealth management product that requires the minimum investment of USD 8,000 and daily pushes messages for returns."}
    {"productName":"Wealth Management Product F","annual_rate":"2.7500%","describe":"3-day popular wealth management product that does not require service fees, requires the minimum investment of USD 500, and pushes messages for returns."}
  ]
}

Procedure

  1. Step 1: Create a cluster
    Create an Alibaba Cloud Elasticsearch V6.7 cluster of the Standard Edition.
  2. Step 2: Access the cluster
    Log on to the Kibana console of the cluster to access the cluster after the state of the cluster becomes Active.
  3. Step 3: Create an index
    Call a RESTful API to create an index.
  4. Step 4: Create documents and insert data into the documents
    Call a RESTful API to create documents and insert data into the documents.
  5. Step 5: Search for data
    Call a RESTful API to perform a full-text search or search for data by condition.
  6. Step 6: (Optional) Delete the index
    Call a RESTful API to delete the index to save resources if you no longer require the index.
  7. Step 7: (Optional) Release the cluster
    Release the cluster if you no longer require the cluster. After a cluster is released, data stored in the cluster cannot be recovered. We recommend that you back up data before you release a cluster.

Video tutorial

Step 1: Create a cluster

  1. Go to the Elasticsearch buy page.
  2. On the buy page, configure cluster launch settings.

    You can configure the parameters on the buy page based on your business requirements. In this example, a V6.7 cluster is used. For more information about the parameters on the buy page, see Parameters on the buy page.

  3. Click Buy Now.
  4. Preview the cluster configurations, read the terms of service, select the check box, and then click Pay.
  5. On the Pay page, click Console. On the page that appears, move your pointer over the More icon in the upper-left corner and select Elasticsearch. The Overview page of the Elasticsearch console appears.
  6. In the top navigation bar, select the region where the created Elasticsearch cluster resides.
  7. In the left-side navigation pane, click Elasticsearch Clusters. On the Elasticsearch Clusters page, view the created Elasticsearch cluster.

Step 2: Access the cluster

After the state of the cluster changes to Active, you can perform the following steps to use Kibana to access the cluster:
Note You can also use a cURL command or a client to access the cluster. For more information, see Access an Elasticsearch cluster.
  1. On the Elasticsearch Clusters page, find the newly created cluster and click the cluster ID.
  2. In the left-side navigation pane of the page that appears, choose Configuration and Management > Data Visualization.
  3. Optional:In the Kibana section of the page that appears, click Edit Configuration. In the Network Access Configuration section of the Kibana Configuration page, add the IP address of your client to a new or existing public IP address whitelist for Kibana.
    If this is the first time you log on to the Kibana console of your cluster over the Internet and you have not modified the configuration for Kibana access over the Internet, you need to modify the configuration as prompted. For more information, see Configure a public or private IP address whitelist for Kibana.
  4. Return to the Data Visualization page. In the Kibana section, click Access over Internet.
  5. On the Kibana logon page, enter the username and password and click Log in.
    The username is elastic. The password is the one that is specified when you create the cluster. If you forget the password, you can reset it. For more information about the procedure and precautions for resetting the password, see Reset the access password for an Elasticsearch cluster.
  6. On the page that appears, click Explore on my own.
    Logon success page
  7. In the left-side navigation pane, click Dev Tools. On the page that appears, click Go to work.
    Note In this example, the Kibana console of an Elasticsearch V6.7 cluster is used. The operations for other versions of clusters may vary. The actual operations in the Kibana console prevail.
  8. On the Console tab, run the following command to access the cluster: 
    GET /
    Console tab
    If the access is successful, the following result is returned: 
    {
  "name" : "tgeAvZe",
  "cluster_name" : "es-cn-nif1z64qj003g****",
  "cluster_uuid" : "IZmmd9IGTmKzqiZiyz****",
  "version" : {
    "number" : "6.7.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "c9c0c3a",
    "build_date" : "2020-12-01T08:00:27.556078Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Step 3: Create an index

Run one of the following commands to create an index named product_info:
  • Command for an Elasticsearch cluster of a version earlier than V7.0
    PUT /product_info
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
    "products": {
      "properties": {
        "productName": {
          "type": "text",
          "analyzer": "ik_smart"
        },
        "annual_rate":{
          "type":"keyword"
        },
        "describe": {
         "type": "text",
         "analyzer": "ik_smart"
        }
      }
    }
  }
}
  • Command for an Elasticsearch cluster of V7.0 or later
    PUT /product_info
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
      "properties": {
        "productName": {
          "type": "text",
          "analyzer": "ik_smart"
        },
        "annual_rate":{
          "type":"keyword"
        },
        "describe": {
          "type": "text",
          "analyzer": "ik_smart"
        }
    }
  }
}
    Notice Mapping types are deprecated in open source Elasticsearch 7.0 and later. However, these mapping types are still supported in versions earlier than Elasticsearch 7.0. If mapping types are used in Alibaba Cloud Elasticsearch V7.0 and later, the system displays the error message "type": "mapper_parsing_exception". For more information about the error details and solution, see Removal of mapping types.

In the preceding example, an index named product_info is created. In a version earlier than Elasticsearch V7.0, the index is of the products type. In Elasticsearch V7.0 or later, the index is of the _doc type. The index contains the productName, annual_rate, and describe fields.

If the command is successfully run, the following result is returned: 
{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "product_info"
}

Step 4: Create documents and insert data into the documents

Call the _bulk API to insert multiple data records at a time.
Notice The following command can be used only for Elasticsearch clusters earlier than V7.0. Before you run the command for an Elasticsearch cluster of V7.0 or later, you must change products in the command to _doc. 
POST /product_info/products/_bulk
{"index":{}}
{"productName":"Wealth Management Product A","annual_rate":"3.2200%","describe":"180-day wealth management product that requires the minimum investment of USD 20,000, enables low-risk investment, and allows you to select whether to receive push messages for returns."}
{"index":{}}
{"productName":"Wealth Management Product B","annual_rate":"3.1100%","describe":"90-day wealth management product that requires the minimum investment of USD 10,000 and daily pushes messages for returns credited to your account."}
{"index":{}}
{"productName":"Wealth Management Product C","annual_rate":"3.3500%","describe":"270-day wealth management product that requires the minimum investment of USD 40,000 and daily pushes messages for returns immediately credited to your account."}
{"index":{}}
{"productName":"Wealth Management Product D","annual_rate":"3.1200%","describe":"90-day wealth management product that requires the minimum investment of USD 12,000 and daily pushes messages for returns credited to your account."}
{"index":{}}
{"productName":"Wealth Management Product E","annual_rate":"3.0100%","describe":"30-day wealth management product that requires the minimum investment of USD 8,000 and daily pushes messages for returns."}
{"index":{}}
{"productName":"Wealth Management Product F","annual_rate":"2.7500%","describe":"3-day popular wealth management product that does not require service fees, requires the minimum investment of USD 500, and pushes messages for returns."}

If "errors" : false is returned, data is inserted into the created documents.

Step 5: Search for data

Notice The following command can be used only for Elasticsearch clusters earlier than V7.0. Before you run the command for an Elasticsearch cluster of V7.0 or later, you must change products in the command to _doc.
  • Full-text search

    Run the following command to search for products whose descriptions contain daily pushes messages for returns credited to your account: 

    GET /product_info/products/_search
{
  "query": {
    "match": {
      "describe": "daily pushes messages for returns credited to your account"
    }
  }
}
    If the command is successfully run, the following result is returned: 
    {
  "took" : 21,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 6,
    "max_score" : 1.3968885,
    "hits" : [
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "WLvWYXAB8Rql5AUxLqUU",
        "_score" : 1.3968885,
        "_source" : {
          "productName" : "Wealth Management Product B",
          "annual_rate" : "3.1100%",
          "describe" : "90-day wealth management product that requires the minimum investment of USD 10,000 and daily pushes messages for returns credited to your account."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "WrvWYXAB8Rql5AUxLqUU",
        "_score" : 1.3968885,
        "_source" : {
          "productName" : "Wealth Management Product D",
          "annual_rate" : "3.1200%",
          "describe" : "90-day wealth management product that requires the minimum investment of USD 12,000 and daily pushes messages for returns credited to your account."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "WbvWYXAB8Rql5AUxLqUU",
        "_score" : 1.3547361,
        "_source" : {
          "productName" : "Wealth Management Product C",
          "annual_rate" : "3.3500%",
          "describe" : "270-day wealth management product that requires the minimum investment of USD 40,000 and daily pushes messages for returns immediately credited to your account."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "W7vWYXAB8Rql5AUxLqUU",
        "_score" : 1.1507283,
        "_source" : {
          "productName" : "Wealth Management Product E",
          "annual_rate" : "3.0100%",
          "describe" : "30-day wealth management product that requires the minimum investment of USD 8,000 and daily pushes messages for returns."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "XLvWYXAB8Rql5AUxLqUU",
        "_score" : 0.5753642,
        "_source" : {
          "productName" : "Wealth Management Product F",
          "annual_rate" : "2.7500%",
          "describe" : "3-day popular wealth management product that does not require service fees, requires the minimum investment of USD 500, and pushes messages for returns."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "V7vWYXAB8Rql5AUxLqUU",
        "_score" : 0.31854028,
        "_source" : {
          "productName" : "Wealth Management Product A",
          "annual_rate" : "3.2200%",
          "describe" : "180-day wealth management product that requires the minimum investment of USD 20,000, enables low-risk investment, and allows you to select whether to receive push messages for returns."
        }
      }
    ]
  }
}
    Note Alibaba Cloud Elasticsearch allows you to use a tokenizer to search for data. It also allows you to sort the records in search results by score. In the preceding result, the descriptions of the first two products contain daily pushes messages for returns credited to your account, and the descriptions of the last two products contain only pushes messages. The higher the ranking of a product in the search result, the higher the matching degree and score of the product.
  • Search by condition
    Run the following command to search for products with an annualized rate of 3.0000% to 3.1300%: 
    GET /product_info/products/_search
{
  "query": {
    "range": {
      "annual_rate": {
        "gte": "3.0000%",
        "lte": "3.1300%"
      }
    }
  }
}
    If the command is successfully run, the following result is returned: 
    {
  "took" : 10,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "WLvWYXAB8Rql5AUxLqUU",
        "_score" : 1.0,
        "_source" : {
          "productName" : "Wealth Management Product B",
          "annual_rate" : "3.1100%",
          "describe" : "90-day wealth management product that requires the minimum investment of USD 10,000 and daily pushes messages for returns credited to your account."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "WrvWYXAB8Rql5AUxLqUU",
        "_score" : 1.0,
        "_source" : {
          "productName" : "Wealth Management Product D",
          "annual_rate" : "3.1200%",
          "describe" : "90-day wealth management product that requires the minimum investment of USD 12,000 and daily pushes messages for returns credited to your account."
        }
      }
    ]
  }
}
    Note The system finds products that meet your requirements based on the search conditions and displays the products in descending order.

    For more information about search methods, see Query DSL.

Step 6: (Optional) Delete the index

After you are familiar with Alibaba Cloud Elasticsearch, you can delete the index by running the following command to avoid wasting resources:
Warning A deleted index cannot be recovered. Proceed with caution. 
DELETE /product_info
If the index is deleted, the following result is returned: 
{
  "acknowledged" : true
}

Step 7: (Optional) Release the cluster

If you no longer require the cluster, you can release it. After the cluster is released, you are no longer charged for the cluster. In addition, the data stored in the cluster is deleted and cannot be recovered. You can release only pay-as-you-go clusters.
Warning After a cluster is released, data stored in the cluster cannot be recovered. We recommend that you back up data before you release a cluster. For more information, see Data backup overview.
  1. On the Elasticsearch Clusters page, find the cluster that you want to release, move your pointer over the More icon in the Actions column, and then select Release.
    Release a cluster
  2. In the message that appears, click OK.

References