Getting started

Scenarios

A finance service enterprise uses an online platform to manage its wealth management products and uses conventional databases to provide the search feature for customers. The wealth management products provided by the enterprise offer satisfactory returns, and the customer base of the enterprise grows rapidly. The expansion of its business system and the increase of customer information cause the inherent issues of conventional databases to become noticeable. These issues include slow search responses, low accuracy, and the low performance of data service devices. To resolve these issues and improve customer satisfaction, the enterprise purchases the Alibaba Cloud Elasticsearch service. This topic uses the preceding scenario to describe how to use Alibaba Cloud Elasticsearch to create a cluster and search for data.

For example, the enterprise provides the following wealth management products:

{
  "products":[
    {"productName":"Daily Wealth Management for Comprehensive Health","annual_rate":"3.2200%","describe":"180-day wealth management product. Minimum investment of USD 20,000. Low-risk investment. Select whether to receive push messages for returns."}
    {"productName":"Western Tongbao","annual_rate":"3.1100%","describe":"90-day wealth management product. Minimum investment of USD 10,000. Daily push messages when returns are credited to your account."}
    {"productName":"Anxiang Livestock Industry","annual_rate":"3.3500%","describe":"270-day wealth management product. Minimum investment of USD 40,000. Daily push messages when returns are immediately credited to your account."}
    {"productName":"Monthly 5G Device Purchase Profit","annual_rate":"3.1200%","describe":"90-day wealth management product. Minimum investment of USD 12,000. Daily push messages when returns are credited to your account."}
    {"productName":"New Energy Power Wealth Management","annual rate":"3.0100%","describe":"30-day wealth management product. Minimum investment of USD 8,000. Daily push messages for returns."}
    {"productName":"Microcredit Profit","annual_rate":"2.7500%","describe":"3-day popular wealth management product. No service fees. Minimum investment of USD 500. Push messages for returns."}
  ]
}

Procedure

Step 1: Create a cluster
Create an Alibaba Cloud Elasticsearch V6.7.0 cluster of the Standard Edition.
Step 2: Access the cluster
Log on to the Kibana console of the cluster to access the cluster after the state of the cluster becomes Active.
Step 3: Create an index
Call a RESTful API to create an index.
Step 4: Create documents and insert data into the documents
Call a RESTful API to create documents and insert data into the documents.
Step 5: Search for data
Call a RESTful API to perform a full-text search or search for data by condition.
Step 6: (Optional) Delete the index
Run a command to delete the index to save resources if you no longer require the index.
Step 7: (Optional) Release the cluster
Release the cluster if you no longer require the cluster. After a cluster is released, data stored in the cluster cannot be recovered. We recommend that you back up data before you release a cluster.

Step 1: Create a cluster

Go to the Elasticsearch buy page.
On the buy page, configure cluster launch settings.

You can configure the parameters on the buy page based on your business requirements. In this example, a V6.7.0 cluster is used. For more information about the parameters on the buy page, see Parameters on the buy page.
Click Buy Now.
After the cluster is created, click Console to go to the Overview page of the Elasticsearch console.
In the left-side navigation pane, click Elasticsearch Clusters.
In the top navigation bar, select a resource group and a region. The Overview page appears. Then, click Elasticsearch Clusters in the left-side navigation pane. On the Elasticsearch Clusters page, view the newly created cluster.

Step 2: Access the cluster

After the state of the cluster changes to Active, you can perform the following steps to use Kibana to access the cluster:

Note You can also use a curl command or a client to access the cluster. For more information, see Access an Elasticsearch cluster.

On the Elasticsearch Clusters page, find the newly created cluster and click the cluster ID.
In the left-side navigation pane of the page that appears, choose Configuration and Management > Data Visualization.
Optional:Configure a public IP address whitelist for Kibana.
127.0.0.1 and ::1 are specified in the default public IP address whitelist of Kibana. This indicates that all public IPv4 and IPv6 addresses cannot be used to access the Kibana console. The first time you log on to the Kibana console, you must modify the whitelist.
1. In the Kibana section of the page that appears, click Edit Configuration.
2. In the Network Access Configuration section of the page that appears, click Update on the right side of Kibana Whitelist.
3. Add the IP address of your client to the whitelist. For more information, see Configure a public or private IP address whitelist for Kibana.
  Notice
  
  If your client is in a home network or in a LAN of an office, add the IP address of the Internet egress to the whitelist rather than the private IP address of the client. We recommend that you query the IP address of the Internet egress in the IP address geolocation library of Taobao.
  
  You can also specify 0.0.0.0/0 as the whitelist. If you make this configuration, all public IPv4 addresses can be used to access the Kibana console. This poses security risks. You must evaluate the risks before you make this configuration.
Return to the Data Visualization page. In the Kibana section, click Access over Internet.
On the Kibana logon page, enter the username and password and click Log in.
The username is elastic. The password is the one that is specified when you create the cluster. If you forget the password, you can reset it. For more information about the procedure and precautions for resetting the password, see Reset the access password for an Elasticsearch cluster.
On the page that appears, click Explore on my own.
In the left-side navigation pane, click Dev Tools. On the page that appears, click Go to work.

On the Console tab, run the following command to access the cluster:

GET /

If the access is successful, the following result is returned:

{
  "name" : "tgeAvZe",
  "cluster_name" : "es-cn-nif1z64qj003g****",
  "cluster_uuid" : "IZmmd9IGTmKzqiZiyz****",
  "version" : {
    "number" : "6.7.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "c9c0c3a",
    "build_date" : "2020-12-01T08:00:27.556078Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Step 3: Create an index

Run one of the following commands to create an index named product_info:

Command for an Elasticsearch cluster of a version earlier than V7.0

PUT /product_info
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
    "products": {
      "properties": {
        "productName": {
       "type": "text",
       "analyzer": "ik_smart"
      },
        "annual_rate":{
       "type":"keyword"
      },
        "describe": {
      "type": "text",
      "analyzer": "ik_smart"
    }
      }
    }
  }
}

Command for an Elasticsearch cluster of V7.0 or later
```
PUT /product_info
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
      "properties": {
        "productName": {
          "type": "text",
          "analyzer": "ik_smart"
        },
        "annual_rate":{
          "type":"keyword"
        },
        "describe": {
          "type": "text",
          "analyzer": "ik_smart"
        }
    }
  }
}
```
Notice Mapping types are deprecated in open source Elasticsearch 7.0 and later. However, these mapping types are still supported in versions earlier than Elasticsearch 7.0. If mapping types are used in Alibaba Cloud Elasticsearch V7.0 and later, the system displays the error message "type": "mapper_parsing_exception". For more information about the error details and solution, see Removal of mapping types.

In the preceding example, an index named product_info is created. In a version earlier than Elasticsearch V7.0, the index is of the products type. In Elasticsearch V7.0 or later, the index is of the _doc type. The index contains the productName, annual_rate, and describe fields.

If the command is successfully run, the following result is returned:

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "product_info"
}

Step 4: Create documents and insert data into the documents

Use the _bulk API to insert multiple data records at a time.

Notice The following command can be used only for an Elasticsearch cluster of a version earlier than V7.0. For an Elasticsearch cluster of V7.0 or later, you must change products in the command to _doc.

POST /product_info/products/_bulk
null
{"productName":"Daily Wealth Management for Comprehensive Health","annual_rate":"3.2200%","describe":"180-day wealth management product. Minimum investment of USD 20,000. Low-risk investment. Select whether to receive push messages for returns."}
null
{"productName":"Western Tongbao","annual_rate":"3.1100%","describe":"90-day wealth management product. Minimum investment of USD 10,000. Daily push messages when returns are credited to your account."}
null
{"productName":"Anxiang Livestock Industry","annual_rate":"3.3500%","describe":"270-day wealth management product. Minimum investment of USD 40,000. Daily push messages when returns are immediately credited to your account."}
null
{"productName":"Monthly 5G Device Purchase Profit","annual_rate":"3.1200%","describe":"90-day wealth management product. Minimum investment of USD 12,000. Daily push messages when returns are credited to your account."}
null
{"productName":"New Energy Power Wealth Management","annual rate":"3.0100%","describe":"30-day wealth management product. Minimum investment of USD 8,000. Daily push messages for returns."}
null
{"productName":"Microcredit Profit","annual_rate":"2.7500%","describe":"3-day popular wealth management product. No service fees. Minimum investment of USD 500. Push messages for returns."}

If "errors" : false is returned, data is inserted into the created documents.

Step 5: Search for data

Notice The following command can be used only for an Elasticsearch cluster of a version earlier than V7.0. For an Elasticsearch cluster of V7.0 or later, you must change products in the command to _doc.

Full-text search

Run the following command to search for products whose descriptions contain Daily push messages when returns are credited to your account:

GET /product_info/products/_search
{
  "query": {
    "match": {
      "describe": "Daily push messages when returns are credited to your account"
    }
  }
}

If the command is successfully run, the following result is returned:

{
  "took" : 21,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 6,
    "max_score" : 1.3968885,
    "hits" : [
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "WLvWYXAB8Rql5AUxLqUU",
        "_score" : 1.3968885,
        "_source" : {
          "productName" : "Western Tongbao",
          "annual_rate" : "3.1100%",
          "describe" : "90-day wealth management product. Minimum investment of USD 10,000. Daily push messages when returns are credited to your account."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "WrvWYXAB8Rql5AUxLqUU",
        "_score" : 1.3968885,
        "_source" : {
          "productName" : "Monthly 5G Device Purchase Profit",
          "annual_rate" : "3.1200%",
          "describe" : "90-day wealth management product. Minimum investment of USD 12,000. Daily push messages when returns are credited to your account."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "WbvWYXAB8Rql5AUxLqUU",
        "_score" : 1.3547361,
        "_source" : {
          "productName" : "Anxiang Livestock Industry",
          "annual_rate" : "3.3500%",
          "describe" : "270-day wealth management product. Minimum investment of USD 40,000. Daily push messages when returns are immediately credited to your account."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "W7vWYXAB8Rql5AUxLqUU",
        "_score" : 1.1507283,
        "_source" : {
          "productName" : "New Energy Power Wealth Management",
          "annual rate" : "3.0100%",
          "describe" : "30-day wealth management product. Minimum investment of USD 8,000. Daily push messages for returns."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "XLvWYXAB8Rql5AUxLqUU",
        "_score" : 0.5753642,
        "_source" : {
          "productName" : "Microcredit Profit",
          "annual_rate" : "2.7500%",
          "describe" : "3-day popular wealth management product. No service fees. Minimum investment of USD 500. Push messages for returns."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "V7vWYXAB8Rql5AUxLqUU",
        "_score" : 0.31854028,
        "_source" : {
          "productName" : "Daily Wealth Management for Comprehensive Health",
          "annual_rate" : "3.2200%",
          "describe" : "180-day wealth management product. Minimum investment of USD 20,000. Low-risk investment. Select whether to receive push messages for returns."
        }
      }
    ]
  }
}

Note Alibaba Cloud Elasticsearch allows you to use a tokenizer to search for data. It also allows you to sort the records in search results by score. In the preceding result, the descriptions of the first two products contain Daily push messages when returns are credited to your account, while the descriptions of the last two products contain only push messages. The higher the ranking of a product in the search result, the higher the matching degree and score of the product.

Search by condition

Run the following command to search for products with an annualized rate of 3.0000% to 3.1300%:

GET /product_info/products/_search
{
  "query": {
    "range": {
      "annual_rate": {
        "gte": "3.0000%",
        "lte": "3.1300%"
      }
    }
  }
}

If the command is successfully run, the following result is returned:

{
  "took" : 10,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "WLvWYXAB8Rql5AUxLqUU",
        "_score" : 1.0,
        "_source" : {
          "productName" : "Monthly 5G Device Purchase Profit",
          "annual_rate" : "3.1200%",
          "describe" : "90-day wealth management product. Minimum investment of USD 12,000. Daily push messages when returns are credited to your account."
        }
      },
      {
        "_index" : "product_info",
        "_type" : "products",
        "_id" : "WrvWYXAB8Rql5AUxLqUU",
        "_score" : 1.0,
        "_source" : {
          "productName" : "Western Tongbao",
          "annual_rate" : "3.1100%",
          "describe" : "90-day wealth management product. Minimum investment of USD 10,000. Daily push messages when returns are credited to your account."
        }
      }
    ]
  }
}

Note The system finds products that meet your requirements based on the search conditions and displays the products in descending order.

For more information about search methods, see Query DSL.

Step 6: (Optional) Delete the index

After you are familiar with Alibaba Cloud Elasticsearch, you can delete the index by running the following command to avoid wasting resources:

Warning Proceed with caution because a deleted index cannot be recovered.

DELETE /product_info

If the index is deleted, the following result is returned:

{
  "acknowledged" : true
}

Step 7: (Optional) Release the cluster

If you no longer require the cluster, you can release it. After the cluster is released, you are no longer charged for the cluster. In addition, the data stored in the cluster is deleted and cannot be recovered. You can release only pay-as-you-go clusters.

Warning After a cluster is released, data stored in the cluster cannot be recovered. We recommend that you back up data before you release a cluster. For more information, see Data backup overview.

On the Elasticsearch Clusters page, find the cluster that you want to release and choose > Release in the Actions column.
In the message that appears, view the information and click OK.
For more information about how to release a cluster, see Release a cluster.

References

Access and configure an Alibaba Cloud Elasticsearch cluster: Access and configure an Elasticsearch cluster
Synchronize data from an ApsaraDB RDS for MySQL database to an Alibaba Cloud Elasticsearch cluster: Select a synchronization method
Learn more about index-related commands: Put Mapping

Prerequisites

Scenarios

Procedure

Step 1: Create a cluster

Step 2: Access the cluster

Step 3: Create an index

Step 4: Create documents and insert data into the documents

Step 5: Search for data

Step 6: (Optional) Delete the index

Step 7: (Optional) Release the cluster

References

FAQ