Elasticsearch:Quick start: Create a cluster and retrieve data

What you will learn

• Create and configure an Alibaba Cloud Elasticsearch cluster.

• Model data by using Dev Tools in Kibana.

  Kibana is built into the Elasticsearch console and does not require manual installation.

• Ingest sample data and perform various search operations.

After completing this tutorial, you can create an index, read and write data, and run basic searches in Alibaba Cloud Elasticsearch.

ES version and cost

Before you begin, review the following key information for this tutorial:

• ES type and version: Vector Enhanced Edition 8.17.0.

  This version componentizes the core algorithm services in the AI search pipeline, including document parsing, chunking, text vectorization, query analysis, retrieval, and ranking, along with large language model (LLM) support. It enables semantic search, allowing you to quickly build applications with RAG (retrieval-augmented generation) and multimodal search capabilities.

• Estimated cost: This tutorial uses the pay-as-you-go billing method. If you follow the recommended specifications and complete the steps within the estimated time, the total cost will be less than USD 5. To avoid unexpected charges, release the cluster immediately after you complete the tutorial.

Prerequisites

• Register and log on to an Alibaba Cloud account. Ensure that the account has the permissions to create and manage resources such as Alibaba Cloud Elasticsearch and VPC.

• You have created a VPC and a vSwitch. Make a note of the region and zone where they are located. You must create your Alibaba Cloud Elasticsearch cluster in the same region and zone.

Procedure

Step 1: Create a cluster (about 20 minutes)

1. Go to the cluster creation page. Configure the key parameters as described in the following table. You can leave the other parameters at their default values.

   Parameter	Description
   Billing Method	Select pay-as-you-go. This allows you to release the cluster at any time.
   Region and Zone	Region: Select the region where you created the VPC and vSwitch. Zone: For this tutorial, select a single zone to reduce the cluster creation time. For a production environment, you can deploy the cluster across multiple zones. This tutorial uses the China (Hangzhou) region and Zone I.
   Network Type	Select the VPC and vSwitch that you created.
   Cluster Type and Elasticsearch version	Vector Enhanced Edition 8.17.0.
   Data node specifications	CPU type: Intel 2-core 4 GiB. Storage type and size per data node: SSD cloud disk, 20 GiB. Number of data nodes: 2. A data node stores index data and handles operations such as creating, reading, updating, deleting (CRUD), and aggregating documents.
   	ES_test
   Password	Create a custom password. You will need it to log on to Kibana.

2. Click Buy Now and wait about 20 minutes for the cluster status to change to Active before you proceed to the next step.

Step 2: Configure and log on to Kibana

Public access to Kibana is enabled by default. For security, access is denied from all IP addresses until you configure a whitelist. You must add your device's public IP address to the whitelist to access the Kibana console.

Authentication requires two steps. First, you must be logged on to your Alibaba Cloud account. Second, you must provide the credentials for your cluster: the username elastic and the password that you created.

1. In the left-side navigation pane, click Data Visualization, and then in the Kibana section, go to Modify Configuration.

2. In the Network Access Configuration section, modify the Kibana public access whitelist.

   Get your device IP

   You can obtain the public IP address of your device based on your access scenario.

   Scenario	IP address	Method
   Access Kibana from your local machine over the internet.	Public IP address of your local machine. Note If your local machine is on a home or corporate network, you need to add the public egress IP address of that network.	Run the curl ipinfo.io/ip command to find the public IP address of your machine.
   Access Kibana from an ECS instance over the internet.	If the ECS instance and the Elasticsearch cluster are in different VPCs, you can access Kibana by using the public IP address of the ECS instance. If so, add the public IP address of the ECS instance to the cluster's public access whitelist.	Log on to the ECS console and find the public IP address in the instance list.

   Add public whitelist

   Obtain the IP address of your device and add it to the public access whitelist.

   1. To the right of the default group, click Configure. In the dialog box that appears, add the IP address to the whitelist.

      Configuration type	Format and example	Important notes
      IPv4 address format	Single IP address: 192.168.0.1 CIDR block: 192.168.0.0/24. We recommend that you consolidate individual IP addresses into CIDR blocks. You can add up to 300 IP addresses or CIDR blocks to a whitelist for a single cluster. Separate multiple entries with a comma (,). Do not add spaces before or after the comma.	Default public whitelist: 127.0.0.1, which denies access from all IPv4 addresses. 0.0.0.0/0: Allows access from all IPv4 addresses. Important For security, we strongly recommend that you do not use 0.0.0.0/0. Some clusters and regions do not support 0.0.0.0/0. Availability is subject to the UI or error messages.

   2. After you complete the configuration, click OK.

3. Click Access over Internet. On the Kibana logon page, enter your username and password. After you log on, you are redirected to the Kibana console.

   • Username: The username is elastic.

   • Password: The password that you set when you created the ES cluster. If you have forgotten the password, you can reset it.

4. In the Dev Tools console, run GET / to query cluster information.

   Verification: If the response pane on the right displays a JSON object containing information such as the version number and cluster name, the connection is successful.

Step 3: Create an index

In Kibana Dev Tools, run the following RESTful API request to create an index mapping named product_info for wealth management product data.

PUT /product_info
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
      "properties": {
        "productName": {
          "type": "text",
          "analyzer": "ik_smart"
        },
        "annual_rate":{
          "type":"keyword"
        },
        "describe": {
          "type": "text",
          "analyzer": "ik_smart"
        }
    }
  }
}

Key parameter descriptions:

settings: Defines the index shard configuration, such as 5 primary shards and 1 replica for each primary shard.

mappings: Defines the fields of an index. For example, an index can have three fields: productName, annual_rate, and describe. Fields of the text type support tokenization and fuzzy search. Fields of the keyword type are used for exact value matching. For more information about field types, see Field data types.

A response that contains "acknowledged": true and "shards_acknowledged": true indicates that the index was created successfully.

Step 4: Insert data

Run the following command to insert test data in bulk:

POST /product_info/_bulk
{"index":{}}
{"productName":"Wealth Management Product A","annual_rate":"3.2200%","describe":"180-day fixed-term product, minimum investment 20000, stable returns, you can choose push notifications."}
{"index":{}}
{"productName":"Wealth Management Product B","annual_rate":"3.1100%","describe":"90-day scheduled investment product, minimum investment 10000, daily credited returns push notifications."}
{"index":{}}
{"productName":"Wealth Management Product C","annual_rate":"3.3500%","describe":"270-day scheduled investment product, minimum investment 40000, daily immediately credited returns push notifications."}
{"index":{}}
{"productName":"Wealth Management Product D","annual_rate":"3.1200%","describe":"90-day scheduled investment product, minimum investment 12000, daily credited returns push notifications."}
{"index":{}}
{"productName":"Wealth Management Product E","annual_rate":"3.0100%","describe":"Recommended 30-day scheduled investment product, minimum investment 8000, daily returns push notifications."}
{"index":{}}
{"productName":"Wealth Management Product F","annual_rate":"2.7500%","describe":"Popular short-term product, 3-day term, no service fees, minimum investment 500, return notifications by SMS."}

Result verification: A response that contains "errors": false indicates that the data was inserted successfully.

Step 5: Retrieve data

GET /product_info/_search
{
  "query": {
    "match": {
      "describe": "Daily credited returns push notifications"
    }
  }
}

GET /product_info/_search
{
  "query": {
    "range": {
      "annual_rate": {
        "gte": "3.0000%",
        "lte": "3.1300%"
      }
    }
  }
}

Clean up resources and next steps

Delete data and release the cluster

1. Run the following command to delete the test index that you created in this tutorial:

   DELETE /product_info

   The following result is returned:

   {
   ”acknowledged“ : true
   }

2. Return to the cluster list in the Elasticsearch console.

3. In the cluster list, find the cluster that you want to release. In the Actions column, click > Release Cluster, select Immediately Delete, and then click OK.

   Important

   Releasing a cluster permanently deletes the cluster and all its data. This action cannot be undone. Proceed with caution.

Next steps

To learn more about Elasticsearch, explore the following topics:

• Data ingestion and migration: Ingest or migrate your business data to Elasticsearch.

  Before you migrate, you need to first determine whether to upgrade the cluster configuration based on the data volume.

• Performance tuning: Explore techniques for index design and query optimization to improve cluster performance.

• Kibana visualization: Learn to use Kibana features such as Discover and Dashboard to create data visualization dashboards.