Elasticsearch:Quick start: Create a cluster and retrieve data

What you will learn

• Create and configure an ES cluster.

• Model data by using Dev Tools in Kibana.

  Kibana is built into the Alibaba Cloud Elasticsearch console and requires no manual installation.

• Insert sample data and perform various search operations.

After completing this tutorial, you will know how to perform fundamental Alibaba Cloud Elasticsearch operations, such as creating an index, reading and writing data, and running searches.

ES version and cost

Before you start, review the following key information for this tutorial:

• ES type and version: Vector Enhanced Edition 8.17.0.

  This version modularizes core algorithm services in the AI search pipeline, including document parsing, chunking, text embedding, query analysis, retrieval, sorting, and large language model (LLM) integration. It supports semantic search to help you quickly build Retrieval-Augmented Generation (RAG) and multimodal search applications.

• Estimated cost: This tutorial uses the pay-as-you-go billing method. If you follow the recommended specifications and duration, the total cost is expected to be less than USD 5. Release the cluster immediately after you finish to avoid unnecessary charges.

Prerequisites

• Register and log on to an Alibaba Cloud account. Ensure that the account has permissions to create and manage Alibaba Cloud Elasticsearch and Virtual Private Cloud (VPC) resources.

• Create a Virtual Private Cloud (VPC) and a vSwitch, and record their region and zone. You must create the ES cluster in the same region and zone to connect it to the VPC.

Procedure

Step 1: Create a cluster (about 20 minutes)

1. On the cluster creation page, create a cluster. Configure the key parameters as shown in the table below, and leave other parameters at their default values.

   Parameter	Description
   Billing Method	Select pay-as-you-go. This allows you to release the cluster at any time.
   Region and zone	Region: Select the region where your VPC and vSwitch are located. Zone: For this tutorial, select a single zone to reduce the cluster creation time. For a production environment, you can upgrade to a multi-zone deployment for higher availability. This tutorial uses China (Hangzhou) / Zone I.
   Network Type	Select the VPC and vSwitch that you created.
   Cluster Type and Elasticsearch Version	Vector Enhanced Edition 8.17.0. This tutorial is based on this version.
   Data Node Specifications	CPU Type: Intel 2 vCPU, 4 GiB memory. Data node storage type and size: SSD cloud disk, 20 GiB storage space per node. Number of data nodes: 2. Data nodes store index data and handle operations such as creating, retrieving, updating, deleting, and aggregating documents.
   	ES_test
   Password	Create a custom password. You will use this password to log in to Kibana to create indexes and explore data.

2. Click Buy Now and wait about 20 minutes for the cluster status to change to Active.

Step 2: Configure and log on to Kibana

Kibana public access is enabled by default, but for security, all IP addresses are denied access. To access the Kibana console, you must add your device's IP address to the public access whitelist.

Authentication is a two-step process. First, you must be logged in to your Alibaba Cloud account. Second, you must use your ES cluster credentials (the username is elastic and its corresponding password) to log in to Kibana.

1. In the left-side navigation pane, click Data Visualization. In the Kibana section, click Modify Configuration.

2. In the Network Access Configuration section, modify the Kibana public access whitelist.

   Device IP address

   Use the following table to find your device's IP address.

   Scenario	IP address	Method
   Access Kibana from your local computer over the internet.	The public IP address of your local computer. Note If your computer is on a home or corporate network, you must add the public egress IP address of the local area network (LAN).	Run the curl ipinfo.io/ip command to find the public IP address of your computer.
   Access Kibana from an ECS instance over the internet.	If the ECS instance and the ES cluster are in different VPCs, use the public IP address of the ECS instance to access Kibana. Add this IP address to the public access whitelist of the ES cluster.	Log on to the ECS console and view the public IP address of the instance in the instance list.

   Public whitelist

   Obtain the IP address of your device and add it to the public access whitelist.

   1. Click Configure to the right of the default group. In the dialog box that appears, add the IP address to the whitelist.

      Configuration type	Format and example	Notes
      IPv4 address format	Single IP address: 192.168.0.1 CIDR block: 192.168.0.0/24. We recommend that you consolidate individual IP addresses into CIDR blocks. You can configure a maximum of 300 IP addresses or CIDR blocks for a single cluster. Separate multiple entries with commas (,). Do not add spaces before or after the commas.	Default public address: 127.0.0.1, which denies access from all IPv4 addresses. 0.0.0.0/0: Allows access from all IPv4 addresses. Important Do not configure 0.0.0.0/0. This setting poses a high security risk. Some clusters and regions do not support 0.0.0.0/0. Refer to the user interface or error messages for accurate information.

   2. After you complete the configuration, click OK.

3. Click Access over Internet. On the Kibana login page, enter your username and password. After you log in, you are redirected to the Kibana console.

   • Username: elastic.

   • Password: The password that you set when you created the ES cluster. If you forget the password, you can reset it.

4. Go to the Developer Tools console and run the GET / command to query cluster information.

   Verification: The connection is successful if the right pane displays a JSON object containing cluster details, such as the version number and name.

Step 3: Create an index

In Kibana Dev Tools, run the following command to create an index named product_info and define a mapping for wealth management product data.

PUT /product_info
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
      "properties": {
        "productName": {
          "type": "text",
          "analyzer": "ik_smart"
        },
        "annual_rate":{
          "type":"keyword"
        },
        "describe": {
          "type": "text",
          "analyzer": "ik_smart"
        }
    }
  }
}

Key parameter descriptions:

settings: Defines the index shard configuration. For example, this configuration defines 5 primary shards and 1 replica for each primary shard.

mappings: Defines the index fields. For example, this configuration defines three fields: productName, annual_rate, and describe. Fields of the text type are tokenized to support full-text search. Fields of the keyword type are used for exact value matching. For more information about field types, see Field data types.

Verification: If the response contains "acknowledged": true and "shards_acknowledged": true, the index was created successfully.

Step 4: Insert data

Run the following command to bulk insert sample data:

POST /product_info/_bulk
{"index":{}}
{"productName":"Wealth Management Product A","annual_rate":"3.2200%","describe":"180-day fixed-term product. Minimum investment: 20,000. Stable returns. Optional push notifications."}
{"index":{}}
{"productName":"Wealth Management Product B","annual_rate":"3.1100%","describe":"90-day scheduled investment product. Minimum investment: 10,000. Daily push notifications for credited returns."}
{"index":{}}
{"productName":"Wealth Management Product C","annual_rate":"3.3500%","describe":"270-day scheduled investment product. Minimum investment: 40,000. Daily push notifications for immediately credited returns."}
{"index":{}}
{"productName":"Wealth Management Product D","annual_rate":"3.1200%","describe":"90-day scheduled investment product. Minimum investment: 12,000. Daily push notifications for credited returns."}
{"index":{}}
{"productName":"Wealth Management Product E","annual_rate":"3.0100%","describe":"Recommended 30-day scheduled investment product. Minimum investment: 8,000. Daily return message notifications."}
{"index":{}}
{"productName":"Wealth Management Product F","annual_rate":"2.7500%","describe":"Popular short-term product. 3-day term, no fees. Minimum investment: 500. Return notifications by SMS."}

Verification: If the response contains "errors": false, the data was inserted successfully.

Step 5: Search for data

GET /product_info/_search
{
  "query": {
    "match": {
      "describe": "Daily push notifications for credited returns"
    }
  }
}

GET /product_info/_search
{
  "query": {
    "range": {
      "annual_rate": {
        "gte": "3.0000%",
        "lte": "3.1300%"
      }
    }
  }
}

Clean up resources and next steps

Delete data and release the cluster

1. Run the following command to delete the test index.

   DELETE /product_info

   The following response is returned:

   {
   "acknowledged" : true
   }

2. Return to the Alibaba Cloud Elasticsearch console.

3. In the cluster list, find the cluster that you want to release. In the Actions column, click > Release Cluster. Select Immediately Delete and then click OK to release the cluster.

   Important

   Releasing a cluster permanently deletes it and all its data. This action cannot be undone. Proceed with caution.

Next steps

To learn more about Alibaba Cloud Elasticsearch, you can explore the following topics:

• Data ingestion: Collect and migrate your business data to an ES cluster.

  Before migration, determine whether you need to upgrade the cluster configuration based on your data volume.

• Performance tuning: Explore index design and query optimization techniques to improve cluster performance.

• Kibana visualization: Learn how to use Kibana features such as Discover and Dashboard to create data visualization dashboards.