:Quick Start: Create a cluster and retrieve data

What you will learn

• Create and configure an ES cluster.

• Model data using Dev Tools in Kibana.

  Kibana is a core component of ES that provides an intuitive graphical interface to explore, analyze, and visualize data stored in ES. For your convenience, Kibana is built into the ES console and does not require manual installation.

• Insert sample data and perform various retrieval operations.

After you complete this tutorial, you will understand the basic capabilities of ES, including how to create indexes, read and write data, and perform searches.

ES version and cost

Before you start, review the following key information for this tutorial:

• ES type and version: Vector Enhanced Edition 8.17.0.

  This version modularizes the core algorithm services in the AI search pipeline. These services include document parsing, slicing, text embedding, query analysis, retrieval, sorting, and large language models (LLMs). The version supports semantic search and helps you quickly build retrieval-augmented generation (RAG) and multimodal search capabilities.

• Estimated cost: This tutorial uses the pay-as-you-go billing method. If you follow the recommended specifications and duration, the total cost is expected to be less than 20 CNY. After you complete the tutorial, release the cluster immediately to avoid unnecessary charges.

Prerequisites

• Register and log on to an Alibaba Cloud account. Ensure that the account has the permissions to create and manage resources such as Elasticsearch and VPC.

• Create a virtual private cloud (VPC) and a vSwitch. Record the region and zone where the VPC and vSwitch are located. You must select the same region and zone when you create the ES cluster. Otherwise, the cluster cannot be associated with the VPC and vSwitch.

Procedure

Step 1: Create a cluster (about 20 minutes)

1. Go to the cluster creation page to create a cluster. Configure the key parameters as described in the following table. You can keep the default values for other parameters.

   Parameter	Description
   Billing Method	Pay-as-you-go. You can release the cluster at any time.
   Region and Zone	Region: Select the region where your VPC and vSwitch are located. Zone: For this tutorial, select a single zone to reduce the cluster creation time. For a production environment, you can upgrade to a multi-zone deployment. This tutorial uses China (Hangzhou) / Zone I.
   Network Type	Select the VPC and vSwitch that you created.
   Cluster Type and Elasticsearch Version	Vector Enhanced Edition 8.17.0. This tutorial is based on this version.
   Data Node Specifications	CPU Type: Intel 2-core 4 GiB. Data Node Storage Type and Size: Standard SSD, 20 GiB storage space per node. Number of Data Nodes: 2. Data nodes store index data and are mainly used for operations such as creating, retrieving, updating, deleting, and aggregating documents.
   	ES_test
   Password	Create a custom password. You will use it later to log on to Kibana to build indexes and explore data.

2. Click Buy Now and wait about 20 minutes for the cluster status to change to Normal before proceeding to the next step.

Step 2: Configure and log on to Kibana

The public endpoint for Kibana is enabled by default. For security reasons, public access is denied from all IP addresses by default. You must add the IP address of your device to the whitelist to access Kibana.

Access authentication uses two-factor verification. First, log on to your Alibaba Cloud account. Then, use the access credentials of the ES cluster for a second verification. The username is fixed as elastic and you must provide the corresponding password.

1. In the navigation pane on the left, click Data Visualization. In the Kibana section, click Modify Configuration.

2. In the Network Access Configuration section, you can modify the Kibana public access whitelist.

   Obtain the IP address of your device

   You can obtain the IP address of your device based on the following scenarios.

   Scenario	IP address to obtain	Method
   Access Kibana from an on-premises device over the Internet.	The public IP address of your on-premises device. Note If your on-premises device is in a home network or a corporate local area network (LAN), you must add the public egress IP address of the LAN.	Run the curl ipinfo.io/ip command to query the public IP address of your on-premises device.
   Access Kibana from an ECS instance over the Internet.	If the ECS instance and the ES cluster are in different VPCs, you can access Kibana using the public IP address of the ECS instance. To do this, you must obtain the public IP address of the ECS instance and add it to the public access whitelist of the ES cluster.	Log on to the ECS console and view the public IP address of the instance in the instance list.

   Add the IP address to the public access whitelist

   Obtain the IP address of your device and add it to the public access whitelist.

   1. To the right of the Configure group, click default. In the dialog box that appears, add the IP address to the whitelist.

      Configuration type	Format and example	Important notes
      IPv4 address format	Single IP address: 192.168.0.1 CIDR block: 192.168.0.0/24. We recommend that you merge scattered IP addresses into CIDR blocks. A maximum of 300 IP addresses or CIDR blocks can be configured for a single cluster. Separate multiple IP addresses or CIDR blocks with commas (,). Do not add spaces before or after the commas.	Default public address: 127.0.0.1. This denies access from all IPv4 addresses. 0.0.0.0/0: Allows access from all IPv4 addresses. Important We strongly recommend that you do not configure 0.0.0.0/0 due to high security risks. Some clusters and regions do not support 0.0.0.0/0. The availability is subject to the information on the UI or error messages.

   2. After completing the configuration, click OK.

3. Click Access over Internet. On the Kibana logon page, enter the username and password. After you log on, you are redirected to the Kibana console, where you can explore ES data.

   • Username: The username is fixed to elastic.

   • Password: The password that you set when you created the ES cluster. If you forget the password, you can reset it.

     

4. In the Dev Tools console, run the GET / command to query cluster information.

   

   Verification: If the right pane displays a JSON object that contains information such as the version number and cluster name, the connection is successful.

   

Step 3: Create an index

In the Kibana Dev Tools, run the following command to create an index named product_info for wealth management product data.

PUT /product_info
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
      "properties": {
        "productName": {
          "type": "text",
          "analyzer": "ik_smart"
        },
        "annual_rate":{
          "type":"keyword"
        },
        "describe": {
          "type": "text",
          "analyzer": "ik_smart"
        }
    }
  }
}

Key parameter descriptions:

settings: Defines the index shard configuration. For example, it defines 5 primary shards and 1 replica for each primary shard.

mappings: Defines the index fields. For example, it defines three fields in the index document: productName, annual_rate, and describe. Fields of the text type support tokenization and fuzzy search. Fields of the keyword type are used for exact value matching. For more information about field types, see Field data types.

Verification: If the result contains "acknowledged": true and "shards_acknowledged": true, the index is created.

Step 4: Insert data

Run the following command to insert test data in bulk:

POST /product_info/_bulk
{"index":{}}
{"productName":"Wealth Management Product A","annual_rate":"3.2200%","describe":"180-day fixed-term product. Minimum investment: 20,000. Stable returns. Optional push notifications."}
{"index":{}}
{"productName":"Wealth Management Product B","annual_rate":"3.1100%","describe":"90-day scheduled investment product. Minimum investment: 10,000. Daily push notifications for credited returns."}
{"index":{}}
{"productName":"Wealth Management Product C","annual_rate":"3.3500%","describe":"270-day scheduled investment product. Minimum investment: 40,000. Daily push notifications for immediately credited returns."}
{"index":{}}
{"productName":"Wealth Management Product D","annual_rate":"3.1200%","describe":"90-day scheduled investment product. Minimum investment: 12,000. Daily push notifications for credited returns."}
{"index":{}}
{"productName":"Wealth Management Product E","annual_rate":"3.0100%","describe":"Recommended 30-day scheduled investment product. Minimum investment: 8,000. Daily push notifications for returns."}
{"index":{}}
{"productName":"Wealth Management Product F","annual_rate":"2.7500%","describe":"Popular 3-day short-term product. No service fees. Minimum investment: 500. Return notifications sent by text message."}

Verification: If the result contains "errors": false, the data is inserted.

Step 5: Retrieve data

GET /product_info/_search
{
  "query": {
    "match": {
      "describe": "Daily push notifications for credited returns"
    }
  }
}

GET /product_info/_search
{
  "query": {
    "range": {
      "annual_rate": {
        "gte": "3.0000%",
        "lte": "3.1300%"
      }
    }
  }
}

Clean up resources and next steps

Delete data and release the ES cluster

1. Run the following command to delete the test index created in this tutorial.

   DELETE /product_info

   The following result is returned:

2. Return to the cluster list in the Elasticsearch console.

3. In the cluster list, find the cluster that you want to release. In the Actions column, click > Release Cluster and select Immediately Delete. Then, click OK to release the cluster.

   Important

   Releasing an cluster permanently deletes the cluster and all data it contains. This action is irreversible. Proceed with caution.

Next steps for learning

To learn more about ES, you can explore the following topics:

• Data collection/migration: Collect or migrate your business data to ES.

  Before migration, you need to determine whether to upgrade the cluster configuration based on your data volume.

• AI services: Use retrieval-augmented generation (RAG) technology to enhance each stage of the search process.

• Performance tuning: Explore index design and query optimization techniques to improve cluster performance.

• Kibana visualization: Learn how to use Kibana features such as Discover and Dashboard to create data visualization dashboards.