Elasticsearch:Quick start: Create a cluster and retrieve data

What you will learn

• Create and configure an ES cluster.

• Model data by using Dev Tools in Kibana.

  Kibana is one of the core components of ES. It lets you explore, analyze, and present the data stored in ES in an intuitive graphical way. Kibana is built into the ES console, so you do not need to install it manually.

• Insert sample data and run various search operations.

After you complete this tutorial, you can create an index, read and write data, and run searches in ES.

ES version and cost

Before you start, review the following key information about this test:

• ES type and version: Vector Enhanced Edition 8.17.0.

  This edition modularizes the core algorithm services in the AI search pipeline, including document parsing, chunking, text embedding, query analysis, retrieval, ranking, and large language model (LLM) integration. It supports semantic search and helps enterprises quickly build Retrieval-Augmented Generation (RAG) and multi-modal search capabilities.

• Estimated cost: This tutorial uses pay-as-you-go billing throughout and applies a minimal configuration to keep the trial cost low: a single zone, the smallest node specifications and storage space, no dedicated master node, and no advanced enhancement features that incur additional fees. If you follow the procedure in this topic, which takes about 45 minutes, and release the cluster immediately after you finish, the estimated total cost is less than USD 5.

With the configuration recommended in this topic, only three pay-as-you-go items are charged: data node specifications, data node storage space, and Kibana node specifications. You can view the billing items and the unit price of each item at the bottom of the purchase page.

ES clusters are settled by the hour, and fees keep accruing as long as the cluster is running. After the test, release the cluster immediately by following "Clean up resources and next steps" in this topic to avoid unnecessary fees.

Prerequisites

• Register and log on to an Alibaba Cloud account. Make sure that the account has the permissions to create and manage resources such as Elasticsearch and Virtual Private Cloud (VPC).

• Create a Virtual Private Cloud (VPC) and a vSwitch, and record the region and zone where the VPC and vSwitch reside. When you create the ES cluster later, you must select the same region and zone. Otherwise, the cluster cannot be associated with them.

Procedure

Step 1: Create a cluster (about 20 minutes)

1. Log on to the Alibaba Cloud Elasticsearch console. On the Elasticsearch Clusters page, click Create to go to the cluster purchase page. Configure the following key parameter settings and leave the other parameters at their default values.

   Parameter	Description
   Billing Method	Pay-as-you-go. You can release the cluster at any time.
   Region and zone	Region: Select the region where the VPC and vSwitch that you created reside. Zone: Three zones are selected by default. For a test, manually switch to a single zone to reduce the waiting time for cluster creation. For a production environment, you can upgrade to multiple zones. This example uses China (Hangzhou) / Zone I. If you keep three zones or two zones, the number of data nodes must be at least 3 and must be a multiple of 3, so you cannot set it to the 2 data nodes that this tutorial uses.
   Network Type	For the network type, select the VPC and vSwitch that you created.
   Cluster Type and Elasticsearch Version	Vector Enhanced Edition 8.17.0. This tutorial is based on this version.
   Scenario and advanced features	Keep the default retrieval scenario. Two advanced enhancement features are enabled by default in this scenario. Handle them as follows: FalconSeek high-performance search engine: Keep it enabled. Enabling it is free and adds no billing item, so keep it to try the accelerated retrieval capability. OpenStore storage engine (high-performance retrieval): Turn it off. This feature is billed by OpenStore storage space (DFS) and cache disk space. When it is enabled, 160 GiB of shared storage is purchased by default and additional fees are incurred. This tutorial does not require it.
   Data Node Specifications	Node specifications: 2 vCPU and 4 GiB of memory, the smallest available option. Storage type: keep the default. Storage space per node: 20 GiB, the smallest available option. Number of data nodes: 2, the smallest value that you can configure in a single zone. These data nodes store index data and mainly create, retrieve, update, delete, and aggregate documents.
   Dedicated Master Node	Selected by default, with three nodes of 2 vCPU and 4 GiB of memory each. This tutorial is a trial scenario, so clear the check box for the dedicated master node to reduce the cost. A dedicated master node manages the cluster metadata. After you clear the check box, the page warns that the two data nodes have a split-brain risk. You can ignore this warning for a short functional trial, but make sure that you enable the dedicated master node before you launch production workloads.
   Cluster Name	ES_test
   Password	Set a custom password. You use this password later to log on to Kibana to build indexes and explore data.

2. Click Buy Now and wait about 20 minutes until the cluster status changes to Active. Then you can proceed to the next step.

Step 2: Configure and log on to Kibana

Public network access to Kibana is enabled by default. For security reasons, public access denies all IP addresses by default, so you must add the IP address of the device that you want to use to the whitelist before you can access Kibana.

Access authentication uses two-factor verification. First, log on to your Alibaba Cloud account. Then, complete a second logon with the access credentials of the ES cluster, where the username is fixed as elastic with its corresponding logon password.

1. In the left-side navigation pane, click Data Visualization. In the Kibana section, click Modify Configuration.

2. In the Network Access Configuration section, modify the Kibana public network access configuration and its whitelist.

   Obtain the IP address of your device

   Get the IP address of the device that you want to use based on the following scenarios.

   Scenario	IP address to obtain	How to obtain it
   Access Kibana from an on-premises device over the internet.	The public IP address of the on-premises device. Note If the device is on a home network or a corporate local area network (LAN), add the public egress IP address of the LAN.	Run the curl ipinfo.io/ip command to query the public IP address of the device.
   Access Kibana from an ECS instance over the internet.	When the ECS instance and the ES cluster are in different VPCs, you can access Kibana through the public IP address of the ECS instance. Obtain the public IP address of the ECS instance and add it to the public access whitelist of the ES cluster.	Log on to the ECS console and view the public IP address of the instance in the instance list.

   Add to the public access whitelist

   Obtain the IP address of your device and add it to the public access whitelist.

   1. Click Configure to the right of the default group. In the dialog box that appears, add IP addresses to the whitelist.

      Configuration type	Format and example	Important notes
      IPv4 address format	Single IP address: 192.168.0.1 CIDR block: 192.168.0.0/24. We recommend that you consolidate scattered IP addresses into CIDR blocks. A single cluster supports a maximum of 300 IP addresses or CIDR blocks. Separate multiple IP addresses or CIDR blocks with commas (,), and do not add spaces before or after the commas.	Default public address: 127.0.0.1, which denies access from all IPv4 addresses. 0.0.0.0/0: Allows access from all IPv4 addresses. Important Do not configure 0.0.0.0/0, because it poses a high security risk. Some clusters and regions do not support 0.0.0.0/0. Refer to the console or the error message for accurate information.

   2. After you complete the configuration, click OK.

3. Click Access over Internet. On the Kibana logon page, enter the username and password. After you log on successfully, you enter the Kibana console and can explore ES data.

   • Username: fixed as elastic.

   • Password: the password that you set when you created the ES cluster. If you forget it, you can reset the password.

4. Go to the Developer Tools console and run the GET / command to query the cluster information.

   Verification: If the right pane returns a JSON object that contains information such as the version number and the cluster name, the connection is normal.

Step 3: Create an index

In Kibana Dev Tools, run a RESTful API operation to create an index (mapping) named product_info for wealth management product data.

PUT /product_info
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
      "properties": {
        "productName": {
          "type": "text",
          "analyzer": "ik_smart"
        },
        "annual_rate":{
          "type":"keyword"
        },
        "describe": {
          "type": "text",
          "analyzer": "ik_smart"
        }
    }
  }
}

Key parameter descriptions:

settings: defines the shard configuration of the index. For example, this configuration defines five primary shards and one replica for each primary shard.

mappings: defines the index fields. For example, this configuration defines three fields in the index file: productName, annual_rate, and describe. Fields of the text type support tokenization and fuzzy search, and fields of the keyword type are used for exact value matching. For more information about field types, see Field data types.

Verification: If the response contains "acknowledged": true and "shards_acknowledged": true, the index was created successfully.

Step 4: Insert data

Run the following code to bulk insert test data:

POST /product_info/_bulk
{"index":{}}
{"productName":"Wealth Management Product A","annual_rate":"3.2200%","describe":"180-day fixed-term product. Minimum investment: 20,000. Stable returns. Optional push notifications."}
{"index":{}}
{"productName":"Wealth Management Product B","annual_rate":"3.1100%","describe":"90-day scheduled investment product. Minimum investment: 10,000. Daily push notifications for credited returns."}
{"index":{}}
{"productName":"Wealth Management Product C","annual_rate":"3.3500%","describe":"270-day scheduled investment product. Minimum investment: 40,000. Daily push notifications for immediately credited returns."}
{"index":{}}
{"productName":"Wealth Management Product D","annual_rate":"3.1200%","describe":"90-day scheduled investment product. Minimum investment: 12,000. Daily push notifications for credited returns."}
{"index":{}}
{"productName":"Wealth Management Product E","annual_rate":"3.0100%","describe":"Recommended 30-day scheduled investment product. Minimum investment: 8,000. Daily return message notifications."}
{"index":{}}
{"productName":"Wealth Management Product F","annual_rate":"2.7500%","describe":"Popular short-term product. 3-day term, no fees. Minimum investment: 500. Return notifications by SMS."}

Verification: If the response contains "errors": false, the data was inserted successfully.

Step 5: Search for data

GET /product_info/_search
{
  "query": {
    "match": {
      "describe": "Daily push notifications for credited returns"
    }
  }
}

GET /product_info/_search
{
  "query": {
    "range": {
      "annual_rate": {
        "gte": "3.0000%",
        "lte": "3.1300%"
      }
    }
  }
}

Clean up resources and next steps

Delete data and release the ES cluster

1. Run the following code to delete the test index that you created in this topic.

   DELETE /product_info

   The following response is returned:

   {
   "acknowledged" : true
   }

2. Return to the cluster list in the Alibaba Cloud Elasticsearch console.

3. In the cluster list, find the cluster that you want to release. In the Actions column of the cluster, click > Release Cluster, select Immediately Delete, and then click OK to release the cluster.

   Important

   Releasing a cluster permanently deletes the cluster and all the data in it. This action is irreversible, so proceed with caution.

Next steps

To learn more about ES, explore the following topics:

• Data ingestion or migration: collect or migrate your business data to ES.

  Before migration, make a preliminary assessment based on your data volume to determine whether you need to upgrade the cluster configuration.

• Performance tuning: Explore index design and query optimization techniques to improve cluster performance.

• Kibana visualization: Learn how to use Kibana features such as Discover and Dashboard to create data visualization dashboards.