All Products
Search
Document Center

Application Real-Time Monitoring Service:Add and use an Elasticsearch data source by using the public IP address of an Elasticsearch cluster

Last Updated:Apr 08, 2024

Alibaba Cloud Managed Service for Grafana is integrated with the Alibaba Cloud Elasticsearch plug-in. You can use the plug-in to synchronize Elasticsearch data and display it in Grafana dashboards in real time. This topic describes how to use the public IP address of an Elasticsearch cluster to synchronize Elasticsearch data to Managed Service for Grafana and use an Elasticsearch data source to create a dashboard.

Background information

Alibaba Cloud Elasticsearch is a fully managed and out-of-the-box cloud service developed based on open source Elasticsearch. Completely compatible with the open source features, it supports the pay-as-you-go billing method. In addition to the out-of-the-box components such as Elasticsearch, Logstash, Kibana, and Beats, Alibaba Cloud Elasticsearch provides the free X-Pack plug-in together with Elastic. Widely used in real-time log analysis processing, information retrieval, and multidimensional data query and analysis, Alibaba Cloud Elasticsearch integrates advanced capabilities such as security, SQL, machine learning, alerting, and monitoring. For more information, see What is Alibaba Cloud Elasticsearch?

Step 1: Add the IP address of a Grafana workspace to the whitelist of the Elasticsearch cluster

In the Elasticsearch console, add the IP address of a Grafana workspace to the whitelist.

  1. Log on to the Alibaba Cloud Elasticsearch console.
  2. In the left-side navigation pane, click Elasticsearch Clusters.
  3. Navigate to the desired cluster.
    1. In the top navigation bar, select the resource group to which the cluster belongs and the region where the cluster resides.
    2. On the Elasticsearch Clusters page, find the cluster and click its ID.
  4. In the left-side navigation pane of the page that appears, choose Configuration and Management > Security.

  5. In the Network Settings section, click Modify on the right side of Public IP Address Whitelist.

    Note

    Before you modify the Public IP Address Whitelist parameter, make sure that Public Network Address is turned on. By default, the switch is turned off.

    1. In the Modify Public IP Address Whitelist panel, click Configure on the right side of the default whitelist.

    2. In the Modify IP Address Whitelist dialog box, enter the IP address of a Grafana workspace in the IP Addresses in Whitelist text box.

      Note

      You can obtain the IP address of the Grafana workspace on the Workspace Information page in the Managed Service for Grafana console. For more information, see View or modify workspace information.

    3. Click OK.

Step 2: Add the data source to Managed Service for Grafana

  1. Log on to the ARMS console. In the left-side navigation pane, choose Managed Service for Grafana > Workspace Management.

  2. On the Workspace Management page, find the workspace that you want to manage and click the URL in the URL column to go to Grafana.

    Note

    You can log on to Grafana with the administrator account of Grafana and the password that you configured when you created the workspace. You can also click Sign in with Alibaba Cloud to log on to Grafana with the current Alibaba Cloud account.

  3. In the left-side navigation pane, choose Data sources.设置

  4. On the Data sources tab, click Add data source.

  5. Enter Elasticsearch in the search box, and then click Elasticsearch.

  6. On the Settings page, set the following parameters.

    Section

    Parameter

    Description

    Example

    Name

    The name of the data source. The name can be user-defined.

    Elasticsearch data source

    HTTP

    URL

    The public IP address of the Elasticsearch cluster. Format: http://{Elasticsearch address}:9200.

    Note

    For more information, see View the basic information of a cluster.

    http://es-cn-xxxxxxxxx.public.elasticsearch.aliyuncs.com:9200

    Auth

    Basic auth

    Turn on Basic auth and specify the parameters in the Basic Auth Details section.

    None

    Basic Auth Details

    User

    Enter the username of the Elasticsearch cluster.

    elastic

    Password

    Enter the password of the Elasticsearch cluster.

    Note

    If you forget the password of the Elasticsearch cluster, you can reset it. Log on to the Elasticsearch console. In the left-side navigation pane of the cluster details page, choose Configuration and Management > Security.

    None

    Elasticsearch details

    Index name

    Enter the name of the index created for the Elasticsearch cluster.

    Note
    • If you have not created an index for the Elasticsearch cluster, you can temporarily use the self-monitoring index of Elasticsearch to check whether Managed Service for Grafana can access the cluster. The self-monitoring index of Elasticsearch is named .monitoring-es-7-2022.01.11. Replace "2022.01.11" in the name with a date within the last two days. Otherwise, data may not be synchronized. Make sure that the date is later than the date when the Elasticsearch cluster is created.

    • For more information, see Grafana documentation.

    .monitoring-es-7-2022.01.11

    Time field name

    Enter the timestamp in the index.

    Note

    If you set the Index name parameter to the self-monitoring index of Elasticsearch, set the value to timestamp.

    timestamp

    Version

    Enter the version of the Elasticsearch cluster.

    Note

    To obtain the version of the Elasticsearch cluster, go to the Basic Information page of the Elasticsearch console.

    7.10+

  7. Click Save & test.

    If the Index OK. Time field name OK. message is returned, the Elasticsearch data source is added to Managed Service for Grafana. For more information, see Grafana documentation.

Step 3: Create a dashboard panel

After you add the Elasticsearch data source, you can create a dashboard panel.

  1. In the left-side navigation pane of the Grafana console, choose Dashboard > + New dashboard.

  2. On the New dashboard page, click Add new panel.

  3. On the Query tab of the Edit Panel page, select the Elasticsearch data source and configure metrics.

    Assume that you want to configure a panel to browse the log list and details, you can set the Metric parameter to Logs and select Logs from the Visualizations drop-down list in the upper-right corner.Elasticsearch数据源创建面板

  4. In the right-side pane, specify the name, type, and graph styles of the monitoring chart.

  5. Click Apply.

    The dashboard panel is created.

  6. In the upper-right corner, click the icon. Then, specify the name of the dashboard and the directory in which the dashboard resides.保存监控大盘

  7. Click Save.

    The dashboard is created.

View monitoring data

  1. In the left-side navigation pane of the Grafana console, choose Dashboard > Browse.

  2. On the Browse tab, find the folder in which the dashboard is stored and click the dashboard.

    View all monitoring charts on the dashboard.