The Grafana is integrated with Alibaba Cloud Search and Analysis Service Elasticsearch Edition. The Elasticsearch plug-in allows you to synchronize Elasticsearch data and present it on the Grafana dashboard in real time. This topic describes how to use a public endpoint to add Elasticsearch data to Grafana and use the Elasticsearch data source to create a dashboard panel.

Background information

Search and Analysis Service Elasticsearch Edition is a fully managed cloud service built on open-source Elasticsearch. It is 100% compatible with open-source features and supports out-of-the-box and pay-as-you-go. In addition to Elastic Stack components such as Elasticsearch, Logstash, Kibana, and Beats, Alibaba Cloud provides the X-Pack plug-in free of charge together with Elastic. X-Pack is integrated into Kibana to provide features, such as security, alerting, monitoring, and machine learning. It also provides SQL capabilities. Alibaba Cloud Elasticsearch is widely used in scenarios such as real-time log analysis and processing, information retrieval, multidimensional data queries, and statistical data analytics. For more information, see What is Alibaba Cloud Elasticsearch?.

Step 1: Configure a public network whitelist

In the Alibaba Cloud Elasticsearch console, add the connection IP address of Grafana as a whitelist.

  1. Log on to the Alibaba Cloud Elasticsearch console.
  2. In the left-side navigation pane, click Elasticsearch Clusters.
  3. Navigate to the desired cluster.
    1. In the top navigation bar, select the resource group to which the cluster belongs and the region where the cluster resides.
    2. On the Elasticsearch Clusters page, find the cluster and click its ID.
  4. In the left-side navigation pane of the page that appears, choose Configuration and Management > Security.
  5. In the Cluster Network Settings section, click Modify to the right of public endpoint Whitelist to configure the IP address whitelist.
    Note When you configure the public endpoint whitelist, make sure that the public endpoint switch is turned on (disabled by default), and then perform the following operations:
    1. In the Modify Internet Whitelist panel, click Configure on the right side of the default whitelist.
    2. In the Add IP Whitelist Group dialog box, enter the connection IP address of Grafana in the IP Address in Whitelist section.
      Note You can obtain the connection IP address of Grafana on the Workspace Information page of the Grafana console. For more information, see Workspace information .
    3. Click OK.

Step 2: Add data sources

  1. Log on to the Grafana console.
  2. On the Workspaces page, click URL in the Actions column corresponding to the workspace to go to Grafana.
    Note To log on to Grafana, you can use the Admin account of Grafana and the password that you specified when you created the workspace. Alternatively, you can click Sign in with Alibaba Cloud to log on to Grafana by using the Alibaba Cloud account that you used to purchase the workspace.
  3. In the left-side navigation pane of Grafana, choose Live stream settings > Data sources.
  4. On the Data Sources tab, click Add data source.
  5. Search for the Elasticsearch by using the text box at the top of the page, and then click Elasticsearch.
  6. On the Settings page, set the following parameters.
    NameThe name of the data source. You can customize the name of the data source. Elasticsearch connection
    HTTPURLThe connection address of the Elasticsearch, in the format of http://{Elasticsearch address}:9200.

    For more information, see View the basic information of a cluster.
    AuthBasic authTurn on the Basic auth switch, and then enter the Basic Auth Details. None
    Basic Auth DetailsUserEnter the username of the Elasticsearch instance. elastic
    PasswordEnter the access password for the Elasticsearch instance.
    Note If you forget the password of the Elasticsearch instance, you can reset the password of the Elasticsearch instance on the Configure and Manage > Security page in the Alibaba Cloud Elasticsearch Elasticsearch console.
    Elasticsearch DetailIndex nameEnter the name of the index created in the Elasticsearch instance.
    • If no index is created for your Elasticsearch instance, you can enter the index of Elasticsearch to check whether Grafana can access your Elasticsearch instance. The index name of Elasticsearch is .monitoring-es-7-2022.01.11. Enter the date after the name in the last two days. Otherwise, there may be no data. Also ensure that the date is the date after the ES instance is created.
    • For more information, see Grafana documentation.
    Time field nameEnter the corresponding timestamp field in Index.
    Note If you are using the Elasticsearch index mentioned in the description of the Index name parameter, the corresponding time field is timestamp.
    VersionEnter the Elasticsearch version.
    Note The Elasticsearch version must be the same as that displayed on the Basic Information page of the instance in the Alibaba Cloud Elasticsearch Elasticsearch console.
  7. Click Save & Test.
    When the page displays Index OK. Time field name OK. indicates that the Elasticsearch data has been added to Grafana. For more information, see Grafana documentation.

Step 3: Create a dashboard panel

After you add a Elasticsearch data source, you can use the added Elasticsearch data source when you create a panel.

  1. In the left-side navigation pane of the Grafana console, choose + > Dashboard.
  2. On the New Dashboard page, click Add Empty Panel.
  3. On the Edit Panel page, select a Elasticsearch data source from the drop-down list in the Query section and set monitoring metrics.
    For example, if you want to configure a panel for browsing log lists and details, you can select Metric as Logs and select Logs from the Visualization in the upper right corner. Elasticsearch Data Source Creation Panel
  4. In the right-side pane, specify the name, type, and layout of the monitoring chart.
  5. In the upper-right corner, click Apply.
    The Dashboard panel is created.
  6. In the upper-right corner, click the Save the dashboard icon. Then, specify the name of the dashboard and the directory in which the dashboard resides.
  7. Click Save.
    The dashboard is created.

View monitoring data

  1. In the left-side navigation pane, choose Dashboard > Manage.
  2. On the Manage tab, click Monitoring Dashboard in the target directory.
    View all monitoring charts on the dashboard.