Managed Service for Grafana:Add and use an Elasticsearch data source by using a public IP address

Background information

Alibaba Cloud Elasticsearch is a fully managed, out-of-the-box cloud service developed based on open source Elasticsearch. It is compatible with open source features and supports the pay-as-you-go billing method. In addition to built-in components such as Elasticsearch, Logstash, Kibana, and Beats, Alibaba Cloud Elasticsearch provides the free X-Pack plugin together with Elastic. Widely used in real-time log analysis and processing, information retrieval, and multidimensional data query and analysis, Alibaba Cloud Elasticsearch integrates advanced capabilities such as security, SQL, machine learning, alerting, and monitoring. For more information, see What is Alibaba Cloud Elasticsearch?

Prerequisites

Before you begin, make sure that you have:

• A Managed Service for Grafana workspace

• An Alibaba Cloud Elasticsearch cluster with Public Network Address turned on

• The public endpoint of the Elasticsearch cluster (available on the Basic Information page in the Elasticsearch console)

• The username and password for the Elasticsearch cluster (default username: elastic)

• The IP address of your Grafana workspace (available on the Workspace Information page in the Managed Service for Grafana console)

• The name of the Elasticsearch index to query and its time field

Step 1: Add the Grafana workspace IP to the Elasticsearch whitelist

Add the Grafana workspace IP address to the Elasticsearch cluster's public IP whitelist so that Grafana can connect to the cluster.

1. Log on to the Alibaba Cloud Elasticsearch console.

2. In the left-side navigation pane, choose Elastic Stack Cloud Native PaaS > Elasticsearch Clusters.

3. In the top navigation bar, select the resource group and region of the target cluster, then click the cluster ID.

4. In the left-side navigation pane, choose Configuration and Management > Security.

5. In the Network Settings section, click Modify next to Public IP Address Whitelist.

   Note

   Public Network Address must be turned on before you can modify the whitelist. By default, this switch is turned off.

6. Click Configure next to the default whitelist.

7. In the Modify IP Address Whitelist dialog box, enter the IP address of your Grafana workspace in the IP Addresses in Whitelist field.

   Note

   To find the Grafana workspace IP address, go to the Workspace Information page in the Managed Service for Grafana console. For more information, see View or modify workspace information.

8. Click OK.

Step 2: Log on to Grafana

1. Log on to the Managed Service for Grafana console. In the left-side navigation pane, click Workspace Management.

2. Find your workspace and click the link in the URL column to open Grafana.

   Note

   Log on with the Grafana administrator account and the password you set when creating the workspace. Alternatively, click Sign in with Alibaba Cloud to use your current Alibaba Cloud account.

Step 3: Add the data source

Open the data source configuration page in Grafana:

• Grafana 9.0.x: In the left-side navigation pane, choose > Data sources, then click Add data source.

• Grafana 10.0.x: Click on the homepage, then choose Management > Data sources and click + Add new data source.

Search for and select Elasticsearch, then configure the following settings on the Settings tab:

Click Save & test.

• Grafana 9.0.x: The message Index OK. Time field name OK confirms a successful connection.

• Grafana 10.0.x: The message Data source is working confirms a successful connection.

Step 4: Create a dashboard panel

1. Create a dashboard:

   • Grafana 9.0.x: In the left-side navigation pane, choose > New dashboard, then click Add a new panel.

   • Grafana 10.0.x: Click , choose Dashboards, select New dashboard from the New drop-down list, then click + Add visualization.

2. On the Query tab (Grafana 9.0.x) or in the Select data source dialog box (Grafana 10.0.x), select the Elasticsearch data source you added in Step 3.

3. Configure the query. For example, to view a log list with details, set Metric to Logs and select Logs from the Visualizations drop-down list in the upper-right corner.

   

4. Click Apply.

5. Click the icon in the upper-right corner, specify a dashboard name and folder, then click Save.

Step 5: View monitoring data

Navigate to the dashboard you saved:

• Grafana 9.0.x: In the left-side navigation pane, choose > Browse. Find your folder and click the dashboard.

• Grafana 10.0.x: Click , then click Dashboards. Find your folder and click the dashboard.

What's next

• Grafana Elasticsearch data source documentation -- Explore advanced query features, template variables, and alerting for Elasticsearch data sources.

• View the basic information of a cluster -- Find endpoint URLs, version numbers, and other cluster details.