Alibaba Cloud Managed Service for Grafana is integrated with Alibaba Cloud Elasticsearch. You can use the Elasticsearch plug-in to synchronize data from Elasticsearch and display the data on Grafana dashboards in real time. This topic describes how to use the public endpoint of an Elasticsearch cluster to synchronize data from Elasticsearch to Managed Service for Grafana and use an Elasticsearch data source to create a dashboard.
Background information
Alibaba Cloud Elasticsearch is a fully managed and out-of-the-box cloud service developed based on open source Elasticsearch. Completely compatible with the open source features, it supports the pay-as-you-go billing method. In addition to the out-of-the-box components such as Elasticsearch, Logstash, Kibana, and Beats, Alibaba Cloud Elasticsearch provides the free X-Pack plug-in together with Elastic. Widely used in real-time log analysis and processing, information retrieval, and multidimensional data query and analysis, Alibaba Cloud Elasticsearch integrates advanced capabilities such as security, SQL, machine learning, alerting, and monitoring. For more information, see What is Alibaba Cloud Elasticsearch?
Step 1: Add the IP address of a Grafana workspace to the whitelist of the Elasticsearch cluster
In the Elasticsearch console, add the IP address of a Grafana workspace to the whitelist.
- Log on to the Alibaba Cloud Elasticsearch console.
In the left-side navigation pane, choose Elastic Stack Cloud Native PaaS > Elasticsearch Clusters.
- Navigate to the desired cluster.
- In the top navigation bar, select the resource group to which the cluster belongs and the region where the cluster resides.
- On the Elasticsearch Clusters page, find the cluster and click its ID.
In the left-side navigation pane of the page that appears, choose .
In the Network Settings section of the Security page, click Modify on the right side of Public IP Address Whitelist.
NoteBefore you modify the Public IP Address Whitelist parameter, make sure that Public Network Address is turned on. By default, the switch is turned off.
In the Modify Public IP Address Whitelist panel, click Configure on the right side of the default whitelist.
In the Modify IP Address Whitelist dialog box, enter the IP address of a Grafana workspace in the IP Addresses in Whitelist field.
NoteYou can obtain the IP address of the Grafana workspace on the Workspace Information page in the Managed Service for Grafana console. For more information, see View or modify workspace information.
Click OK.
Step 2: Log on to the Grafana console
Log on to the Managed Service for Grafana console. In the left-side navigation pane, click Workspace Management.
On the Workspace Management page, find the workspace that you want to manage and click the URL in the URL column to go to Grafana.
NoteYou can log on to Grafana with the administrator account of Grafana and the password that you configured when you created the workspace. You can also click Sign in with Alibaba Cloud to log on to Grafana with the current Alibaba Cloud account.
Step 3: Add a data source
Grafana 9.0.x
In the left-side navigation pane of Grafana, choose
> Data sources. On the page that appears, click Add data source. On the Add data source page, search for and click Elasticsearch.
On the Settings tab, set the following parameters.
Section
Parameter
Description
Example
Name
The name of the data source. You can specify a custom name.
Elasticsearch data source
HTTP
URL
The public endpoint of the Elasticsearch cluster. Format:
http://{Elasticsearch endpoint}:9200.NoteYou can obtain the public endpoint of the Elasticsearch cluster on the Basic Information page of the cluster in the Elasticsearch console.
To use the HTTPS protocol, you must enable HTTPS in the Elasticsearch console.
For more information, see View the basic information of a cluster.
http://es-cn-xxxxxxxxx.public.elasticsearch.aliyuncs.com:9200Auth
Basic auth
Turn on Basic auth and configure the parameters in the Basic Auth Details section.
N/A
Basic Auth Details
User
The username of the Elasticsearch cluster.
elastic
Password
The password of the Elasticsearch cluster.
NoteIf you forget the password of the Elasticsearch cluster, you can reset the password by performing the following steps: Log on to the Elasticsearch console. Go to the cluster details page. In the left-side navigation pane, choose . In the Access Settings section, click Reset Password.
N/A
Elasticsearch details
Index name
The name of the index created for the Elasticsearch cluster.
NoteIf you have not created an index for the Elasticsearch cluster, you can temporarily use the self-monitoring index of Elasticsearch to check whether Managed Service for Grafana can access the cluster. The self-monitoring index of Elasticsearch is named .monitoring-es-7-2022.01.11. Replace "2022.01.11" in the name with a date within the previous two days. Otherwise, data may not be synchronized. Make sure that the date is later than the date when the Elasticsearch cluster is created.
For more information, see Grafana documentation.
.monitoring-es-7-2022.01.11
Time field name
The timestamp in the index.
NoteIf you set the Index name parameter to the self-monitoring index of Elasticsearch, the value is set to timestamp.
timestamp
ElasticSearch version
The version of the Elasticsearch cluster.
NoteTo obtain the version of the Elasticsearch cluster, go to the Basic Information page of the cluster in the Elasticsearch console.
7.10+
Click Save & test.
If the
Index OK. Time field name OKmessage appears, the Elasticsearch data source is added to Managed Service for Grafana. For more information, see Grafana documentation.
Grafana 10.0.x
On the Grafana homepage, click the
icon. In the left-side navigation pane, choose .
On the Data sources tab, click + Add new data source.
On the Add data source page, search for and click Elasticsearch.
On the Settings tab, set the following parameters.
Section
Parameter
Description
Example
Name
The name of the data source. You can specify a custom name.
Elasticsearch data source
HTTP
URL
The public endpoint of the Elasticsearch cluster. Format:
http://{Elasticsearch endpoint}:9200.NoteYou can obtain the public endpoint of the Elasticsearch cluster on the Basic Information page of the cluster in the Elasticsearch console.
To use the HTTPS protocol, you must enable HTTPS in the Elasticsearch console.
For more information, see View the basic information of a cluster.
http://es-cn-xxxxxxxxx.public.elasticsearch.aliyuncs.com:9200Auth
Basic auth
Turn on Basic auth and configure the parameters in the Basic Auth Details section.
N/A
Basic Auth Details
User
The username of the Elasticsearch cluster.
elastic
Password
The password of the Elasticsearch cluster.
NoteIf you forget the password of the Elasticsearch cluster, you can reset the password by performing the following steps: Log on to the Elasticsearch console. Go to the cluster details page. In the left-side navigation pane, choose . In the Access Settings section, click Reset Password.
N/A
Elasticsearch details
Index name
The name of the index created for the Elasticsearch cluster.
NoteIf you have not created an index for the Elasticsearch cluster, you can temporarily use the self-monitoring index of Elasticsearch to check whether Managed Service for Grafana can access the cluster. The self-monitoring index of Elasticsearch is named .monitoring-es-7-2022.01.11. Replace "2022.01.11" in the name with a date within the previous two days. Otherwise, data may not be synchronized. Make sure that the date is later than the date when the Elasticsearch cluster is created.
For more information, see Grafana documentation.
.monitoring-es-7-2022.01.11
Time field name
The timestamp in the index.
NoteIf you set the Index name parameter to the self-monitoring index of Elasticsearch, the value is set to timestamp.
timestamp
Click Save & test.
If the
Data source is workingmessage appears, the data source is added to Grafana.
Step 4: Create a dashboard panel
Grafana 9.0.x
In the left-side navigation pane of Grafana, choose
> New dashboard. On the New dashboard page, click Add a new panel.
On the Query tab of the Edit Panel page, select the data source that you added in Step 3 from the Data source drop-down list.
For example, if you want to configure a dashboard panel to view the log list and log details, set the Metric parameter to Logs and select Logs from the Visualizations drop-down list in the upper right corner.
Click Apply.
In the upper-right corner, click the
icon. Then, specify the name of the dashboard and the directory in which the dashboard resides. Click Save.
Grafana 10.0.x
On the Grafana homepage, click the
icon. In the left-side navigation pane, click Dashboards. On the page that appears, select New dashboard from the New drop-down list.
On the New dashboard page, click + Add visualization.
On the right side of the Select data source dialog box, select the data source that you added in Step 3.
For example, if you want to configure a dashboard panel to view the log list and log details, set the Metric parameter to Logs and select Logs from the Visualizations drop-down list in the upper right corner.
Click Apply.
In the upper-right corner, click the
icon. Then, specify the name of the dashboard and the directory in which the dashboard resides. Click Save.
Step 5: View monitoring data
Grafana 9.0.x
In the left-side navigation pane of Grafana, choose
> Browse. On the Browse tab, find the directory and click the dashboard to view the charts.
Grafana 10.0.x
On the Grafana homepage, click the
icon in the upper-left corner. In the left-side panel that appears, click Dashboards. On the Dashboards page, find the directory and click the dashboard to view the charts.