Use Alibaba Cloud Managed Service for Prometheus to observe applications deployed in ECS instances

This topic describes how to use Alibaba Cloud Managed Service for Prometheus to observe the metrics of applications deployed in Elastic Compute Service (ECS) instances.

Typical deployment scenarios of applications deployed in ECS instances

Scenario 1: Deploy applications over public cloud VPCs

In this scenario, business applications are deployed over one or more virtual private clouds (VPCs), and multiple ECS instances are configured in each VPC. Basic components, such as databases and middleware, and business applications are deployed in these ECS instances. Therefore, you need to observe the metrics of the Linux or Windows operating system, basic components, and business applications for each ECS instance.

Scenario 2: Deploy applications in a data center over a VPC

In this scenario, you need to deploy a VPC to connect to your data center. In most cases, you can use a leased line to connect the VPC to the data center. Therefore, you need a complete metric observation platform to observe the metrics of a VPC and a data center at the same time.

Scenario 3: Deploy applications in multi-cloud ECS instances over a VPC

In this scenario, you need to deploy applications over a VPC, and connect various ECS instances across clouds over the Internet. Therefore, you need a complete metric observation platform to observe metrics from an integrated and global perspective.

Observation of applications deployed in ECS instances based on a self-built Prometheus service

If you use a self-built Prometheus service to observe applications deployed in ECS instances, you may face the following challenges:

To ensure security and facilitate organization management, you may need to deploy your business in separate VPCs. In this case, you must deploy a self-built Prometheus service in each VPC to monitor your business. This increases the deployment costs and O&M costs.
For each self-built observation system, you must install and configure Prometheus, Grafana, AlertManager, and Exporter. This process is complex and time-consuming.
Compared with Alibaba Cloud Managed Service for Prometheus, a self-built Prometheus service is not equipped with service discovery that can be quickly integrated with ECS. Therefore, you cannot flexibly define or capture Targets based on ECS tags. If you want to implement service discovery, you must write Golang code to call the POP API of Alibaba Cloud ECS to integrate with the code of the open source Prometheus service. Then, you must compile and compress the code, and then deploy the open source Prometheus service. This process is complex and makes version upgrade difficult.
The open source Grafana dashboards of common components do not allow you to customize dashboards based on the implementation or best practices of observation components.
You must configure alert templates for common components on your own, which is difficult and complex.

Methods for metric data collection that Alibaba Cloud Managed Service for Prometheus provides for applications deployed in ECS instances

Managed Exporters: Alibaba Cloud Managed Service for Prometheus allows you to manage and deploy several common components, such as MySQL and Redis. To monitor the metrics of the components deployed in an ECS instances over a VPC, you only need to configure the monitoring component settings (such as IP address and port) in the Alibaba Cloud Managed Service for Prometheus console. The managed exporters can also collect the metrics from the components deployed in a data center over the leased line that connects the data center to the VPC.
Unmanaged Exporters: For components that cannot be managed in Managed Service for Prometheus, or the custom metrics of your business applications, you can deploy custom Exporters in a VPC or data center. Then, you can configure custom service discovery in the Alibaba Cloud Managed Service for Prometheus console and use Alibaba Cloud Managed Service for Prometheus to discover these Exporters to regularly capture and store metrics.
Node Exporter or Windows Exporter: Node Exporter and Windows Exporter cannot be managed because they need to be deployed in each ECS instance to collect observation information from the operating system of each ECS instance. Alibaba Cloud Managed Service for Prometheus provides native support for Node Exporter.

Alibaba Cloud Managed Service for Prometheus provides maintenance-free, out-of-the-box metric observation capabilities for the operating systems of the ECS instances deployed in VPCs or data centers connected to VPCs, common middleware, and business applications. This ensures the collaboration and closed loop of one-stop metric observation in cloud-native and other environments. To use Alibaba Cloud Managed Service for Prometheus to observe applications deployed in an ECS instance, perform the following steps:

Step 1: Create a Prometheus instance for ECS

Create a Prometheus instance for ECS in the Managed Service for Prometheus console. For more information, see Create a Prometheus instance to monitor an ECS instance.

Step 2: Install and configure an Exporter

Alibaba Cloud Managed Service for Prometheus supports multiple Exporters such as Node Exporter, MySQL, Redis, Elasticsearch, Kafka, NGINX, MongoDB, PostgreSQL, RabbitMQ, RocketMQ, and BlackBox. Managed Service for Prometheus also supports static_configs and aliyun_sd_configs that are commonly used for service discovery. This way, you can configure Exporters to observe ECS instances.

In this example, MySQL is used to describe how to install and configure an Exporter.

Log on to the Managed Service for Prometheus console.
In the left-side navigation pane, click Monitoring List.
Click the name of the Prometheus instance instance that you want to manage to go to the Integration Center page.
On the Integration Center page, click Install next to MySQL.
In the Integrate MySQL panel, configure the parameters. For more information, see Install and configure a MySQL exporter.
After you configure the parameters, click OK. The Targets tab appears.

Step 3: View the dashboard

By default, Managed Service for Prometheus is integrated with common Grafana dashboards. You can view the dashboard of each component without the need to install Grafana.

After you configure an Exporter in Step 2: Install and configure an Exporter, click the Exporter card in the Installed section of the Integration Center page. In the panel that appears, click the Dashboards tab to view the thumbnail and hyperlink of the Exporter. Click the hyperlink to go to the Alibaba Cloud Grafana page to view the dashboard.

Step 4: Configure alerting

On the Alerts tab, you can create a Prometheus alert rule for MySQL Exporter to view alert information. For more information, see Create an alert rule for a Prometheus instance.