This topic describes how to integrate container event monitoring with Managed Service for Prometheus and how to view dashboards and set alert rules. Event monitoring is a monitoring method in Kubernetes that complements resource monitoring by addressing its limitations in timeliness, accuracy, and scenario coverage. You can monitor cluster anomalies and issues in real time by using NPD (node-problem-detector) with the Kubernetes event center of SLS, configuring NPD cluster checks and an abnormal event offline feature, and using DingTalk, SLS offline Kubernetes events, and EventBridge offline Kubernetes events.
Prerequisites
-
You have activated Managed Service for Prometheus. For more information, see Billing of Prometheus instances.
-
You have created a Kubernetes cluster. For more information, see Create an ACK managed cluster.
-
You have activated Resource Center. For more information, see Activate Resource Center.
Step 1: Integrate container event monitoring
-
Log on to the or the Prometheus console. In the left-side navigation pane, click Integration Center.
-
Click the Kubernetes Event card, select the ACK cluster to integrate, and then follow the on-screen instructions to complete the component integration.
NoteAfter integration, full data ingestion takes approximately one to two minutes. Dashboards do not display data until ingestion is complete.
Step 2: View dashboards
Managed Service for Prometheus provides a variety of built-in dashboards for container monitoring, covering cluster overviews, core components, nodes, and pods. You can access these dashboards from the Container Service for Kubernetes (ACK) console, ARMS console, and Prometheus console.
-
Log on to the or the Prometheus console. In the left-side navigation pane, click Integration Management.
-
On the Integration Management page, click the Query Dashboards tab. From the Select Environment drop-down list, select the target container environment, for example,
mengtu-test-api. You can filter the dashboards by using the Scenario and Source Component drop-down lists, and by using label filters such as api-server, etcd, kubernetes, node, and storage. The list updates to show the available dashboards that match your criteria.
Step 3: Set up alerts
-
Log on to the or the Prometheus console. In the left-side navigation pane, click Integration Management.
-
On the Integration Management page, click the Integrated Environments tab. Select Container Service, and then click the name of the target environment to open its details page.
-
Click the Component Management tab. In the component type list on the left, select Kubernetes Event. Then, on the right, click the Alert Rule tab to view the built-in alert rules.
-
Built-in alert rules generate warning events, but they do not send alert notifications. If you want to send alert notifications to an email address or another platform, click Edit to configure a notification policy.
On the alert configuration page, you can also customize alert thresholds, duration, and content. For more information on configuring alerts, see Create a Prometheus alert rule.
In the Alert Notification section, select Normal Mode. From the Quickly specify notification policy drop-down list, select a notification policy or click Create notification policy. Expand Advanced Settings and set the Alert check cycle to
1minute.
Collected metrics
|
Metric |
Type |
Description |
|
eventer_events_error_total |
COUNTER |
The total number of error events. |
|
eventer_events_normal_total |
COUNTER |
The total number of normal events. |
|
eventer_events_warning_total |
COUNTER |
The total number of warning events. |
|
eventer_exporter_duration_milliseconds |
SUMMARY |
The time in milliseconds spent exporting events. |
|
eventer_manager_last_time_seconds |
GAUGE |
The Unix timestamp in seconds of the last eventer housekeeping task. |
|
eventer_scraper_duration_milliseconds |
SUMMARY |
The time in milliseconds spent scraping events. |
|
eventer_scraper_events_total_number |
COUNTER |
The total number of scraped events. |
|
eventer_scraper_last_time_seconds |
GAUGE |
The Unix timestamp in seconds of the last scraped event. |