How to use Container Monitoring Pro Edition - Application Real-Time Monitoring Service

Supported cluster types

ACK Pro cluster
ACK Lingjun cluster
ACK Dedicated cluster

Prerequisites

Before you use Container Monitoring Pro Edition, activate Managed Service for Prometheus (billed by ingested data volume activation link or by ingested sample count activation link), and then activate Container Monitoring Pro Edition.

Container Monitoring Pro Edition billing

Billing item

Description

Billing method

Billing cycle

Cluster scale monitoring fee

OCU usage is calculated based on the number of nodes in your Kubernetes cluster. One OCU equals 10 cluster nodes.

Note

OCU: The Observability Capacity Unit (OCU) is a billing unit for Alibaba Cloud Native Observability. OCU usage is automatically calculated based on hourly resource consumption. The price is USD 0.023 per OCU.

Pay-as-you-go: Daily cluster scale monitoring fee = Sum of hourly OCU count × OCU unit price

Note

Hourly OCU count = ceil(Maximum number of nodes in the current billing cycle / 10)

The billing cycle is hourly. After 00:00 each day, Managed Service for Prometheus calculates the daily fee by determining the maximum node count per hour for the previous day, converting each count to OCUs, summing all hourly OCUs, and multiplying by the OCU unit price.

Prometheus instance fee

For more information, see Prometheus instance billing.

Enable Container Monitoring Pro Edition

Method 1: Select Pro during integration

On the Integration Center page, select Kubernetes Cluster Monitoring.
In the Kubernetes Cluster Monitoring panel, select your target Container Service for Kubernetes (ACK) cluster, choose Container Monitoring Pro Edition, and click OK.

Method 2: Upgrade to Pro Edition

Important

After you upgrade to Container Monitoring Pro Edition, you cannot downgrade to Basic Edition.

On the Integration Management page, choose Integrated Environments > Container Service.
Find the container monitoring integration you want to upgrade, and in its Actions column, click Upgrade. In the confirmation dialog box that appears, click OK.

Check for a leftover unmanaged collector agent after the upgrade

Container Monitoring Basic Edition deploys an unmanaged collector agent, the arms-prometheus-ack-arms-prometheus Deployment, in your cluster. After you upgrade to Pro Edition, this Deployment can remain in the cluster. If it does, it collects metrics such as GPU metrics in parallel with the managed collector agent. A single metric then appears as two time series that differ only in the job label, which doubles your monitoring data.

To detect and remove a leftover unmanaged collector agent, perform the following steps:

Check whether an unmanaged collector agent remains in the arms-prom namespace:
```
kubectl -n arms-prom get deployments.apps
```
If the output includes arms-prometheus-ack-arms-prometheus, the unmanaged collector agent is still running.
Verify duplicate collection by using PromQL. On the metric query page of your Prometheus instance or in Grafana, run the following query. The GPU framebuffer total metric is used as an example:
```
count(DCGM_CUSTOM_DEV_FB_TOTAL)
```
A result greater than 1 means that the metric has multiple time series, which indicates duplicate collection. To confirm, check that the series report the same value but carry different job labels. For example, job=gpu-exporter is the managed collector agent, and job=node-gpu-exporter is the collection path of the unmanaged collector agent. You can verify other GPU metrics, such as DCGM_FI_DEV_GPU_UTIL, in the same way.

Scale the leftover unmanaged collector agent down to zero replicas:

kubectl -n arms-prom scale deployment arms-prometheus-ack-arms-prometheus --replicas=0

Wait about 5 minutes for the metric staleness window, and then run the count query again. Verify that the query returns 1 and that only the time series of the managed collector agent remains. After the monitoring data returns to normal, delete the Deployment:
```
kubectl -n arms-prom delete deployment arms-prometheus-ack-arms-prometheus
```

Basic Edition vs. Pro Edition

Category	Basic Edition	Pro Edition
Metric retention for basic container cluster metrics	7 days	90 days
Prometheus collector	Deploys an in-cluster agent that you manage. This agent consumes cluster resources (3 CPU cores and 4 GB of memory per replica by default).	Uses a fully managed collector agent that eliminates resource costs on your cluster and provides a 99.95% production-level SLA.
Dashboards	Basic built-in dashboards.	A comprehensive set of built-in dashboards.
Automatic synchronization of `PrometheusRule`	Supported. You can define `PrometheusRule` resources by using `CRDs` to enable automatic scanning and synchronization.	Not supported. The Pro Edition is a fully managed service whose collector agent is managed by Alibaba Cloud, and you cannot modify the parameters of underlying components. You can manage alert rules on the Prometheus alert rules page of the console, which supports two detection types: static thresholds and custom PromQL.

Supported dashboards in Container Monitoring Pro Edition

Type	Dashboard
Monitoring overview	Cluster monitoring overview
Monitoring overview	Cluster Namespace dashboard
Cluster core components	ACK Pro API server
	ACK Pro ETCD
	ACK Pro Scheduler
	ACK Pro Cloud Controller Manager
	ACK Pro Kube Controller Manager
Node monitoring	Node pool overview
Node monitoring	Cluster node monitoring details
Application monitoring	Deployment monitoring
	StatefulSet monitoring
	DaemonSet monitoring
	Cluster Pod monitoring
Network monitoring	CoreDNS component monitoring
Network monitoring	Cluster Ingress traffic monitoring
Storage monitoring	CSI storage component monitoring (cluster level)
	CSI storage component monitoring (node level)
	Pod IO monitoring (Pod level)
	Frontend storage IO monitoring (cluster level)
GPU monitoring	Cluster GPU monitoring (cluster level)
	Cluster GPU monitoring (node level)
	Cluster GPU monitoring (by application Pod)
Cost analysis/Resource optimization	Resource profile
Others	Backend storage IO monitoring (cluster level)
	k8s-reclaimed-resource
	Cluster Prometheus self-monitoring
	Virtual node (ECI) overview

Default alert rules

Alert rule name	Alert group	Template
Node CPU usage greater than 75%	node	Node {{ $labels.instance }} CPU usage is greater than 75%. Current usage: {{ printf "%.2f" $value }}%
Node CPU usage greater than 85%	node	Node {{ $labels.instance }} CPU usage is greater than 85%. Current usage: {{ printf "%.2f" $value }}%
Node memory usage greater than 75%	node	Node {{ $labels.instance }} memory usage is greater than 75%. Current usage: {{ printf "%.2f" $value }}%
Node memory usage greater than 85%	node	Node {{ $labels.instance }} memory usage is greater than 85%. Current usage: {{ printf "%.2f" $value }}%
Node anomalies	node	Node {{$labels.node}} has been unavailable for more than 10 minutes.
Disk usage greater than 95%	node	Node {{ $labels.instance }} disk {{ $labels.device }} usage has exceeded 95%. Current usage: {{ printf "%.2f" $value }}%
Deployment Pod availability less than 50%	workload	Namespace: {{$labels.namespace}} / Deployment: {{$labels.deployment}} Pod availability is less than 50%. Current number of unavailable Pods: {{ $value }}
Job execution failed	workload	Namespace: {{$labels.namespace}}/Job: {{$labels.job_name}} execution failed.
Pod startup timeout	workload	Namespace: {{$labels.namespace}}/Pod: {{$labels.pod_name}} has failed to start for more than 15 minutes. Reason for waiting: {{$labels.reason}}
Pod status abnormal	workload	Namespace: {{$labels.namespace}}/Pod: {{$labels.pod_name}} has been in the {{$labels.phase}} state for more than 10 minutes.
Frequent Pod restarts	workload	Namespace: {{$labels.namespace}}/Pod: {{$labels.pod_name}} has restarted more than {{ $labels.metrics_params_value}} times in the last {{$labels.metrics_params_time}} minutes. Current restart count: {{ $value }}
Container CPU usage exceeds 85%	workload	In Namespace {{$labels.namespace}}, the CPU usage of container {{$labels.container}} in Pod {{$labels.pod_name}} exceeds 85%. Current usage: {{ printf "%.2f" $value }}%
Container CPU usage exceeds 75%	workload	In Namespace {{$labels.namespace}}, the CPU usage of container {{$labels.container}} in Pod {{$labels.pod_name}} exceeds 75%. Current usage: {{ printf "%.2f" $value }}%
Container memory usage exceeds 75%	workload	In Namespace {{$labels.namespace}}, the memory usage of container {{$labels.container}} in Pod {{$labels.pod_name}} exceeds 75%. Current usage: {{ printf "%.2f" $value }}%
Container memory usage exceeds 85%	workload	In Namespace {{$labels.namespace}}, the memory usage of container {{$labels.container}} in Pod {{$labels.pod_name}} exceeds 85%. Current usage: {{ printf "%.2f" $value }}%