Vector Retrieval Service for Milvus:Configure monitoring alarms

Background information

Milvus monitoring alarms are powered by CloudMonitor. You can set alert rules for specific metrics. When a rule is triggered, all contacts in the specified alert contact group are notified. You can also manage alert contact groups to ensure the right contacts receive notifications promptly. For more information, see What is CloudMonitor?.

Prerequisites

You have created a Milvus instance. For more information, see Create a Milvus instance.

Enable one-click alarm

1. Go to the Milvus page.

   1. Log on to the Alibaba Cloud Milvus console.

   2. In the left-side navigation pane, click Instances.

   3. In the top menu bar, select a region.

2. On the Instances page, click the instance name.

3. Click the Monitoring & Alerting tab.

4. Click Alert Settings.

5. On the Alert Settings page, turn on the One-click Alert switch.

   The default alert rules are described in the following table.

   Rule name	Metric	Metric sampling period	Alert rule
   SystemDefault_acs_milvus_ProcessResidentMemoryUtilization	memory utilization	60 seconds	An alert is triggered if the average memory utilization over 5 consecutive periods is greater than or equal to 80%.
   SystemDefault_acs_milvus_ProcessCPUUtilization	CPU utilization	60 seconds	An alert is triggered if the average CPU utilization over 5 consecutive periods is greater than or equal to 90%.

Create an alert rule

1. On the Alert Settings tab on the Monitoring & Alerting page, click Alert Rule Settings.

2. On the Alert Rules page, click Create alert rule.

3. In the Create alert rule panel, set Product to Vector Retrieval Service for Milvus, set Resource range to All resources, configure the remaining parameters, and then click Confirm.

   For more information about how to create an alert rule for Vector Retrieval Service for Milvus, see Create an alert rule.

   The form also includes fields for the Rule description (which includes the rule name, rule description, and resource description), Mute period (default is 24 hours), Effective period, Tags, and an Alert contact group. Advanced settings include options for Alert callback, Push channels, Recovery notification, and No data handling.

   The following table describes the supported monitoring metrics. Select metrics and configure alert rules based on your business requirements.

   Name	Metric name	Description
   Node alive status	ProcessAliveStatus	The health status of the node. A value of 1 indicates that the node is healthy. A value of 0 indicates that the node is unhealthy.
   CPU Usage	SystemDefault_acs_milvus_ProcessCPUUtilization	Legacy metric. This metric applies only to instances with a 1:4 vCPU-to-memory ratio. We recommend using the new metric, ProcessCPUUtilizationV2.
   CPU utilization (new)	ProcessCPUUtilizationV2	The CPU utilization of the pod container.
   Memory Usage	SystemDefault_acs_milvus_ProcessResidentMemoryUtilization	Legacy metric. This metric applies only to instances with a 1:4 vCPU-to-memory ratio. We recommend using the new metric, ProcessResidentMemoryUtilizationV2.
   memory utilization (new)	ProcessResidentMemoryUtilizationV2	The memory utilization of the pod container.
   query QPS	QueryQPS	The rate of query operations (requests per second).
   search QPS	SearchQPS	The rate of search operations (requests per second).
   insert QPS	InsertQPS	The rate of insert operations (requests per second).
   upsert QPS	UpsertQPS	The rate of upsert operations (requests per second).
   bulk insert QPS	BulkInsertQPS	The rate of bulk insert operations (requests per second).
   delete QPS	DeleteQPS	The rate of delete operations (requests per second).
   vector search QPS	SearchVectorsQPS	The rate of vector search operations (requests per second).
   vector insert QPS	InsertVectorsQPS	The rate of vector insert operations (requests per second).
   vector upsert QPS	UpsertVectorsQPS	The rate of vector upsert operations (requests per second).
   vector delete QPS	DeleteVectorsQPS	The rate of vector delete operations (requests per second).
   rate-limited failed request QPS	RateLimitFailQPS	The rate of requests rejected due to rate limiting (requests per second).
   Slow query count per two minutes	milvus_slow_query	The number of slow queries that take longer than 5 seconds to execute within a two-minute period.
   query/search failure rate	milvus_search_query_failure_rate	The failure rate of query or search requests, as a percentage.
   Insert/delete/upsert operation failure rate	milvus_mutation_failure_rate	The failure rate of insert, delete, or upsert operations, as a percentage.
   99th percentile latency for queries/searches per two minutes	milvus_search_query_latency	The 99th percentile (P99) latency for query or search operations over a two-minute period, measured in milliseconds.
   99th percentile latency for insert/delete/upsert operations per two minutes	milvus_mutation_latency	The 99th percentile (P99) latency for insert, delete, or upsert operations over a two-minute period, measured in milliseconds.
   Average latency for insert/delete/upsert operations	milvus_mutation_latency_average	The average latency of insert, delete, or upsert operations, in milliseconds.
   Average latency for query/search operations	milvus_search_query_latency_average	The average latency of query or search operations, in milliseconds.
   Flush count	FlushCount	The number of flush operations performed on the cluster per minute.
   storage size	Storage	The current storage usage, in GiB.
   Number of loaded entities	milvus_rootcoord_loaded_entity_num	The number of entities loaded into memory.
   total number of collections	milvus_rootcoord_collection_num	The total number of collections in the cluster.
   total number of entities	milvus_rootcoord_entity_num	The total number of entities in the cluster.

Manage alert rules

On the Alert Settings page, click Alert Rule Settings to go to the Alert Rules page of CloudMonitor. On this page, you can modify, delete, or disable existing alert rules. For more information, see Manage alert rules.