How do I troubleshoot exceptions in multi-metric alert rules?

Updated at:
Copy as MD

If you create an alert rule based on combined metrics and configure multiple alert rule expressions, exceptions may occur. Use this guide to identify the root cause and apply the right fix.

Troubleshooting

  1. One metric in the rule has no data

    Symptom: The alert rule shows no data, even though some metrics have data.

    Cause: When any single metric in a multi-metric alert rule has no data, the entire rule returns no data.

    Fix: Split the rule. Configure separate alert rules for metrics that have no data and metrics that have data.

  2. A Dimensions field has the same name but different meanings across metrics

    Note

    For more information about the Dimensions parameter of metrics, see Appendix 1: Metrics.

    Symptom: The alert rule shows no data, but individual metrics appear to have data.

    Cause: When multiple metrics share the same field name in their Dimensions parameter but that field represents different things, the system's integrity check fails. The values cannot be matched, so the rule returns no data.

    Example: The disk metric disk_readiops of an Elastic Compute Service (ECS) instance and the network metric networkout_rate both have a Dimensions parameter of [userId, instanceId, device]. However, the device field means disk name for disk_readiops and network device name for networkout_rate. Because the values never match, the alert rule has no data.

    Fix: Configure independent alert rules for disk_readiops and networkout_rate.

  3. Alert notifications multiply after metrics with different Dimensions fields are combined

    Symptom: The alert rule fires more notifications than expected.

    Cause: When metrics in a rule have different Dimensions fields, the system performs a JOIN across all metrics using a Cartesian product. Each possible combination of dimension values becomes a separate data point, which multiplies the number of alerts.

    Note

    A Cartesian product pairs every value from one set with every value from another set. For example, if set A = {1, 2} and set B = {'a', 'b'}, the Cartesian product A × B = {(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b')} — four pairs from two sets of two. In database terms, this is equivalent to a JOIN with no matching condition, where every row from one table is paired with every row from another.

    Example: A rule monitors cpu_total [userId, instanceId] and disk_readiops [userId, instanceId, device] for an ECS instance with multiple disks attached. If cpu_total is in an alert state and disk_readiops is normal, the rule still sends one alert notification per disk.

    Fix: Configure independent alert rules for metrics with different Dimensions fields.

  4. A monitoring object (such as a mount point or instance) no longer exists

    Symptom: The alert rule continues to fire alerts even though the monitored resource no longer exists. Typical examples include disk mount point alerts that persist after the mount point is unmounted or an ECS instance is released. Running df -h on the server confirms the mount point is absent, but the rule still triggers alerts or reports no data.

    Cause: When a monitoring object—such as an ECS disk mount point or an ECS instance—is deleted, unmounted, or renamed, the Dimensions values in the alert rule still reference the now-invalid resource. The rule evaluates against a resource that no longer exists, which causes persistent false alerts or a no-data state.

    Fix:

    1. Verify that the monitored object still exists.

      • For disk mount points: Log on to the ECS instance and run df -h to list all active mount points. If the mount point path specified in the alert rule Dimensions is not listed, the mount point no longer exists.

      • For ECS instances: In the ECS console, confirm that the instance ID specified in the alert rule Dimensions is still in a running state.

    2. If the monitored object no longer exists, modify the alert rule to remove the invalid Dimensions value—such as the stale mount point path or the released instance ID—from the monitoring scope.

FAQ

Q: What does a multi-metric alert rule exception mean, and how do I determine whether it is a configuration error or a real workload issue?

A: A multi-metric alert rule exception typically means the rule expression failed to evaluate, returned no data, or encountered a logical conflict. This does not necessarily indicate a real business issue.

To determine the root cause:

  1. Check individual metric curves: In CloudMonitor, view the monitoring chart for each metric in the rule individually—for example, the CPU usage curve. If a metric shows genuine high utilization or data gaps, the alert reflects a real workload issue.

  2. Check rule configuration if metrics appear normal: If individual metric data looks correct but the alert rule still reports an exception, review the rule logic: verify that Dimensions field names are consistent across all metrics and that the aggregation method is correctly configured.

For step-by-step checks, see the Troubleshooting section of this document.