All Products
Search
Document Center

Managed Service for OpenTelemetry:Configure dynamic thresholds

Last Updated:Mar 11, 2026

Metrics like response time (RT), queries per second (QPS), and CPU utilization naturally fluctuate throughout the day. A static threshold that works during peak hours triggers false alerts at night -- or misses genuine anomalies during off-peak windows. Dynamic thresholds automatically learn each metric's normal range over time and alert only when behavior deviates from the expected pattern.

Dynamic thresholds work best for metrics with stable, recurring trends -- metrics that follow predictable daily or weekly cycles. If a metric has no discernible pattern (for example, a counter that resets at random intervals), use static thresholds instead.

How dynamic thresholds work

Managed Service for OpenTelemetry uses the Prophet algorithm to power dynamic thresholds. Every 24 hours, the system:

  1. Analyzes 7 days of historical data for each monitored metric.

  2. Extracts the metric's trend (long-term direction) and seasonality (recurring patterns).

  3. Generates a predicted data range for the next 24 hours, with upper and lower boundaries based on observed fluctuations.

Because the model retrains daily, the expected range adapts automatically as usage patterns evolve -- no manual threshold updates needed.

In the data preview chart, blue represents actual data points and green represents the allowed data range.

Threshold calculation preview

Example: detecting a nighttime attack

A website normally receives more than 1,000 page views between 10:00 and 18:00. If page views remain above 1,000 between 22:00 and 06:00, the site may be under attack. A static threshold of 1,000 triggers daytime alerts whenever traffic dips below that number, but misses the abnormal nighttime spike entirely. Dynamic thresholds adapt the expected range to each time window, catching anomalies that static values cannot.

Use cases

ScenarioMonitored metricsWhat dynamic thresholds detect
Application performance monitoringResponse time, request rateSudden RT spikes or traffic drops that deviate from normal daily patterns
Server resource optimizationCPU utilization, memory usageSustained resource consumption above expected levels, helping you rebalance before a crash
Connection pool analysisQuery rate, concurrent connectionsThread-level anomalies that signal degraded program performance
Microservice monitoringPer-service resource usage, response timeAnomalies in individual microservices that could cascade through dependencies

Prerequisites

Before you begin, make sure that you have:

  • Application data reported to Managed Service for OpenTelemetry -- see Connection description for setup instructions

Create a dynamic threshold alert rule

  1. Log on to the Managed Service for OpenTelemetry console.

  2. In the left-side navigation pane, choose Alert Management > Alert Rule.

  3. On the Alert Rules page, click Create Alert Rule.

  4. Enter an alert rule name and set Alert Detection Type to Interval Detection.

  5. In the Alert Contact section, configure the application, metric type, and filter condition.

    The available values for Alert Condition and Filter Condition depend on the selected Metric Type. Initial rendering takes approximately 2 to 4 seconds.
    ParameterDescription
    Select ApplicationsThe application to monitor. Only one application can be selected per dynamic threshold rule.
    Metric TypeThe metric to detect. Once selected, the system calculates the upper and lower threshold boundaries and renders a real-time preview in the Alert Condition section. For supported metrics, see Alert rule metrics.
    Filter ConditionNarrows the monitoring scope to specific metric dimensions. Options: Traverse (all dimension values), No Dimension (aggregated sum), =, !=, Contain, Do Not Contain, Match Regular Expression.

    Alert Contact configuration

  6. In the Alert rules section, configure the Alert Condition. In the data preview, blue represents actual data points and green represents the allowed data range.

    ParameterDescription
    Alert trigger modeSet to Single condition.
    Alert ConditionDefine the detection rule with the following elements:
    Last X minutes -- The monitoring window (maximum: 60 minutes).
    Data -- The data type to monitor, such as call count or response time.
    Calculation method -- How the data is aggregated: average, maximum, minimum, or other methods depending on the metric.
    Comparison method -- How the calculated value is evaluated against the dynamic threshold:
    - Outside the range of the dynamic threshold: Alerts when a data point falls outside the upper or lower boundary. Best for metrics where both spikes and drops are concerning, such as request count.
    - Larger than the maximum value of the dynamic threshold: Alerts when a data point exceeds the upper boundary. Best for metrics like error rate, where only increases matter.
    - Lower than the minimum value of the dynamic threshold: Alerts when a data point falls below the lower boundary. Best for metrics like throughput, where only drops indicate a problem.
    Alert level -- The severity: P1, P2, P3, or P4.
    ToleranceControls the width of the expected data range. A higher tolerance widens the range and reduces alert frequency. A lower tolerance narrows the range and increases alert sensitivity.
    Alert Quantity PredictionDisplays the predicted number of alerts for the configured time window. Click the count to see the specific historical data points that would have triggered alerts. Use this feature every time you create or modify a rule to validate your threshold settings before saving. For details, see Alert quantity prediction.
  7. Configure the Alert Notification and Advanced Alert Settings.

    ParameterDescription
    Alert Notification
    Simple Mode- Notification Objects: Select or create recipients. For details, see Notification objects.
    - Notification Period: The time window during which alert notifications are sent.
    - Whether to Resend Notifications: Without an escalation policy, each notification is sent only once before the alert is cleared. To receive repeat notifications, specify a resend interval. Notifications repeat at this interval until the alert clears.
    Standard Mode- Do Not Specify Notification Policy: No notification is sent when alerts trigger. Notifications are sent only when a notification policy's matching rules are met.
    - Specify a notification policy: Application Real-Time Monitoring Service (ARMS) sends notifications based on the selected policy. Select an existing policy or create one. For details, see Create and manage a notification policy.
    Advanced Alert Settings
    No dataHandles data anomalies such as missing data, abnormal composite metrics, or abnormal period-over-period comparison results. When fixable, the system automatically adjusts the alert data to 0 or 1, or suppresses the alert. For details, see Terms.
  8. Click Save.

Alert quantity prediction

The alert quantity prediction feature analyzes historical data to estimate how many alerts a given threshold configuration would generate within a specified time window. Based on metric data from the last 24 hours, ARMS calculates how many times each threshold boundary was exceeded and predicts the expected alert volume going forward. Detailed data for each breach -- including the exact timestamp -- is also available.

Use alert quantity prediction every time you create or modify an alert rule. Review the predicted count and adjust the tolerance or comparison method until the alert volume matches your operational expectations.

What to do next

  • View alert events: In the left-side navigation pane, choose Alert Management > Alert Event History. For details, see View historical alert events.

  • Manage alert notifications: In the left-side navigation pane, choose Alert Management > Alert Sending History. From here, you can claim, clear, or block alerts, or assign a handler. For details, see View historical alerts.