You can view the log consumption status and set alerts in the Log Service console.

Prerequisites

The Important Logs feature is enabled. For more information, see Enable, modify, and disable the operations log feature.

Background information

A consumer group consists of multiple consumers. Each consumer consumes data from one or more shards of a Logstore.

The data model of shards can be regarded as a queue. New data written to shards is added to the end of the queue. Each log entry in the queue has a write timestamp. The following figure shows the data model of shards.
Figure 1. Data model of shards
Data model of shards
Basic terms in collaborative consumption latency alerts:
  • Consumption process: the process in which a consumer reads data from the start of the queue in sequence.
  • Consumption progress: the write timestamp of the data that a consumer reads.
  • Consumption latency: the difference between the current consumption progress and the latest data write timestamp in the queue. Unit: seconds.

The consumption latency of a consumer group is calculated based on the maximum consumption latency among all shards from which the consumer group consumes data. If the consumption latency exceeds the predefined threshold, an alert is triggered.

Procedure

  1. Log on to the Log Service console. On the page that appears, click the target project name.
    The target project is the project that stores service logs.
  2. In the left-side navigation pane, click Dashboard.
  3. Click the Consumer group monitoring dashboard.
    View the consumer groups, the consumption status of the groups, consumed data in shards, consumption latency, and other information on the dashboard.
  4. Optional: Choose Alerts > Create in the upper-right corner of the page.
  5. Optional: Set an alert rule.
    Select the chart named Top 10 consumer group latency and configure an alert that is triggered when the latency exceeds 100 seconds. For more information about how to configure an alert, see Configure an alert.