This topic describes how to configure recording rules and view the details of a recording rule metric.

Background information

Recording rules can be used for custom development of metric data. Some queries may require a large number of computations on the queried server, which increases workloads on the server. You can configure recording rules to advance the computing process to the server to which results are written. This reduces the resource usage on the queried server. In scenarios in which large-scale clusters are used and business is complex, recording rules can reduce the complexity of PromQL expressions. This improves the query performance and simplifies user configurations.

Note You can configure recording rules in Managed Service for Prometheus in the same manner as you configure recording rules in open source Prometheus. Recording rules exist in a rule group. Each rule group can contain multiple rules. The names of recording rules must be valid metric names. Rules within a group are run sequentially at a regular interval. The metrics are stored in a remote database based on the new rule name.

Configure recording rules

  1. Log on to the ARMS console.
  2. In the left-side navigation pane, choose Prometheus Service > Prometheus Instances.
  3. In the upper-left corner of the page, select the region where the Prometheus instance resides. Then, click the name of the Prometheus instance.
  4. In the left-side navigation pane, click Settings. Click the Settings tab. Then, click Edit RecordingRule.yaml.
  5. In the dialog box that appears, delete the default configurations, enter the configurations of the recording rules, and then click Save.
    Note You need to configure only one RecordingRule.yaml file for a cluster. The names of rule groups must be unique.
    bc

    The following section provides an example of a rule group:

    groups:
      - name: apiserver_request_total
        interval: 60s
        rules:
          - record: job_instance_mode:apiserver_request_total:avg_rate5m
            expr: avg by (job, instance, mode) (rate(apiserver_request_total[5m]))
            labels:
              team: operations
          - record: job:apiserver_request_total:sum_rate10m
            expr: sum by (job)(rate(apiserver_request_total[10m]))
            labels:
              team: operations
    ParameterDescription
    groupsThe rule groups. You can configure multiple rule groups in a RecordingRule.yaml file.
    nameThe name of the rule group. The name of the rule group must be unique within a file.
    intervalOptional. The interval at which the rule group is evaluated. The default value is 60s.
    rulesThe recording rules. A rule group can contain multiple recording rules.
    recordThe name of the rule. The names of recording rules must be valid metric names.
    Note Recommended format for recording rule names: level:metric:operations.
    • level indicates the aggregation level and labels of the rule output.
    • metric indicates the metric name.
    • operations is a list of operations that are applied to the metric. The latest operation is displayed first.
    exprThe expression. Managed Service for Prometheus calculates the metrics by using this expression. The expression must be a valid PromQL expression.
    labelsOptional. The labels of the metric.

View the details of a recording rule metric

Method 1: Use the Grafana dashboard

  1. Log on to the on-premises Grafana system as an administrator.
  2. In the left-side navigation pane, click Explore.
  3. On the Explore page, select the Kubernetes cluster name, and then enter the name of the recording rule metric in the text box next to Metrics. The details of the metric are displayed. fg

Method 2: Use the Managed Service for Prometheus console

  1. Log on to the ARMS console.
  2. In the left-side navigation pane, choose Prometheus Service > Prometheus Instances.
  3. In the upper-left corner of the page, select the region where the Prometheus instance resides. Then, click the name of the Prometheus instance.
  4. In the left-side navigation pane, click Service Discovery. Click the Metrics tab. Then, enter the name of a recording rule metric in the search box to view the details of the metric.

Configure remote write for recording rules

Managed Service for Prometheus supports the remote write feature of open source Prometheus.

  1. Log on to the ARMS console.
  2. In the left-side navigation pane, choose Prometheus Service > Prometheus Instances.
  3. In the upper-left corner of the page, select the region where the Prometheus instance for which you want to configure remote write resides. Then, click the name of the Prometheus instance.
  4. In the left-side navigation pane, click Settings. Click the Settings tab. Then, click Edit Prometheus.yaml.
  5. In the Edit Prometheus.yaml dialog box, configure remote write.
    jk
    Note You can set the remote_write parameter in Managed Service for Prometheus the same way as in open source Prometheus. You can specify an array of remote write configurations. In addition, you can specify write relabel configurations.
    The remote write configurations take effect on both the agent and the recording rules. If you need to remotely write only the metric data generated by recording rules to other databases, you can specify write relabel configurations. Example:
    remote_write:
      - url: "https://xxxx/api/v1/prom/write?db=dbname&u=username&p=password"    // The endpoint of the remote write database.
        write_relabel_configs:
          - source_labels: [__name__]
            regex: job_instance_mode:apiserver_request_total:avg_rate5m
            action: keep
    Note In the preceding configuration, only the recording rule metric (job_instance_mode:apiserver_request_total:avg_rate5m) is remotely written to other libraries.

Related operations

To remotely write new metric data generated by recording rules, you must set the URL to a public endpoint. To configure a network whitelist for recording rules, you must add IP addresses based on the region.

 China (Beijing): 101.200.XX.XX
 China (Hangzhou): 118.31.XX.XX
 China (Shanghai): 106.14.XX.XX.XX
 China (Shenzhen): 8.129.XX.XX
 China (Zhangjiakou): 39.103.XX.XX
 China (Qingdao): 139.129.XX.XX
 China (Chengdu): 47.108.XX.XX
 China (Hong Kong): 47.242.XX.XX
 Singapore: 47.241.XX.XX

To obtain the preceding public endpoints, submit a ticket or contact the technical support of Managed Service for Prometheus (DingTalk ID: aliprometheus).