All Products
Search
Document Center

Alibaba Cloud Service Mesh:Konfigurasikan dasbor Grafana dan aturan alarm untuk pemutusan sirkuit dan perlindungan throttling

Last Updated:Jun 28, 2025

Service Mesh (ASM) mendukung pemutusan sirkuit non-intrusif, throttling, dan pemantauan trafik. Topik ini menjelaskan cara mengonfigurasi dasbor dan aturan alarm untuk pemutusan sirkuit serta throttling.

Pemutusan sirkuit

Pemutusan sirkuit adalah mekanisme perlindungan beban berlebih yang mencegah crash sistem akibat lonjakan trafik dalam waktu singkat. Dalam panggilan timur-barat antar layanan cloud-native, kegagalan satu layanan (seperti respons lambat atau peningkatan tingkat kegagalan) dapat menyebabkan kegagalan berantai di serangkaian layanan dalam jejak.

Anda dapat mengonfigurasi aturan pemutusan sirkuit untuk trafik panggilan timur-barat antar layanan guna menolak permintaan dari layanan hulu ketika tingkat kegagalan atau jumlah timeout respons mencapai ambang batas tertentu. Ini melindungi layanan hulu dan secara efektif mencegah kegagalan memengaruhi seluruh jejak, sehingga menghindari crash sistem.

Setelah mengonfigurasi aturan pemutusan sirkuit, setiap proxy ASM menghitung tingkat kegagalan atau jumlah timeout respons berdasarkan permintaan yang diterimanya. Oleh karena itu, untuk layanan hulu yang sama, waktu terjadinya pemutusan sirkuit pada proxy ASM yang berbeda mungkin sedikit berbeda.

Dasbor

image

Deskripsi untuk setiap panel adalah sebagai berikut:

Panel

Deskripsi

Permintaan

Statistik permintaan mencakup metrik berikut:

  • Jumlah total permintaan (total_requests), dengan nilai spesifik ditampilkan pada panel Permintaan Total.

  • Jumlah permintaan yang memicu pemutusan sirkuit (throttled_requests), dengan nilai spesifik ditampilkan pada panel Permintaan Dithrottle.

  • Jumlah total permintaan sukses (ok_requests), dengan nilai spesifik ditampilkan pada panel Permintaan OK.

Persentase Permintaan OK

Proporsi permintaan sukses terhadap total permintaan.

Persentase Permintaan Dithrottle

Proporsi permintaan yang memicu pemutusan sirkuit terhadap total permintaan.

Anda dapat mengimpor file JSON berikut ke dasbor Grafana untuk membuat dasbor yang menampilkan permintaan yang memicu pemutusan sirkuit di kluster Anda. Anda juga dapat memodifikasi atau menyesuaikan dasbor berdasarkan konten JSON berikut.

Perluas untuk melihat konten JSON

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": 168,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "stacking": {
              "group": "A",
              "mode": "none"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": [
          {
            "matcher": {
              "id": "byName",
              "options": "total_requests"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "blue",
                  "mode": "fixed"
                }
              }
            ]
          },
          {
            "matcher": {
              "id": "byName",
              "options": "unlimited_requests"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "dark-green",
                  "mode": "fixed"
                }
              }
            ]
          },
          {
            "matcher": {
              "id": "byName",
              "options": "limited_requests"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "dark-red",
                  "mode": "fixed"
                }
              }
            ]
          }
        ]
      },
      "gridPos": {
        "h": 8,
        "w": 12,
        "x": 0,
        "y": 0
      },
      "id": 1,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none",
          "targets": []
        }
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "exemplar": false,
          "expr": "sum(rate(istio_requests_total{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[60s])) * 30",
          "format": "time_series",
          "instant": false,
          "legendFormat": "total_requests",
          "range": true,
          "refId": "A"
        },
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "exemplar": false,
          "expr": "sum(rate(istio_requests_total{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[60s])) * 30 - sum(rate(envoy_asm_circuit_breaker_total_broken_requests{namespace=~\"$namespace\",pod_name=~\"$pod\",cluster=~\".*$service.*\"}[60s])) * 30",
          "format": "time_series",
          "hide": false,
          "instant": false,
          "legendFormat": "ok_requests",
          "range": true,
          "refId": "B"
        },
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(rate(envoy_asm_circuit_breaker_total_broken_requests{namespace=~\"$namespace\",pod_name=~\"$pod\",cluster=~\".*$service.*\"}[60s])) * 30",
          "hide": false,
          "instant": false,
          "legendFormat": "throttled_requests",
          "range": true,
          "refId": "C"
        }
      ],
      "title": "Requests",
      "transparent": true,
      "type": "aliyun-timeseries-panel"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 4,
        "x": 0,
        "y": 8
      },
      "id": 2,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(istio_requests_total{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[$__range]))",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests Total",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 4,
        "x": 4,
        "y": 8
      },
      "id": 5,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(istio_requests_total{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[$__range])) - sum(increase(envoy_asm_circuit_breaker_total_broken_requests{namespace=~\"$namespace\",pod_name=~\"$pod\",cluster=~\".*$service.*\"}[$__range]))",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests OK",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 4,
        "x": 8,
        "y": 8
      },
      "id": 4,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_asm_circuit_breaker_total_broken_requests{namespace=~\"$namespace\",pod_name=~\"$pod\",cluster=~\".*$service.*\"}[$__range]))",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests Throttled",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "custom": {
            "neutral": -1
          },
          "mappings": [],
          "max": 100,
          "min": 0,
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "red",
                "value": null
              },
              {
                "color": "green",
                "value": 90
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 7,
        "w": 6,
        "x": 0,
        "y": 16
      },
      "id": 6,
      "options": {
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "showThresholdLabels": false,
        "showThresholdMarkers": true
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "(sum(increase(istio_requests_total{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[$__range])) - sum(increase(envoy_asm_circuit_breaker_total_broken_requests{namespace=~\"$namespace\",pod_name=~\"$pod\",cluster=~\".*$service.*\"}[$__range]))) / sum(increase(istio_requests_total{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[$__range])) * 100",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests OK Percent",
      "type": "gauge"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "custom": {
            "neutral": -1
          },
          "mappings": [],
          "max": 100,
          "min": 0,
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 10
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 7,
        "w": 6,
        "x": 6,
        "y": 16
      },
      "id": 7,
      "options": {
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "showThresholdLabels": false,
        "showThresholdMarkers": true
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_asm_circuit_breaker_total_broken_requests{namespace=~\"$namespace\",pod_name=~\"$pod\",cluster=~\".*$service.*\"}[$__range])) / sum(increase(istio_requests_total{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[$__range])) * 100",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests Throttled Percent",
      "type": "gauge"
    }
  ],
  "refresh": false,
  "schemaVersion": 38,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": [
      {
        "current": {
          "selected": false,
          "text": "datesource",
          "value": "datesource"
        },
        "hide": 0,
        "includeAll": false,
        "label": "datasource",
        "multi": false,
        "name": "DS_PROMETHEUS",
        "options": [],
        "query": "prometheus",
        "queryValue": "datesource",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "type": "datasource"
      },
      {
        "allValue": ".*",
        "current": {
          "selected": false,
          "text": "All",
          "value": "$__all"
        },
        "datasource": {
          "type": "prometheus",
          "uid": "ykIxRIhNk"
        },
        "definition": "query_result(sum(envoy_asm_circuit_breaker_total_broken_requests) by (namespace))",
        "hide": 0,
        "includeAll": true,
        "label": "namespace",
        "multi": false,
        "name": "namespace",
        "options": [],
        "query": {
          "query": "query_result(sum(envoy_asm_circuit_breaker_total_broken_requests) by (namespace))",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "/.*namespace=\"(.*)\".*/",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      },
      {
        "allValue": ".*",
        "current": {
          "selected": false,
          "text": "All",
          "value": "$__all"
        },
        "datasource": {
          "type": "prometheus",
          "uid": "ykIxRIhNk"
        },
        "definition": "query_result(sum(envoy_asm_circuit_breaker_total_broken_requests) by (pod_name))",
        "description": "",
        "hide": 0,
        "includeAll": true,
        "label": "pod",
        "multi": false,
        "name": "pod",
        "options": [],
        "query": {
          "query": "query_result(sum(envoy_asm_circuit_breaker_total_broken_requests) by (pod_name))",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "/.*pod_name=\"(.*)\".*/",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      },
      {
        "allValue": ".*",
        "current": {
          "selected": false,
          "text": "All",
          "value": "$__all"
        },
        "datasource": {
          "type": "prometheus",
          "uid": "ykIxRIhNk"
        },
        "definition": "query_result(sum(envoy_asm_circuit_breaker_total_broken_requests) by (cluster))",
        "hide": 0,
        "includeAll": true,
        "multi": false,
        "name": "service",
        "options": [],
        "query": {
          "query": "query_result(sum(envoy_asm_circuit_breaker_total_broken_requests) by (cluster))",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "/.*_.*_.*_(.*)\".*/",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      }
    ]
  },
  "time": {
    "from": "2025-03-19T07:40:00.388Z",
    "to": "2025-03-19T07:43:09.493Z"
  },
  "timepicker": {},
  "timezone": "browser",
  "title": "Pemutus Sirkuit",
  "uid": "5d5bcd97-33a9-499f-94f6-aea7081b9777",
  "version": 5,
  "weekStart": ""
}

Contoh aturan alarm

Konfigurasi Alarm

Contoh

Deskripsi

Pernyataan PromQL Kustom

(sum by(cluster, namespace) (increase(envoy_asm_circuit_breaker_total_broken_requests[1m]))) > 10

Pernyataan increase menanyakan jumlah permintaan yang ditolak karena pemutusan sirkuit dalam satu menit terakhir. Jumlah permintaan dikelompokkan berdasarkan namespace dan nama layanan yang memicu pemutusan sirkuit. Alarm dilaporkan ketika jumlah permintaan yang ditolak karena pemutusan sirkuit dalam satu menit lebih besar dari 10.

Pesan Alarm

Pemutusan sirkuit tingkat layanan terjadi! Namespace: {{$labels.namespace}}, Layanan yang memicu pemutusan sirkuit: {{$labels.cluster}}. Jumlah permintaan yang ditolak karena pemutusan sirkuit dalam satu menit saat ini: {{$value}}

Informasi alarm menunjukkan namespace layanan yang memicu pemutusan sirkuit, nama layanan, dan jumlah permintaan yang dikirim ke layanan tetapi ditolak karena pemutusan sirkuit dalam satu menit terakhir.

Throttling Global

Throttling global Envoy adalah mekanisme untuk mengontrol laju permintaan dalam instance ASM. Ini diimplementasikan berdasarkan layanan pembatasan laju Envoy, yang memproses lalu lintas seluruh instance ASM secara terpusat dan membatasi laju permintaan sesuai dengan aturan dan kuota yang telah ditentukan.

Throttling global dapat digunakan bersama dengan throttling lokal untuk memberikan tingkat throttling yang berbeda.

Contoh Dasbor

image

Deskripsi untuk setiap panel adalah sebagai berikut:

Panel

Deskripsi

Permintaan

Statistik permintaan mencakup metrik berikut:

  • Jumlah total permintaan (total_requests), dengan nilai spesifik ditampilkan pada panel Permintaan Total.

  • Jumlah permintaan yang memicu throttling (throttled_requests), dengan nilai spesifik ditampilkan pada panel Permintaan Mencapai Batas.

  • Jumlah total permintaan sukses (ok_requests), dengan nilai spesifik ditampilkan pada panel Permintaan OK.

Persentase Permintaan OK

Proporsi permintaan sukses terhadap total permintaan.

Persentase Permintaan Mencapai Batas

Proporsi permintaan yang memicu throttling terhadap total permintaan.

Anda dapat mengimpor file JSON berikut ke dasbor Grafana untuk membuat dasbor yang menampilkan throttling global untuk kluster Anda. Anda juga dapat memodifikasi atau menyesuaikan dasbor berdasarkan konten JSON berikut.

Perluas untuk melihat file JSON

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": 166,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "stacking": {
              "group": "A",
              "mode": "none"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": [
          {
            "matcher": {
              "id": "byName",
              "options": "total_requests"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "blue",
                  "mode": "fixed"
                }
              }
            ]
          },
          {
            "matcher": {
              "id": "byName",
              "options": "unlimited_requests"
            },
            "properties": [
              {
               {
                "id": "color",
                "value": {
                  "fixedColor": "dark-green",
                  "mode": "fixed"
                }
              }
            ]
          },
          {
            "matcher": {
              "id": "byName",
              "options": "limited_requests"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "dark-red",
                  "mode": "fixed"
                }
              }
            ]
          }
        ]
      },
      "gridPos": {
        "h": 8,
        "w": 12,
        "x": 0,
        "y": 0
      },
      "id": 1,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none",
          "targets": []
        }
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "exemplar": false,
          "expr": "sum(rate(envoy_cluster_ratelimit_ok{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[60s])) * 30 + sum(rate(envoy_cluster_ratelimit_over_limit{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[60s])) * 30",
          "format": "time_series",
          "instant": false,
          "legendFormat": "total_requests",
          "range": true,
          "refId": "A"
        },
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "exemplar": false,
          "expr": "sum(rate(envoy_cluster_ratelimit_ok{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[60s])) * 30",
          "format": "time_series",
          "hide": false,
          "instant": false,
          "legendFormat": "unlimited_requests",
          "range": true,
          "refId": "B"
        },
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(rate(envoy_cluster_ratelimit_over_limit{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[60s])) * 30",
          "hide": false,
          "instant": false,
          "legendFormat": "limited_requests",
          "range": true,
          "refId": "C"
        }
      ],
      "title": "Requests",
      "transparent": true,
      "type": "aliyun-timeseries-panel"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 4,
        "x": 0,
        "y": 8
      },
      "id": 2,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_cluster_ratelimit_ok{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range])) + sum(increase(envoy_cluster_ratelimit_over_limit{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range]))",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests Total",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 4,
        "x": 4,
        "y": 8
      },
      "id": 5,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_cluster_ratelimit_ok{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range]))",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests OK",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 4,
        "x": 8,
        "y": 8
      },
      "id": 4,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_cluster_ratelimit_over_limit{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range]))",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests Reached Limits",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "custom": {
            "neutral": -1
          },
          "mappings": [],
          "max": 100,
          "min": 0,
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "red",
                "value": null
              },
              {
                "color": "green",
                "value": 90
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 7,
        "w": 6,
        "x": 0,
        "y": 16
      },
      "id": 6,
      "options": {
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "showThresholdLabels": false,
        "showThresholdMarkers": true
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_cluster_ratelimit_ok{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range])) / (sum(increase(envoy_cluster_ratelimit_ok{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range])) + sum(increase(envoy_cluster_ratelimit_over_limit{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range]))) * 100",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests OK Percent",
      "type": "gauge"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "custom": {
            "neutral": -1
          },
          "mappings": [],
          "max": 100,
          "min": 0,
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 10
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 7,
        "w": 6,
        "x": 6,
        "y": 16
      },
      "id": 7,
      "options": {
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "showThresholdLabels": false,
        "showThresholdMarkers": true
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_cluster_ratelimit_over_limit{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range])) / (sum(increase(envoy_cluster_ratelimit_ok{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range])) + sum(increase(envoy_cluster_ratelimit_over_limit{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range]))) * 100",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests Reached Limits Percent",
      "type": "gauge"
    }
  ],
  "refresh": "",
  "schemaVersion": 38,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": [
      {
        "current": {
          "selected": false,
          "text": "datasource",
          "value": "datasource"
        },
        "hide": 0,
        "includeAll": false,
        "label": "datasource",
        "multi": false,
        "name": "DS_PROMETHEUS",
        "options": [],
        "query": "prometheus",
        "queryValue": "datasource",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "type": "datasource"
      },
      {
        "allValue": ".*",
        "current": {
          "selected": false,
          "text": "All",
          "value": "$__all"
        },
        "datasource": {
          "type": "prometheus",
          "uid": "ykIxRIhNk"
        },
        "definition": "query_result(sum(envoy_cluster_ratelimit_over_limit) by (namespace) or sum(envoy_cluster_ratelimit_ok) by (namespace))",
        "hide": 0,
        "includeAll": true,
        "label": "namespace",
        "multi": false,
        "name": "namespace",
        "options": [],
        "query": {
          "query": "query_result(sum(envoy_cluster_ratelimit_over_limit) by (namespace) or sum(envoy_cluster_ratelimit_ok) by (namespace))",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "/.*namespace=\"(.*)\".*/",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      },
      {
        "allValue": ".*",
        "current": {
          "selected": false,
          "text": "All",
          "value": "$__all"
        },
        "datasource": {
          "type": "prometheus",
          "uid": "ykIxRIhNk"
        },
        "definition": "query_result(sum(envoy_cluster_ratelimit_over_limit) by (pod_name) or sum(envoy_cluster_ratelimit_ok) by (pod_name))",
        "description": "",
        "hide": 0,
        "includeAll": true,
        "label": "pod",
        "multi": false,
        "name": "pod",
        "options": [],
        "query": {
          "query": "query_result(sum(envoy_cluster_ratelimit_over_limit) by (pod_name) or sum(envoy_cluster_ratelimit_ok) by (pod_name))",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "/.*pod_name=\"(.*)\".*/",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      },
      {
        "allValue": ".*",
        "current": {
          "selected": false,
          "text": "All",
          "value": "$__all"
        },
        "datasource": {
          "type": "prometheus",
          "uid": "ykIxRIhNk"
        },
        "definition": "query_result(sum(envoy_cluster_ratelimit_over_limit) by (service_istio_io_canonical_name) or sum(envoy_cluster_ratelimit_ok) by (service_istio_io_canonical_name))",
        "hide": 0,
        "includeAll": true,
        "label": "service",
        "multi": false,
        "name": "service",
        "options": [],
        "query": {
          "query": "query_result(sum(envoy_cluster_ratelimit_over_limit) by (service_istio_io_canonical_name) or sum(envoy_cluster_ratelimit_ok) by (service_istio_io_canonical_name))",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "/.*service_istio_io_canonical_name=\"(.*)\".*/",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      }
    ]
  },
  "time": {
    "from": "now-5m",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "browser",
  "title": "Pembatasan Laju Global",
  "uid": "d8f95bbc-efb6-41e7-8929-25551b22f3d2",
  "version": 25,
  "weekStart": ""
}

Contoh aturan alarm

Konfigurasi Alarm

Contoh

Deskripsi

Pernyataan PromQL Kustom

sum (increase(envoy_cluster_ratelimit_over_limit[1m])) by (namespace, service_istio_io_canonical_name) > 10

Pernyataan increase menanyakan jumlah permintaan yang ditolak karena throttling dalam satu menit terakhir. Jumlah permintaan dikelompokkan berdasarkan namespace dan nama layanan yang memicu throttling. Alarm dilaporkan ketika jumlah permintaan yang ditolak karena throttling dalam satu menit lebih besar dari 10.

Pesan Alarm

Throttling dipicu! Namespace: {{$labels.namespace}}, Layanan yang memicu throttling: {{$labels.service_istio_io_canonical_name}}. Jumlah permintaan yang ditolak karena throttling dalam satu menit saat ini: {{$value}}

Informasi alarm menunjukkan namespace layanan yang memicu throttling, nama layanan, dan jumlah permintaan yang dikirim ke layanan tetapi ditolak karena throttling dalam satu menit terakhir.

Throttling Lokal

Proxy Envoy menggunakan algoritma token bucket untuk mengimplementasikan throttling lokal. Algoritma token bucket adalah metode yang membatasi jumlah permintaan yang dikirim ke layanan berdasarkan sejumlah token dalam sebuah bucket. Token diisi ke bucket dengan laju konstan. Ketika permintaan dikirim ke layanan, satu token diambil dari bucket. Jika bucket kosong, permintaan ditolak.

Throttling lokal dapat digunakan bersama dengan throttling global untuk memberikan tingkat throttling yang berbeda.

Contoh Dasbor

image

Deskripsi untuk setiap panel adalah sebagai berikut:

Panel

Deskripsi

Permintaan

Statistik permintaan mencakup metrik berikut:

  • Jumlah total permintaan (total_requests), dengan nilai spesifik ditampilkan pada panel Permintaan Total.

  • Jumlah permintaan yang memicu throttling (throttled_requests), dengan nilai spesifik ditampilkan pada panel Permintaan Mencapai Batas.

  • Jumlah total permintaan sukses (ok_requests), dengan nilai spesifik ditampilkan pada panel Permintaan OK.

Persentase Permintaan OK

Proporsi permintaan sukses terhadap total permintaan.

Persentase Permintaan Mencapai Batas

Proporsi permintaan yang memicu throttling terhadap total permintaan.

Anda dapat mengimpor file JSON berikut ke dasbor Grafana untuk membuat dasbor yang menampilkan throttling lokal untuk kluster Anda. Anda juga dapat memodifikasi atau menyesuaikan dasbor berdasarkan konten JSON berikut.

Perluas untuk melihat file JSON

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": 167,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "none",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "stacking": {
              "group": "A",
              "mode": "none"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": [
          {
            "matcher": {
              "id": "byName",
              "options": "total_requests"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "blue",
                  "mode": "fixed"
                }
              }
            ]
          },
          {
            "matcher": {
              "id": "byName",
              "options": "unlimited_requests"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "dark-green",
                  "mode": "fixed"
                }
              }
            ]
          },
          {
            "matcher": {
              "id": "byName",
              "options": "limited_requests"
            },
            "properties": [
              {
                "id": "color",
                "value": {
                  "fixedColor": "dark-red",
                  "mode": "fixed"
                }
              }
            ]
          }
        ]
      },
      "gridPos": {
        "h": 8,
        "w": 12,
        "x": 0,
        "y": 0
      },
      "id": 1,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none",
          "targets": []
        }
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "exemplar": false,
          "expr": "sum(rate(envoy_http_local_rate_limiter_http_local_rate_limit_enabled{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[60s])) * 30",
          "format": "time_series",
          "instant": false,
          "legendFormat": "total_requests",
          "range": true,
          "refId": "A"
        },
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "exemplar": false,
          "expr": "sum(rate(envoy_http_local_rate_limiter_http_local_rate_limit_ok{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[60s])) * 30",
          "format": "time_series",
          "hide": false,
          "instant": false,
          "legendFormat": "unlimited_requests",
          "range": true,
          "refId": "B"
        },
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(rate(envoy_http_local_rate_limiter_http_local_rate_limit_enforced{namespace=~\"$namespace\",pod_name=~\"$pod\",service_istio_io_canonical_name=~\"$service\"}[60s])) * 30",
          "hide": false,
          "instant": false,
          "legendFormat": "limited_requests",
          "range": true,
          "refId": "C"
        }
      ],
      "title": "Requests",
      "transparent": true,
      "type": "aliyun-timeseries-panel"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 4,
        "x": 0,
        "y": 8
      },
      "id": 2,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_http_local_rate_limiter_http_local_rate_limit_enabled{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range]))",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests Total",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 4,
        "x": 4,
        "y": 8
      },
      "id": 5,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_http_local_rate_limiter_http_local_rate_limit_ok{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range]))",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests OK",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 4,
        "x": 8,
        "y": 8
      },
      "id": 4,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_http_local_rate_limiter_http_local_rate_limit_enforced{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range]))",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Requests Reached Limits",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "custom": {
            "neutral": -1
          },
          "mappings": [],
          "max": 100,
          "min": 0,
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "red",
                "value": null
              },
              {
                "color": "green",
                "value": 90
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 7,
        "w": 6,
        "x": 0,
        "y": 16
      },
      "id": 6,
      "options": {
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "showThresholdLabels": false,
        "showThresholdMarkers": true
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_http_local_rate_limiter_http_local_rate_limit_ok{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range])) / sum(increase(envoy_http_local_rate_limiter_http_local_rate_limit_enabled{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range])) * 100",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Persentase Permintaan OK",
      "type": "gauge"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "custom": {
            "neutral": -1
          },
          "mappings": [],
          "max": 100,
          "min": 0,
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 10
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 7,
        "w": 6,
        "x": 6,
        "y": 16
      },
      "id": 7,
      "options": {
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "showThresholdLabels": false,
        "showThresholdMarkers": true
      },
      "pluginVersion": "10.0.9",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum(increase(envoy_http_local_rate_limiter_http_local_rate_limit_enforced{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range])) / sum(increase(envoy_http_local_rate_limiter_http_local_rate_limit_enabled{namespace=~\"$namespace\", pod_name=~\"$pod\", service_istio_io_canonical_name=~\"$service\"}[$__range])) * 100",
          "instant": false,
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Persentase Permintaan Mencapai Batas",
      "type": "gauge"
    }
  ],
  "refresh": "",
  "schemaVersion": 38,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": [
      {
        "allValue": ".*",
        "current": {
          "selected": false,
          "text": "datasource",
          "value": "datasource"
        },
        "hide": 0,
        "includeAll": false,
        "label": "datasource",
        "multi": false,
        "name": "DS_PROMETHEUS",
        "options": [],
        "query": "prometheus",
        "queryValue": "datasource",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "type": "datasource"
      },
      {
        "allValue": ".*",
        "current": {
          "selected": false,
          "text": "All",
          "value": "$__all"
        },
        "datasource": {
          "type": "prometheus",
          "uid": "ykIxRIhNk"
        },
        "definition": "query_result(sum(envoy_http_local_rate_limiter_http_local_rate_limit_enabled) by (namespace))",
        "hide": 0,
        "includeAll": true,
        "label": "namespace",
        "multi": false,
        "name": "namespace",
        "options": [],
        "query": {
          "query": "query_result(sum(envoy_http_local_rate_limiter_http_local_rate_limit_enabled) by (namespace))",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "/.*namespace=\"(.*)\".*/",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      },
      {
        "allValue": ".*",
        "current": {
          "selected": false,
          "text": "All",
          "value": "$__all"
        },
        "datasource": {
          "type": "prometheus",
          "uid": "ykIxRIhNk"
        },
        "definition": "query_result(sum(envoy_http_local_rate_limiter_http_local_rate_limit_enabled) by (pod_name))",
        "description": "",
        "hide": 0,
        "includeAll": true,
        "label": "pod",
        "multi": false,
        "name": "pod",
        "options": [],
        "query": {
          "query": "query_result(sum(envoy_http_local_rate_limiter_http_local_rate_limit_enabled) by (pod_name))",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "/.*pod_name=\"(.*)\".*/",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      },
      {
        "allValue": ".*",
        "current": {
          "selected": false,
          "text": "All",
          "value": "$__all"
        },
        "datasource": {
          "type": "prometheus",
          "uid": "ykIxRIhNk"
        },
        "definition": "query_result(sum(envoy_http_local_rate_limiter_http_local_rate_limit_enabled) by (service_istio_io_canonical_name))",
        "hide": 0,
        "includeAll": true,
        "label": "service",
        "multi": false,
        "name": "service",
        "options": [],
        "query": {
          "query": "query_result(sum(envoy_http_local_rate_limiter_http_local_rate_limit_enabled) by (service_istio_io_canonical_name))",
          "refId": "PrometheusVariableQueryEditor-VariableQuery"
        },
        "refresh": 1,
        "regex": "/.*service_istio_io_canonical_name=\"(.*)\".*/",
        "skipUrlSync": false,
        "sort": 0,
        "type": "query"
      }
    ]
  },
  "time": {
    "from": "now-5m",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "browser",
  "title": "Pembatasan Laju Lokal",
  "uid": "859fe8e7-c7dd-46f4-a1c0-c0c7010bcf19",
  "version": 7,
  "weekStart": ""
}

Contoh aturan alarm

Konfigurasi Alarm

Contoh

Deskripsi

Pernyataan PromQL Kustom

sum (increase(envoy_http_local_rate_limiter_http_local_rate_limit_enforced[1m])) by (namespace, service_istio_io_canonical_name) > 10

Pernyataan increase menanyakan jumlah permintaan yang ditolak karena throttling dalam satu menit terakhir. Jumlah permintaan dikelompokkan berdasarkan namespace dan nama layanan yang memicu throttling. Alarm dilaporkan ketika jumlah permintaan yang ditolak karena throttling dalam satu menit lebih besar dari 0.

Pesan Alarm

Throttling dipicu! Namespace: {{$labels.namespace}}, Layanan yang memicu throttling: {{$labels.service_istio_io_canonical_name}}. Jumlah permintaan yang ditolak karena throttling dalam satu menit saat ini: {{$value}}

Informasi alarm menunjukkan namespace layanan yang memicu throttling, nama layanan, dan jumlah permintaan yang dikirim ke layanan tetapi ditolak karena throttling dalam satu menit terakhir.

Operasi Terkait

Impor Dasbor Grafana

  1. Masuk ke Konsol ARMS.

  2. Di bilah navigasi di sebelah kiri, klik Integration Management.

  3. Di tab Integrated Environments, pilih Container Service. Masukkan nama kluster untuk mencari lingkungan, lalu klik nama lingkungan target. Klik Dashboard Directory.

    image

    image

  4. Di sisi kanan tab Dashboards, klik tombol Import.

    image

  5. Tempelkan konten file JSON ke bagian Import via panel json, lalu klik tombol Load. Pertahankan pengaturan default dan klik Import untuk menyelesaikan impor.

    Catatan

    Anda juga dapat mengimpor metrik ke dasbor dengan mengunggah file JSON.