可觀測監控 Prometheus 版支援使用CRD ServiceMonitor的方式來滿足您自訂服務發現的採集需求。通過使用ServiceMonitor，您可以自行定義Pod發現的Namespace範圍以及通過matchLabel來選擇監聽的Service。本文將基於SpringBoot架構示範如何通過ServiceMonitor建立服務發現。
Demo
您可以通過下載Demo工程，同步體驗通過ServiceMonitor建立服務發現的完整過程。
步驟一：建立基礎代碼依賴
建立一個Maven應用，並在pom.xml檔案中添加以下依賴。
<dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>io.micrometer</groupId> <artifactId>micrometer-registry-prometheus</artifactId> <version>1.6.6</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-configuration-processor</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies>在專案的src/resources/applications.properties檔案中添加以下配置。
management.endpoints.web.exposure.include=prometheus啟動專案，通過瀏覽器訪問
http://{host}:{port}/actuator/prometheus。返回的 Prometheus 格式指標資料樣本如下。
# HELP jvm_threads_daemon_threads The current number of live daemon threads # TYPE jvm_threads_daemon_threads gauge jvm_threads_daemon_threads 23.0 # HELP tomcat_sessions_rejected_sessions_total # TYPE tomcat_sessions_rejected_sessions_total counter tomcat_sessions_rejected_sessions_total 0.0 # HELP jvm_memory_committed_bytes The amount of memory in bytes that is committed for the Java virtual machine to use # TYPE jvm_memory_committed_bytes gauge jvm_memory_committed_bytes{area="heap",id="PSSurvivorSpace",} 1.31072E7 jvm_memory_committed_bytes{area="heap",id="PSOldGen",} 1.30023424E8 jvm_memory_committed_bytes{area="heap",id="PSEdenSpace",} 1.56762112E8 jvm_memory_committed_bytes{area="nonheap",id="Metaspace",} 3.670016E7 jvm_memory_committed_bytes{area="nonheap",id="CodeCache",} 7143424.0 jvm_memory_committed_bytes{area="nonheap",id="CompressedClassSpace",} 5242880.0 # HELP jvm_classes_loaded_classes The number of classes that are currently loaded in the Java virtual machine # TYPE jvm_classes_loaded_classes gauge jvm_classes_loaded_classes 6877.0 # HELP jvm_threads_peak_threads The peak live thread count since the Java virtual machine started or peak was reset # TYPE jvm_threads_peak_threads gauge jvm_threads_peak_threads 28.0 # HELP system_cpu_count The number of processors available to the Java virtual machine # TYPE system_cpu_count gauge system_cpu_count 12.0 # HELP tomcat_sessions_expired_sessions_total # TYPE tomcat_sessions_expired_sessions_total counter tomcat_sessions_expired_sessions_total 0.0 # HELP process_files_max_files The maximum file descriptor count # TYPE process_files_max_files gauge process_files_max_files 10240.0 # HELP jvm_buffer_total_capacity_bytes An estimate of the total capacity of the buffers in this pool # TYPE jvm_buffer_total_capacity_bytes gauge jvm_buffer_total_capacity_bytes{id="direct",} 8192.0 jvm_buffer_total_capacity_bytes{id="mapped",} 0.0 # HELP jvm_threads_states_threads The current number of threads having NEW state # TYPE jvm_threads_states_threads gauge jvm_threads_states_threads{state="runnable",} 9.0 jvm_threads_states_threads{state="blocked",} 0.0 jvm_threads_states_threads{state="waiting",} 12.0 jvm_threads_states_threads{state="timed-waiting",} 6.0 jvm_threads_states_threads{state="new",} 0.0 jvm_threads_states_threads{state="terminated",} 0.0 # HELP jvm_buffer_count_buffers An estimate of the number of buffers in the pool # TYPE jvm_buffer_count_buffers gauge jvm_buffer_count_buffers{id="direct",} 1.0 jvm_buffer_count_buffers{id="mapped",} 0.0 # HELP process_files_open_files The open file descriptor count # TYPE process_files_open_files gauge process_files_open_files 93.0 # HELP jvm_gc_memory_promoted_bytes_total Count of positive increases in the size of the old generation memory pool before GC to after GC # TYPE jvm_gc_memory_promoted_bytes_total counter jvm_gc_memory_promoted_bytes_total 5225384.0 # HELP jvm_memory_max_bytes The maximum amount of memory in bytes that can be used for memory management # TYPE jvm_memory_max_bytes gauge jvm_memory_max_bytes{area="heap",id="PSSurvivorSpace",} 1.31072E7 jvm_memory_max_bytes{area="heap",id="PSOldGen",} 2.863661056E9 jvm_memory_max_bytes{area="heap",id="PSEdenSpace",} 1.40509184E9 jvm_memory_max_bytes{area="nonheap",id="Metaspace",} -1.0 jvm_memory_max_bytes{area="nonheap",id="CodeCache",} 2.5165824E8 jvm_memory_max_bytes{area="nonheap",id="CompressedClassSpace",} 1.073741824E9 # HELP jvm_gc_memory_allocated_bytes_total Incremented for an increase in the size of the (young) heap memory pool after one GC to before the next # TYPE jvm_gc_memory_allocated_bytes_total counter jvm_gc_memory_allocated_bytes_total 1.14749824E8 # HELP jvm_classes_unloaded_classes_total The total number of classes unloaded since the Java virtual machine has started execution # TYPE jvm_classes_unloaded_classes_total counter jvm_classes_unloaded_classes_total 10.0 # HELP tomcat_sessions_created_sessions_total # TYPE tomcat_sessions_created_sessions_total counter
步驟二：部署Kubernetes叢集
構建一個鏡像，並將構建鏡像的Dockerfile檔案上傳至鏡像倉庫。有關鏡像的更多資訊，請參見繫結來源代碼託管平台。
參考以下內容建立Deployment。
apiVersion: apps/v1 kind: Deployment metadata: name: micrometer-prometheus namespace: default labels: app: demo-prometheus spec: replicas: 3 selector: matchLabels: app: demo-prometheus template: metadata: labels: app: demo-prometheus spec: containers: - name: micrometer-prometheus image: manjusakalza/micrometer-prometheus:latest ports: - containerPort: 8080參考以下內容建立Service。
apiVersion: v1 kind: Service metadata: name: prometheus-metrics-demo namespace: default labels: micrometer-prometheus-discovery: 'true' spec: selector: app: demo-prometheus ports: - protocol: TCP port: 8080 targetPort: 8080 name: metrics
步驟三：建立ServiceMonitor
將寫好的YAML檔案儲存至本地，並執行
kubectl apply -f {YAML檔案所在的路徑}使YAML檔案生效。
ServiceMonitor的YAML檔案樣本如下：
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: micrometer-demo
namespace: default
spec:
endpoints:
- interval: 15s
path: /actuator/prometheus
port: metrics #注意：這裡port配置的是連接埠名，並非連接埠號碼。
namespaceSelector:
any: true
selector:
matchLabels:
micrometer-prometheus-discovery: 'true'在這段YAML檔案中，各程式碼片段的含義如下：
metadata下的name和namespace將指定ServiceMonitor所需的一些關鍵元資訊。spec的endpoints為服務端點，代表Prometheus所需的採集Metrics的地址。endpoints為一個數組，同時可以建立多個endpoints。每個endpoints包含三個欄位，每個欄位的含義如下：interval：指定Prometheus對當前endpoints採集的周期。單位為秒，在本次樣本中設定為15s。path：指定Prometheus的採集路徑。在本次樣本中，指定為/actuator/prometheus。port：指定採集資料需要通過的連接埠，設定的連接埠為步驟二建立Service時連接埠所設定的name。在本次樣本中，設定為metrics。重要這裡port配置的是連接埠名，並非連接埠號碼。
spec的namespaceSelector為需要發現的Service的範圍。namespaceSelector包含兩個互斥欄位，欄位的含義如下：any：有且僅有一個值true，當該欄位被設定時，將監聽所有符合Selector過濾條件的Service的變動。matchNames：數組值，指定需要監聽的namespace的範圍。例如，只想監聽default和arms-prom兩個命名空間中的Service，那麼matchNames設定如下：namespaceSelector: matchNames: - default - arms-prom
spec的selector用於選擇Service。在本次樣本所使用的Service有micrometer-prometheus-discovery: 'true' Label，所以
selector設定如下：selector: matchLabels: micrometer-prometheus-discovery: 'true'
如需使用 basic auth 功能，請參考如下YAML檔案。
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: cloud-open-api-monitor # 設定 ServiceMonitor 唯一的名字
namespace: default # 設定當前 ServiceMonitor 所在的命名空間
spec:
endpoints:
- interval: 30s
# 指定 Prometheus 對當前 endpoints 採集的周期
port: tcp-8080
# 填寫 Prometheus exporter代碼中暴露的地址
path: /api/actuator/prometheus
basicAuth:
password:
name: basic-auth
key: <password>
username:
name: basic-auth
key: <userName>
scheme: http
namespaceSelector:
any: true
selector:
matchLabels:
# 匹配具有如下標籤的 Service
edas.oam.acname: cloud-open-api如果使用 basic auth 功能時沒有許可權， 需要在叢集裡添加一個有對應許可權的 ClusterRole，然後將此 ClusterRole 通過 ClusterRoleBinding 綁定到 arms-prom 命名空間下名字為 arms-prom-operator 的 ServiceAccount 上，這樣可以讓 Prometheus Agent 具有對應許可權。
ClusterRole YAML 檔案
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: prometheus-agent-role labels: app: prometheus-agent rules: - apiGroups: [""] resources: ["pods", "services", "endpoints", "nodes"] verbs: ["get", "list", "watch"] #根據具體需求調整 - apiGroups: ["monitoring.coreos.com"] # 根據需要調整apiGroups resources: ["*"] verbs: ["get", "list", "watch", "create", "update", "delete"] # 根據需要調整ClusterRoleBinding YAML 檔案
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: prometheus-agent-binding subjects: - kind: ServiceAccount name: arms-prom-operator # 服務賬戶名稱 namespace: arms-prom # 服務賬戶所在的命名空間 roleRef: kind: ClusterRole name: prometheus-agent-role # 上面定義的 ClusterRole 名稱 apiGroup: rbac.authorization.k8s.io
步驟四：驗證ServiceMonitor
通過以下操作，驗證Prometheus是否成功進行服務發現。
登入ARMS控制台，在左側導覽列單擊接入管理。
在已接入環境頁簽，查看容器環境列表，單擊目標容器環境名稱。
單擊自監控頁簽，然後單擊Targets頁簽。
在Targets頁簽，查看是否存在名稱為{namespace}/{serviceMonitorName}/x的Target。
頁面中顯示目標組 default/micrometer-demo/0 (3/3 up)，表示 ServiceMonitor 已被 Prometheus 正確識別，所有目標的 State 均為綠色 UP，驗證通過。
單擊{namespace}/{serviceMonitorName}/x所在行展開Target，然後單擊Endpoint連結。
頁面顯示 Prometheus 格式的 JVM 與 Tomcat 運行時指標資料，表明 ServiceMonitor 已正確配置。
# HELP jvm_buffer_total_capacity_bytes An estimate of the total capacity of the buffers in this pool # TYPE jvm_buffer_total_capacity_bytes gauge jvm_buffer_total_capacity_bytes{id="mapped",} 0.0 jvm_buffer_total_capacity_bytes{id="direct",} 81968.0 # HELP jvm_threads_daemon_threads The current number of live daemon threads # TYPE jvm_threads_daemon_threads gauge jvm_threads_daemon_threads 29.0 # HELP jvm_memory_committed_bytes The amount of memory in bytes that is committed for the Java virtual machine to use # TYPE jvm_memory_committed_bytes gauge jvm_memory_committed_bytes{area="nonheap",id="miscellaneousnon-heapstorage",} 2.3068672E7 jvm_memory_committed_bytes{area="nonheap",id="classstorage",} 2.8471712E7 jvm_memory_committed_bytes{area="nonheap",id="JITcodecache",} 2.68435456E8 jvm_memory_committed_bytes{area="heap",id="tenured-LOA",} 671744.0 jvm_memory_committed_bytes{area="nonheap",id="JITdatacache",} 2097152.0 jvm_memory_committed_bytes{area="heap",id="nursery-survivor",} 1638400.0 jvm_memory_committed_bytes{area="heap",id="nursery-allocate",} 6881280.0 jvm_memory_committed_bytes{area="heap",id="tenured-SOA",} 1.2763136E7 # HELP process_start_time_seconds Start time of the process since the unix epoch. # TYPE process_start_time_seconds gauge process_start_time_seconds 1.622297808811E9 # HELP tomcat_sessions_expired_sessions_total # TYPE tomcat_sessions_expired_sessions_total counter tomcat_sessions_expired_sessions_total 0.0 # HELP jvm_threads_states_threads The current number of threads having NEW state # TYPE jvm_threads_states_threads gauge jvm_threads_states_threads{state="runnable",} 20.0 jvm_threads_states_threads{state="blocked",} 0.0 jvm_threads_states_threads{state="waiting",} 10.0 jvm_threads_states_threads{state="timed-waiting",} 3.0 jvm_threads_states_threads{state="new",} 0.0 jvm_threads_states_threads{state="terminated",} 0.0 # HELP jvm_classes_unloaded_classes_total The total number of classes unloaded since the Java virtual machine has started execution # TYPE jvm_classes_unloaded_classes_total counter jvm_classes_unloaded_classes_total 0.0 # HELP tomcat_sessions_rejected_sessions_total # TYPE tomcat_sessions_rejected_sessions_total counter tomcat_sessions_rejected_sessions_total 0.0 # HELP tomcat_sessions_created_sessions_total # TYPE tomcat_sessions_created_sessions_total counter tomcat_sessions_created_sessions_total 0.0 # HELP jvm_threads_live_threads The current number of live threads including both daemon and non-daemon threads # TYPE jvm_threads_live_threads gauge jvm_threads_live_threads 33.0 # HELP jvm_memory_max_bytes The maximum amount of memory in bytes that can be used for memory management # TYPE jvm_memory_max_bytes gauge