All Products
Search

This Product

    Container Service for Kubernetes:Konfigurasikan Pemantauan Prometheus untuk KServe

    Document Center

    Container Service for Kubernetes:Konfigurasikan Pemantauan Prometheus untuk KServe

    Last Updated:Aug 21, 2026

    KServe menyediakan metrik Prometheus bawaan untuk memantau kinerja dan status layanan model. Topik ini menjelaskan cara mengonfigurasi pemantauan Prometheus untuk KServe.

    Prasyarat

    Langkah 1: Deploy aplikasi KServe

    1. Deploy aplikasi KServe berbasis Scikit-learn:

      arena serve kserve \
    --name=sklearn-iris \
    --image=kube-ai-registry.cn-shanghai.cr.aliyuncs.com/ai-sample/kserve-sklearn-server:v0.12.0 \
    --cpu=1 \
    --memory=200Mi \
    --enable-prometheus=true \
    --metrics-port=8080 \
    "python -m sklearnserver --model_name=sklearn-iris --model_dir=/models --http_port=8080"

      Output yang diharapkan:

      service/sklearn-iris-metric-svc created # Layanan bernama sklearn-iris-metric-svc dibuat.
inferenceservice.serving.kserve.io/sklearn-iris created # Resource KServe InferenceService bernama sklearn-iris dibuat.
servicemonitor.monitoring.coreos.com/sklearn-iris-svcmonitor created # Resource ServiceMonitor dibuat untuk mengintegrasikan dengan sistem pemantauan Prometheus dan mengumpulkan data pemantauan yang diekspos oleh layanan sklearn-iris-metric-svc.
INFO[0004] The Job sklearn-iris has been submitted successfully # Tugas berhasil dikirim ke kluster.
INFO[0004] You can run `arena serve get sklearn-iris --type kserve -n default` to check the job status

      Arena telah mendeploy layanan KServe dengan model scikit-learn dan pemantauan Prometheus.

    2. Tulis input inferensi ke file ./iris-input.json:

      cat <<EOF > "./iris-input.json"
{
  "instances": [
    [6.8,  2.8,  4.8,  1.4],
    [6.0,  3.4,  4.5,  1.6]
  ]
}
EOF

    3. Ambil alamat IP gerbang NGINX Ingress dan hostname InferenceService:

      NGINX_INGRESS_IP=`kubectl -n kube-system get svc nginx-ingress-lb -ojsonpath='{.status.loadBalancer.ingress[0].ip}'`
SERVICE_HOSTNAME=$(kubectl get inferenceservice sklearn-iris -o jsonpath='{.status.url}' | cut -d "/" -f 3)

    4. Gunakan tool uji stres Hey untuk mengirim permintaan berulang dan menghasilkan data pemantauan:

      Catatan

      Lihat Hey.

      hey -z 2m -c 20 -m POST -host $SERVICE_HOSTNAME -H "Content-Type: application/json" -D ./iris-input.json http://${NGINX_INGRESS_IP}:80/v1/models/sklearn-iris:predict

      Output yang diharapkan:

      Output yang diharapkan

      Summary:
  Total:	120.0296 secs
  Slowest:	0.1608 secs
  Fastest:	0.0213 secs
  Average:	0.0275 secs
  Requests/sec:	727.3875
  Total data:	1833468 bytes
  Size/request:	21 bytes
Response time histogram:
  0.021 [1]	|
  0.035 [85717]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.049 [1272]	|■
  0.063 [144]	|
  0.077 [96]	|
  0.091 [44]	|
  0.105 [7]	|
  0.119 [0]	|
  0.133 [0]	|
  0.147 [11]	|
  0.161 [16]	|
Latency distribution:
  10% in 0.0248 secs
  25% in 0.0257 secs
  50% in 0.0270 secs
  75% in 0.0285 secs
  90% in 0.0300 secs
  95% in 0.0315 secs
  99% in 0.0381 secs
Details (average, fastest, slowest):
  DNS+dialup:	0.0000 secs, 0.0213 secs, 0.1608 secs
  DNS-lookup:	0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:	0.0000 secs, 0.0000 secs, 0.0225 secs
  resp wait:	0.0273 secs, 0.0212 secs, 0.1607 secs
  resp read:	0.0001 secs, 0.0000 secs, 0.0558 secs
Status code distribution:
  [200]	87308 responses

      Output tersebut menampilkan metrik kinerja seperti kecepatan pemrosesan, throughput, dan latensi respons.

    5. (Opsional) Verifikasi bahwa metrik aplikasi diekspos dengan benar.

      Kumpulkan metrik pemantauan dari Pod sklearn-iris secara lokal, tanpa login ke Pod atau mengekspos port-nya.

      1. Teruskan port 8080 dari Pod sklearn-iris (ditentukan oleh $POD_NAME) ke host lokal Anda:

        # Dapatkan nama Pod.
POD_NAME=`kubectl get po|grep sklearn-iris |awk -F ' ' '{print $1}'
# Teruskan port 8080 Pod ke host lokal.
kubectl port-forward pod/$POD_NAME 8080:8080

        Output yang diharapkan:

        Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

        Koneksi IPv4 dan IPv6 keduanya diteruskan ke port 8080 Pod.

      2. Buka URL berikut di browser untuk melihat metrik Pod:

        http://localhost:8080/metrics

        Output yang diharapkan:

        Output yang diharapkan

        # HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 10298.0
python_gc_objects_collected_total{generation="1"} 1826.0
python_gc_objects_collected_total{generation="2"} 0.0
# HELP python_gc_objects_uncollectable_total Uncollectable object found during GC
# TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
# HELP python_gc_collections_total Number of times this generation was collected
# TYPE python_gc_collections_total counter
python_gc_collections_total{generation="0"} 660.0
python_gc_collections_total{generation="1"} 60.0
python_gc_collections_total{generation="2"} 5.0
# HELP python_info Python platform information
# TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="9",patchlevel="18",version="3.9.18"} 1.0
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 1.406291968e+09
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 2.73207296e+08
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.71533439115e+09
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 228.18
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 16.0
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1.048576e+06
# HELP request_preprocess_seconds pre-process request latency
# TYPE request_preprocess_seconds histogram
request_preprocess_seconds_bucket{le="0.005",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="0.01",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="0.025",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="0.05",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="0.075",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="0.1",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="0.25",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="0.5",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="0.75",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="1.0",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="2.5",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="5.0",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="7.5",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="10.0",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_bucket{le="+Inf",model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_count{model_name="sklearn-iris"} 259709.0
request_preprocess_seconds_sum{model_name="sklearn-iris"} 1.7146860011853278
# HELP request_preprocess_seconds_created pre-process request latency
# TYPE request_preprocess_seconds_created gauge
request_preprocess_seconds_created{model_name="sklearn-iris"} 1.7153354578475933e+09
# HELP request_postprocess_seconds post-process request latency
# TYPE request_postprocess_seconds histogram
request_postprocess_seconds_bucket{le="0.005",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="0.01",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="0.025",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="0.05",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="0.075",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="0.1",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="0.25",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="0.5",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="0.75",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="1.0",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="2.5",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="5.0",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="7.5",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="10.0",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_bucket{le="+Inf",model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_count{model_name="sklearn-iris"} 259709.0
request_postprocess_seconds_sum{model_name="sklearn-iris"} 1.625360683305189
# HELP request_postprocess_seconds_created post-process request latency
# TYPE request_postprocess_seconds_created gauge
request_postprocess_seconds_created{model_name="sklearn-iris"} 1.7153354578482144e+09
# HELP request_predict_seconds predict request latency
# TYPE request_predict_seconds histogram
request_predict_seconds_bucket{le="0.005",model_name="sklearn-iris"} 259708.0
request_predict_seconds_bucket{le="0.01",model_name="sklearn-iris"} 259708.0
request_predict_seconds_bucket{le="0.025",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="0.05",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="0.075",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="0.1",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="0.25",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="0.5",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="0.75",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="1.0",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="2.5",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="5.0",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="7.5",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="10.0",model_name="sklearn-iris"} 259709.0
request_predict_seconds_bucket{le="+Inf",model_name="sklearn-iris"} 259709.0
request_predict_seconds_count{model_name="sklearn-iris"} 259709.0
request_predict_seconds_sum{model_name="sklearn-iris"} 47.95311741752084
# HELP request_predict_seconds_created predict request latency
# TYPE request_predict_seconds_created gauge
request_predict_seconds_created{model_name="sklearn-iris"} 1.7153354578476949e+09
# HELP request_explain_seconds explain request latency
# TYPE request_explain_seconds histogram

        Output tersebut mengonfirmasi bahwa metrik diekspos dan permintaan diteruskan ke aplikasi dalam Pod.

    Langkah 2: Kueri metrik KServe

    1. Login ke Konsol ARMS.

    2. Pada panel navigasi kiri, klik Integration Management, lalu klik Query Dashboards.

    3. Pada halaman Dashboards, klik dasbor Kubernetes Pod untuk membuka halaman Grafana.

    4. Pada panel navigasi kiri, klik Explore. Masukkan kueri request_predict_seconds_bucket untuk melihat nilai metrik.

      Catatan

      Data mungkin memerlukan waktu hingga 5 menit untuk muncul.

      Graf menampilkan grafik garis metrik request_predict_seconds_bucket dari waktu ke waktu.

    FAQ

    Pertanyaan

    Bagaimana cara memverifikasi bahwa data untuk metrik request_predict_seconds_bucket telah berhasil dikumpulkan?

    Solusi

    1. Login ke Konsol ARMS.

    2. Pada Connected Environments, pada tab Container Service, klik lingkungan kontainer target, lalu klik tab Self-Monitoring.

    3. Pada panel navigasi kiri, klik Targets. Jika ditampilkan default/sklearn-iris-svcmonitor/0 (1/1 up), pengumpulan metrik berhasil.

    Referensi

    Untuk metrik KServe bawaan, lihat KServe Prometheus Metrics.