Alibaba Cloud Service Mesh:Use Mixerless Telemetry to observe ASM instances

Update the ConfigMap

1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

2. On the Clusters page, find your cluster and click its name or click Details in the Actions column.

3. In the left-side navigation pane, choose Configurations > ConfigMaps.

4. On the ConfigMap page, select istio-system from the Namespace drop-down list. Find the ConfigMap named prometheus and click Edit in the Actions column.

5. In the Edit panel, paste the scrape configuration into the Value field and click OK. Get the scrape configuration from GitHub.

Restart Prometheus to apply changes

1. In the left-side navigation pane, choose Workloads > Pods.

2. On the Pods page, find the pod whose name contains prometheus. Choose More > Delete in the Actions column.

3. In the Note dialog box, click OK. Kubernetes automatically recreates the pod with the updated configuration.

Verify the scrape configuration

Run the following command to confirm that the expected scrape jobs appear:

kubectl --kubeconfig <kubeconfig-path> get cm prometheus -n istio-system -o jsonpath='{.data.prometheus\.yml}' | grep job_name

Expected output:

- job_name: 'istio-mesh'
- job_name: 'envoy-stats'
- job_name: 'istio-policy'
- job_name: 'istio-telemetry'
- job_name: 'pilot'
- job_name: 'sidecar-injector'
- job_name: prometheus
  job_name: kubernetes-apiservers
  job_name: kubernetes-nodes
  job_name: kubernetes-nodes-cadvisor
- job_name: kubernetes-service-endpoints
- job_name: kubernetes-service-endpoints-slow
  job_name: prometheus-pushgateway
- job_name: kubernetes-services
- job_name: kubernetes-pods
- job_name: kubernetes-pods-slow

Deploy the podinfo application

1. Create a file named podinfo.yaml with the following content:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: podinfo
   spec:
     minReadySeconds: 3
     revisionHistoryLimit: 5
     progressDeadlineSeconds: 60
     strategy:
       rollingUpdate:
         maxUnavailable: 0
       type: RollingUpdate
     selector:
       matchLabels:
         app: podinfo
     template:
       metadata:
         annotations:
           prometheus.io/scrape: "true"
           prometheus.io/port: "9797"
         labels:
           app: podinfo
       spec:
         containers:
         - name: podinfod
           image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/podinfo:6.7.1
           imagePullPolicy: IfNotPresent
           ports:
           - name: http
             containerPort: 9898
             protocol: TCP
           - name: http-metrics
             containerPort: 9797
             protocol: TCP
           - name: grpc
             containerPort: 9999
             protocol: TCP
           command:
           - ./podinfo
           - --port=9898
           - --port-metrics=9797
           - --grpc-port=9999
           - --grpc-service-name=podinfo
           - --level=info
           - --random-delay=false
           - --random-error=false
           env:
           - name: PODINFO_UI_COLOR
             value: "#34577c"
           livenessProbe:
             exec:
               command:
               - podcli
               - check
               - http
               - localhost:9898/healthz
             initialDelaySeconds: 5
             timeoutSeconds: 5
           readinessProbe:
             exec:
               command:
               - podcli
               - check
               - http
               - localhost:9898/readyz
             initialDelaySeconds: 5
             timeoutSeconds: 5
           resources:
             limits:
               cpu: 2000m
               memory: 512Mi
             requests:
               cpu: 100m
               memory: 64Mi
           volumeMounts:
             - name: data
               mountPath: /data
         volumes:
           - name: data
             emptyDir: {}
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: podinfo
   spec:
     type: ClusterIP
     selector:
       app: podinfo
     ports:
       - name: http
         port: 9898
         protocol: TCP
         targetPort: http
       - port: 9999
         targetPort: grpc
         protocol: TCP
         name: grpc

2. Deploy the application:

   kubectl apply -f podinfo.yaml

Generate traffic

Send 10 HTTP requests to the podinfo service to generate metric data:

podinfo_pod=$(kubectl get po -n test -l app=podinfo -o jsonpath='{.items..metadata.name}')
for i in {1..10}; do
   kubectl --kubeconfig "$USER_CONFIG" exec $podinfo_pod -c podinfod -n test -- curl -s podinfo:9898/version
  echo
done

Check Envoy metrics

After sending traffic, verify that the Envoy sidecar emits Istio metrics.

Check istio_requests_total:

kubectl --kubeconfig <kubeconfig-path> exec $podinfo_pod -n test -c istio-proxy -- curl -s localhost:15090/stats/prometheus | grep istio_requests_total

Expected output (abbreviated):

# TYPE istio_requests_total counter
istio_requests_total{response_code="200",reporter="destination",source_workload="podinfo",...,connection_security_policy="mutual_tls",...} 10
istio_requests_total{response_code="200",reporter="source",source_workload="podinfo",...,connection_security_policy="unknown",...} 10

Both the destination reporter (server-side proxy) and source reporter (client-side proxy) record 10 requests, matching the number of curl requests sent.

Check istio_request_duration_milliseconds:

kubectl --kubeconfig <kubeconfig-path> exec $podinfo_pod -n test -c istio-proxy -- curl -s localhost:15090/stats/prometheus | grep istio_request_duration

Expected output (abbreviated):

# TYPE istio_request_duration_milliseconds histogram
istio_request_duration_milliseconds_bucket{response_code="200",reporter="destination",...,le="0.5"} 10
istio_request_duration_milliseconds_bucket{response_code="200",reporter="destination",...,le="1"} 10
...

All 10 requests fall within the 0.5 ms bucket, indicating sub-millisecond response times.