Use service discovery selectors to improve the efficiency of pushing ASM configurations - Alibaba Cloud Service Mesh

When you encounter issues such as long configuration push time, overloaded control plane, or inaccurate resource configurations, you can configure service discovery selectors to improve the efficiency of pushing service configurations to sidecar proxies on the data plane from the control plane. You can configure service discovery selectors to ensure that the control plane discovers and processes only services in a specified namespace. This reduces invalid communication, accelerates the configuration synchronization process, ensures stable running of Service Mesh (ASM), and improves O&M efficiency.

Prerequisites

A Container Service for Kubernetes (ACK) cluster is added to your ASM instance whose version is 1.10.5.32 or later. For more information, see The cluster is added to the ASM instance.,
A kubectl client is connected to the cluster.

Background information

Note

In this topic, sidecar proxy configurations refer to the ASM configurations that a sidecar proxy receives from the control plane.

By default, sidecar proxies on the data plane store the configurations of services in all the namespaces of a cluster, including the namespaces whose workloads are not injected with sidecar proxies. The control plane monitors services in all the namespaces and pushes service changes to all the sidecar proxies.

You can configure label selectors based on the labels of the namespaces in the cluster. The label selectors ensure that the control plane discovers and processes only services in the selected namespaces. This way, sidecar proxies store only the configurations of services in the selected namespaces, because the control plane does not push the configurations of services in unselected namespaces to sidecar proxies.

The label selectors support the following matching rules:

Exact match: You can specify a label name and a label value. A namespace can be selected only when its label name and value match the ones you specify.
Match by expression: You can specify a label name, an expression operator, and some label values to select data-plane namespaces whose labels meet the specified conditions. The following items describe the meanings of operators:
- In: A namespace on the data plane can be selected only when it has a label whose name matches the specified name and the corresponding label value is among the specified values.
- NotIn: A namespace on the data plane can be selected only when it has a label whose name matches the specified name, but the corresponding label value is not among the specified values.
- Exists: A namespace on the data plane can be selected when it has a label whose name matches the specified name. No requirements are imposed on the corresponding label value.
- DoesNotExist: A namespace on the data plane can be selected only when none of its labels matches the specified label name. No requirements are imposed on label values.

Step 1: Create two namespaces in ASM

Create two namespaces, ns-in-mesh and ns-not-in-mesh. For more information, see Create a namespace.
Enable automatic sidecar proxy injection for the ns-in-mesh namespace. For more information, see Enable automatic sidecar proxy injection.

Step 2: Deploy a sample application in the two namespaces

Run the following command to add the asm-discovery=enabled label to the ns-in-mesh namespace:
```
kubectl label namespace ns-in-mesh asm-discovery=enabled
```

Create a file named httpbin.yaml and copy the following content to the file:

Show the httpbin.yaml file

apiVersion: v1
kind: ServiceAccount
metadata:
  name: httpbin
---
apiVersion: v1
kind: Service
metadata:
  name: httpbin
  labels:
    app: httpbin
    service: httpbin
spec:
  ports:
  - name: http
    port: 8000
    targetPort: 80
  selector:
    app: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpbin
      version: v1
  template:
    metadata:
      labels:
        app: httpbin
        version: v1
    spec:
      serviceAccountName: httpbin
      containers:
      - image: docker.io/kennethreitz/httpbin
        imagePullPolicy: IfNotPresent
        name: httpbin
        ports:
        - containerPort: 80

Run the following commands to create the HTTPBin application in the ns-in-mesh and ns-not-in-mesh namespaces:
```
kubectl apply -f httpbin.yaml -n ns-in-mesh
kubectl apply -f httpbin.yaml -n ns-not-in-mesh
```

Step 3: Check whether the control plane pushes the configurations to sidecar proxies

View configurations in sidecar proxies.
1. Run the following command to obtain the name of the pod where the HTTPBin application resides in the ns-in-mesh namespace:
```
kubectl get pods -n ns-in-mesh
```
  Expected output:
```
NAME                       READY   STATUS    RESTARTS   AGE
httpbin-6fcb98998c-46qhr   2/2     Running   0          22m
```
2. Run the following command to download configurations in sidecar proxies.
  Replace httpbin-6fcb98998c-46qhr in the command with the name of the pod where the HTTPBin application resides that you obtained in the previous substep.
```
kubectl exec -it httpbin-6fcb98998c-46qhr -c istio-proxy -n ns-in-mesh -- curl -s localhost:15000/config_dump > config_dump.json
```
3. Open the downloaded config_dump.json file and search for httpbin.ns-not-in-mesh.
  If httpbin.ns-not-in-mesh is found, sidecar proxies store service configurations in the ns-not-in-mesh namespace even though automatic sidecar proxy injection is not enabled for the namespace.

View control-plane logs.

Enable control-plane log collection.
- For more information about control-plane log collection for an ASM instance whose version is earlier than 1.17.2.35, see Enable control-plane log collection and log-based alerting in an ASM instance of a version earlier than 1.17.2.35.
- For more information about control-plane log collection for an ASM instance whose version is 1.17.2.35 or later, see Enable control-plane log collection and log-based alerting in an ASM instance of version 1.17.2.35 or later.

Deploy the sleep application in the ns-not-in-mesh namespace of the ACK cluster.

Create a file named sleep.yaml and copy the following content to the file:

Show the sleep.yaml file

##################################################################################################
# Sleep service
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: sleep
  labels:
    app: sleep
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      containers:
      - name: sleep
        image: pstauffer/curl
        command: ["/bin/sleep", "3650d"]
        imagePullPolicy: IfNotPresent
---

Run the following command to deploy the sleep application in the ns-not-in-mesh namespace:
```
kubectl apply -f sleep.yaml -n ns-not-in-mesh
```

View logs.

For ASM instances whose versions are earlier than 1.17.2.35

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.
On the details page of the ASM instance, choose ASM Instance > Base Information in the left-side navigation pane.

On the page that appears, click View log next to Control-plane log collection.

In the upper-right corner of the project page that appears, set Time Range to 5 Minutes to narrow down the scope of the logs that are displayed. On the Raw Logs tab, you can see logs about the creation of the sleep application. The logs indicate that the control plane pushes service configurations in the ns-not-in-mesh namespace even though the namespace does not have sidecar proxy injection enabled.

{"content":"2024-01-04T10:27:00.056248Z\tinfo\tads\tPush debounce stable[16] 1 for config ServiceEntry/ns-not-in-mesh/sleep.ns-not-in-mesh.svc.cluster.local: 100.183738ms since last change, 100.183497ms since last push, full=true","_time_":"2024-01-04T18:27:00.056310074+08:00","_source_":"stdout","_container_name_":"discovery","__pack_meta__":"1|MTcwNDM2MjU0MjUyMjE4OTYxMA==|10|4","__topic__":"asm_istiod_discovery","__source__":"log_service","__time__":"1704364020"}
{"content":"2024-01-04T10:26:59.956023Z\tinfo\tads\tFull push, new service ns-not-in-mesh/sleep.ns-not-in-mesh.svc.cluster.local","_time_":"2024-01-04T18:26:59.956087922+08:00","_source_":"stdout","_container_name_":"discovery","__pack_meta__":"1|MTcwNDM2MjU0MjUyMjE4OTYxMA==|10|3","__topic__":"asm_istiod_discovery","__source__":"log_service","__time__":"1704364020"}

For ASM instances whose versions are 1.17.2.35 or later

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

On the Log Center page, click the Control-Plane Logs tab and set Time Range to 5 Minutes to narrow down the scope of the logs that are displayed.

On the Raw Logs tab, you can see logs about the creation of the sleep application. The logs indicate that the control plane pushes service configurations in the ns-not-in-mesh namespace even though the namespace does not have sidecar proxy injection enabled.

{"content":"2024-01-04T10:33:21.180016Z\tinfo\tads\tPush debounce stable[28] 2 for config Address//c847e048f73054cb192835f8a60ea5219//Pod/ns-not-in-mesh/sleep-fc5cdb9c5-pkfvr and 1 more configs: 100.488198ms since last change, 121.179298ms since last push, full=true","_time_":"2024-01-04T18:33:21.180104591+08:00","_source_":"stdout","_container_name_":"discovery","__pack_meta__":"0|MTcwNDM1NjA0OTcxOTcxNzAyMw==|11|9","__topic__":"asm_istiod_discovery","__source__":"log_service","__time__":"1704364401"}
{"content":"2024-01-04T10:33:21.079470Z\tinfo\tmodel\tFull push, new service ns-not-in-mesh/sleep.ns-not-in-mesh.svc.cluster.local","_time_":"2024-01-04T18:33:21.07954976+08:00","_source_":"stdout","_container_name_":"discovery","__pack_meta__":"0|MTcwNDM1NjA0OTcxOTcxNzAyMw==|11|8","__topic__":"asm_istiod_discovery","__source__":"log_service","__time__":"1704364401"}

Step 4: Configure a service discovery selector in ASM

You can configure a service discovery selector to make the control plane push only the service configurations in the ns-in-mesh namespace with the asm-discovery label to sidecar proxies. You can select the desired namespace or edit the label selector to configure a service discovery selector.

Method 1: Select the desired namespace

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose ASM Instance > Service Discovery Selectors.
On the Service Discovery Selectors page, set Mesh Discovery Mode to Automatically Discover Services in the Selected Namespace of a Kubernetes Cluster on the Data Plane.
On the Select Namespaces tab, select the desired cluster. In the list below, find all namespaces except ns-in-mesh. Click unselect next to all namespaces except ns-in-mesh. In the lower part of the page, click OK. In the Submit message, click OK.
Check whether the configuration is successful.
1. On the details page of the ASM instance, choose ASM Instance > Base Information in the left-side navigation pane.
2. On the page that appears, check Status of the ASM instance.
  If Status is Running, the configuration is successful.

Method 2: Edit the label selector

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose ASM Instance > Service Discovery Selectors.
On the Service Discovery Selectors page, set Mesh Discovery Mode to Automatically Discover Services in the Selected Namespace of a Kubernetes Cluster on the Data Plane and click the Edit Discovery Selectors Directly.
Set Key to asm-discovery and Operator to Exists. In the lower part of the page, click OK. In the Submit message, click OK.
Check whether the configuration is successful.
1. On the details page of the ASM instance, choose ASM Instance > Base Information in the left-side navigation pane.
2. On the page that appears, check Status of the ASM instance.
  If Status is Running, the configuration is successful.

Step 5: Verify that the service discovery selector takes effect

View configurations in sidecar proxies.
1. Run the following command to download configurations in sidecar proxies:
  Replace httpbin-6fcb98998c-46qhr in the command with the name of the pod where the HTTPBin application resides.
```
kubectl exec -it httpbin-6fcb98998c-46qhr -c istio-proxy -n ns-in-mesh -- curl -s localhost:15000/config_dump > config_dump.json
```
2. Open the downloaded config_dump.json file and search for httpbin.ns-not-in-mesh.
  If httpbin.ns-not-in-mesh is not found, sidecar proxies do not store the service configurations in the ns-not-in-mesh namespace.
View control-plane logs.
1. Run the following command to delete the sleep application from the ns-not-in-mesh namespace of the ACK cluster:
```
kubectl delete -f sleep.yaml -n ns-not-in-mesh
```
2. View control-plane logs.
  For ASM instances whose versions are earlier than 1.17.2.35
  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
  2. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.
  3. On the details page of the ASM instance, choose ASM Instance > Base Information in the left-side navigation pane.
  4. On the page that appears, click View log next to Control-plane log collection.
    In the upper-right corner of the project page that appears, set Time Range to 15 Minutes to narrow down the scope of the logs that are displayed. If no logs about the deleted sleep application appear, the service discovery selector takes effect. The control plane does not push service configurations to sidecar proxies when service changes occur in namespaces that are not selected.
  For ASM instances whose versions are 1.17.2.35 or later
  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.
  3. On the Log Center page, click the Control-Plane Logs tab and set Time Range to 15 Minutes to narrow down the scope of the logs that are displayed.
    If no logs about the deleted sleep application appear, the service discovery selector takes effect. The control plane does not push service configurations to sidecar proxies when service changes occur in namespaces that are not selected.

Related operation

Pods with specified labels are not selected by service discovery selector

ASM can discover all services and pods in a Kubernetes cluster on the data plane by default. The control plane of ASM cannot discover the pod outside the cluster, and no request will be sent from the sidecar proxy to the pod. With ASM of version 1.20 or later, you can configure the label selector to exclude the pods with specified labels from ASM service. This helps quickly transfer all traffic received by a specific pod.

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose ASM Instance > Service Discovery Selectors.
Click Show Advanced Settings, and click Logs with specific labels are not selected. Enter Key and Value to exclude the selected pods from the ASM service. In addition, you can also use topology.kubernetes.io/region to exclude pods in a specified region or use topology.kubernetes.io/zone to specify a pod that resides in a specific zone.
In the lower part of the page, click Ok. In the dialog box that appears, click Ok.