Alibaba Cloud Service Mesh:Dynamic subset load balancing

Prerequisites

A Container Service for Kubernetes (ACK) cluster is added to an ASM instance of v1.18 or later. For more information, see Add a cluster to an ASM instance.

Feature description

ASM allows you to use destination rules to divide the workloads of a service into different subsets by using tags, use virtual services to declare routing rules, and route traffic to specific subsets. This static grouping and static routing method is relatively simple and can meet the needs in most situations. However, in some scenarios, this method may not be convenient enough. For example, in a scenario where workloads are divided into different subsets by version, the versions may be evolving. This requires O&M personnel to add or remove the corresponding subset configurations in destination rules when later versions are released or earlier versions are phased out. In a scenario where multiple applications and versions are frequently released, it is burdensome for O&M personnel to update subset configurations.

To improve user experience, ASM supports dynamic subset load balancing since v1.18. You can specify a dimension, such as version, to dynamically divide workloads with the same value in the specified dimension into the same dynamic subset. If dynamic subnets are used in the preceding scenario, O&M personnel do not need to manually configure subsets for each new version. When you deploy workloads, ASM automatically classifies workloads into dynamic subsets based on application characteristics. In addition, ASM allows you to configure specific headers in requests to match the subset characteristics and routes the requests to a specified dynamic subset based on the headers carried in the requests.

Step 1: Deploy an application

In this example, a hashicorp/http-echo application is used, and two environments, dev and prod, are deployed to simulate the development environment and production environment respectively. The v1, v2, and v3 versions are deployed in the dev environment, and only the v2 and v3 versions are deployed in the prod environment. Each helloworld application listens on port 5678 and replies with its own environment and version when it receives a request. The helloworld service is deployed to expose port 8000. In addition, the helloworld service is associated with the workloads for which the value of the app tag is helloworld.

Use kubectl to connect to the ACK cluster based on the information in the kubeconfig file. Then, use the following YAML code to deploy the v1, v2, and v3 versions of the helloworld application in the dev environment, the v2 and v3 versions of the helloworld application in the prod environment, the helloworld service, and the sleep application that is used to initiate testing. For more information about how to deploy an application, see Deploy an application in an ASM instance.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld-dev-v1
  labels:
    app: helloworld
    version: v1
    stage: dev 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: helloworld 
      version: v1
      stage: dev
  template:
    metadata:
      labels:
        app: helloworld 
        version: v1
        stage: dev
    spec:
      containers:
      - name: helloworld 
        env:
        - name: PODIP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: STAGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['stage']
        - name: VERSION 
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['version']
        command: ["/http-echo"]
        args:
          - "-text"
          - "Welcome to helloworld stage: $(STAGE), version: $(VERSION), ip: $(PODIP)"
        image: hashicorp/http-echo:0.2.3
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 5678
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld-dev-v2
  labels:
    app: helloworld
    version: v2
    stage: dev 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: helloworld 
      version: v2
      stage: dev
  template:
    metadata:
      labels:
        app: helloworld 
        version: v2
        stage: dev
    spec:
      containers:
      - name: helloworld 
        env:
        - name: PODIP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: STAGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['stage']
        - name: VERSION 
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['version']
        command: ["/http-echo"]
        args:
          - "-text"
          - "Welcome to helloworld stage: $(STAGE), version: $(VERSION), ip: $(PODIP)"
        image: hashicorp/http-echo:0.2.3
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 5678
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld-dev-v3
  labels:
    app: helloworld
    version: v3
    stage: dev 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: helloworld 
      version: v3
      stage: dev
  template:
    metadata:
      labels:
        app: helloworld 
        version: v3
        stage: dev
    spec:
      containers:
      - name: helloworld 
        env:
        - name: PODIP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: STAGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['stage']
        - name: VERSION 
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['version']
        command: ["/http-echo"]
        args:
          - "-text"
          - "Welcome to helloworld stage: $(STAGE), version: $(VERSION), ip: $(PODIP)"
        image: hashicorp/http-echo:0.2.3
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 5678

apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld-prod-v2
  labels:
    app: helloworld
    version: v2
    stage: prod 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: helloworld 
      version: v2
      stage: prod
  template:
    metadata:
      labels:
        app: helloworld 
        version: v2
        stage: prod
    spec:
      containers:
      - name: helloworld 
        env:
        - name: PODIP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: STAGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['stage']
        - name: VERSION 
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['version']
        command: ["/http-echo"]
        args:
          - "-text"
          - "Welcome to helloworld stage: $(STAGE), version: $(VERSION), ip: $(PODIP)"
        image: hashicorp/http-echo:0.2.3
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 5678
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld-prod-v3
  labels:
    app: helloworld
    version: v3
    stage: prod 
spec:
  replicas: 2
  selector:
    matchLabels:
      app: helloworld 
      version: v3
      stage: prod
  template:
    metadata:
      labels:
        app: helloworld 
        version: v3
        stage: prod
    spec:
      containers:
      - name: helloworld 
        env:
        - name: PODIP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: STAGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['stage']
        - name: VERSION 
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['version']
        command: ["/http-echo"]
        args:
          - "-text"
          - "Welcome to helloworld stage: $(STAGE), version: $(VERSION), ip: $(PODIP)"
        image: hashicorp/http-echo:0.2.3
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 5678

apiVersion: v1
kind: Service
metadata:
  name: helloworld
  labels:
    app: helloworld
    service: helloworld
spec:
  ports:
  - port: 8000 
    name: http
    targetPort: 5678
  selector:
    app: helloworld

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      terminationGracePeriodSeconds: 0
      serviceAccountName: sleep
      containers:
      - name: sleep
        image: curlimages/curl
        command: ["/bin/sleep", "infinity"]
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /etc/sleep/tls
          name: secret-volume
      volumes:
      - name: secret-volume
        secret:
          secretName: sleep-secret
          optional: true

Step 2: Access a specified version in a specified environment

1. Create a destination rule and a virtual service.

   1. Use the following content to create a destination rule for the helloworld service. For more information, see Manage destination rules.

      After the helloworld application is deployed, requests that access the helloworld service are allocated to any pod of the helloworld application by the Kubernetes load balancing feature by default. In this example, two environments and multiple versions are deployed for the helloworld application: the v1, v2, and v3 versions in the dev environment and the v2 and v3 versions in the prod environment. To access the specified environment and version, you must configure a destination rule for the helloworld service.

      apiVersion: networking.istio.io/v1beta1
      kind: DestinationRule
      metadata:
        name: helloworld
        namespace: default
      spec:
        host: helloworld.default.svc.cluster.local
        trafficPolicy:
          loadBalancer:
            dynamicSubset:
              subsetSelectors:
                - keys:
                    - stage
                    - version

      The preceding destination rule groups the workloads of the helloworld service by the stage and version tags. In this example, the deployed workloads are divided into the following subsets by this rule.

      Subset	Pod	IP address
      stage = dev version = v1	helloworld-dev-v1-67b6876778-nf7pz	192.168.0.5
      stage = dev version = v2	helloworld-dev-v2-68f65bbc99-v957l	192.168.0.1
      stage = dev version = v3	helloworld-dev-v3-7f6978bc56-hqzgg	192.168.0.252
      stage = prod version = v2	helloworld-prod-v2-b5745b949-p8rc4	192.168.0.103
      stage = prod version = v3	helloworld-prod-v3-6768bf56f8-6bd6h	192.168.0.104
      	helloworld-prod-v3-6768bf56f8-6bd6h	192.168.0.6

   2. Use the following content to create a virtual service for the helloworld service and establish a mapping between requests and dynamic subnets specified by the key values. For more information, see Manage virtual services.

      Expand to view the YAML code of the virtual service

      apiVersion: networking.istio.io/v1beta1
      kind: VirtualService
      metadata:
        name: helloworld
        namespace: default
      spec:
        hosts:
          - helloworld.default.svc.cluster.local
        http:
          - headerToDynamicSubsetKey:
              - defaultValue: v3
                header: x-version
                key: version
              - defaultValue: prod
                header: x-stage
                key: stage
            name: default
            route:
              - destination:
                  host: helloworld.default.svc.cluster.local
                  port:
                    number: 8000
      

      The preceding virtual service specifies the following operations:

      • Maps the header named x-version in a request to the subnet key named version. If a request does not carry the header named x-version, the default value v3 is used.

      • Maps the header named x-stage in a request to the subset key named stage. If a request does not carry the header named x-stage, the default value prod is used.

2. Use kubectl to connect to the ACK cluster based on the information in the kubeconfig file. Then, run the following command to enable the sleep application to access the v1 version of the helloworld application in the dev environment.

   For more information about how to use kubectl to manage a cluster, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

   kubectl exec -it deploy/sleep -c sleep -- curl -H 'x-stage: dev' -H 'x-version: v1' helloworld:8000

   Expected output:

   Welcome to helloworld stage: dev, version: v1, ip: 192.168.0.5

   The output indicates that the requests are routed to the pod whose stage is dev and version is v1 as expected.

Step 3: Configure a fallback policy for a subnet

The v1 version is not deployed in the prod environment. If you attempt to access the v1 version in the prod environment, an error is reported because the subnet does not exist.

kubectl exec -it deploy/sleep -c sleep -- curl -H 'x-stage: prod' -H 'x-version: v1' helloworld:8000
Expected output:
no healthy upstream # An error is reported because the subnet does not exist. 

To avoid access failures when a subnet does not exist, you can configure a fallback policy for dynamic grouping rules. The following section describes the three fallback policies supported by ASM dynamic grouping: NO_FALLBACK, ANY_ENDPOINT, and DEFAULT_SUBSET.

NO_FALLBACK

When this fallback policy applies to dynamic grouping rules, a no healthy upstream error is reported for a request if no subnet is matched.

1. Use the following destination rule to set fallbackPolicy to NO_FALLBACK for the grouping rule specified by stage and version. For more information, see Manage destination rules.

   apiVersion: networking.istio.io/v1beta1
   kind: DestinationRule
   metadata:
     name: helloworld
     namespace: default
   spec:
     host: helloworld.default.svc.cluster.local
     trafficPolicy:
       loadBalancer:
         dynamicSubset:
           subsetSelectors:
             - fallbackPolicy: NO_FALLBACK
               keys:
                 - stage
                 - version

2. Run the following command to access the helloworld application whose stage is prod and version is v1:

   kubectl exec -it deploy/sleep -c sleep -- curl -H 'x-stage: prod' -H 'x-version: v1' helloworld:8000

   Expected output:

   no healthy upstream # An error is reported because the subnet does not exist.

ANY_ENDPOINT

When this fallback policy applies to dynamic grouping rules, requests will be routed to any endpoint of the service if no subnet is matched.

1. Use the following destination rule to set fallbackPolicy to ANY_ENDPOINT for the grouping rule specified by stage and version. For more information, see Manage destination rules.

   apiVersion: networking.istio.io/v1beta1
   kind: DestinationRule
   metadata:
     name: helloworld
     namespace: default
   spec:
     host: helloworld.default.svc.cluster.local
     trafficPolicy:
       loadBalancer:
         dynamicSubset:
           subsetSelectors:
             - fallbackPolicy: ANY_ENDPOINT
               keys:
                 - stage
                 - version

2. Run the following command to access the helloworld application whose stage is prod and version is v1.

   • First access:

     kubectl exec -it deploy/sleep -c sleep -- curl -H 'x-stage: prod' helloworld:8000

     Expected output:

     Welcome to helloworld stage: prod, version: v2, ip: 192.168.0.103 # The first access is routed to the helloworld application of the v2 version in the prod environment.

   • Second access:

     kubectl exec -it deploy/sleep -c sleep -- curl -H 'x-stage: prod' -H 'x-version: v1' helloworld:8000

     Expected output:

     Welcome to helloworld stage: dev, version: v2, ip: 192.168.0.1 # The second access is routed to the helloworld application of the v2 version in the dev environment.

     The output indicates that requests are routed to random pods of the helloworld service.

DEFAULT_SUBSET

When this fallback policy applies to dynamic grouping rules, requests will be routed to the subnet that matches the values of keys specified in the defaultSubset field if no subnet is matched.

1. Use the following destination rule to set fallbackPolicy to DEFAULT_SUBSET for the grouping rule specified by stage and version, and configure keys and values in the defaultSubset field. For more information, see Manage destination rules.

   key	value
   stage	prod
   version	v3

   apiVersion: networking.istio.io/v1beta1
   kind: DestinationRule
   metadata:
     name: helloworld
     namespace: default
   spec:
     host: helloworld.default.svc.cluster.local
     trafficPolicy:
       loadBalancer:
         dynamicSubset:
           defaultSubset:
             stage: prod
             version: v3
           subsetSelectors:
             - fallbackPolicy: DEFAULT_SUBSET
               keys:
                 - stage
                 - version

2. Run the following command to access the subnet whose stage is prod and version is v1.

   • First access:

     kubectl exec -it deploy/sleep -c sleep -- curl -H 'x-stage: prod' -H 'x-version: v1' helloworld:8000

     Expected output:

     Welcome to helloworld stage: prod, version: v3, ip: 192.168.0.6

   • Second access:

     kubectl exec -it deploy/sleep -c sleep -- curl -H 'x-stage: prod' -H 'x-version: v1' helloworld:8000

     Expected output:

     Welcome to helloworld stage: prod, version: v3, ip: 192.168.0.104

     The output is as expected: No subnet is matched. Therefore, a fallback is triggered, and the requests are routed to the subnet whose stage is prod and version is v3.

Step 4: Access a specific pod

If you want to access a specific pod, you can divide pods into independent subnets by setting keys of ASM dynamic grouping to %ip%.

1. Use the following destination rule to create a grouping rule that groups pods by IP address. For more information, see Manage destination rules.

   apiVersion: networking.istio.io/v1beta1
   kind: DestinationRule
   metadata:
     name: helloworld
     namespace: default
   spec:
     host: helloworld.default.svc.cluster.local
     trafficPolicy:
       loadBalancer:
         dynamicSubset:
           defaultSubset:
             stage: prod
             version: v3
           subsetSelectors:
             - keys:
                 - '%ip%'

2. Use the following virtual service to map the x-ip header of a request to the subset specified by %ip%, so that the request can access the pod whose IP address is specified by x-ip. For more information, see Manage virtual services.

   apiVersion: networking.istio.io/v1beta1
   kind: VirtualService
   metadata:
     name: helloworld
     namespace: default
   spec:
     hosts:
       - helloworld.default.svc.cluster.local
     http:
       - headerToDynamicSubsetKey:
           - header: x-ip
             key: '%ip%'
         name: default
         route:
           - destination:
               host: helloworld.default.svc.cluster.local
               port:
                 number: 8000
   

3. Run the following command to access the Pod helloworld-prod-v3-6768bf56f8-6bd6h whose IP address is 192.168.0.6. Perform the access operations for multiple times.

   • First access:

     kubectl exec -it deploy/sleep -c sleep -- curl -H 'x-ip: 192.168.0.6'  helloworld:8000

     Expected output:

     Welcome to helloworld stage: prod, version: v3, ip: 192.168.0.6

   • Second access:

     kubectl exec -it deploy/sleep -c sleep -- curl -H 'x-ip: 192.168.0.6'  helloworld:8000

     Expected output:

     Welcome to helloworld stage: prod, version: v3, ip: 192.168.0.6

   • Third access:

     kubectl exec -it deploy/sleep -c sleep -- curl -H 'x-ip: 192.168.0.6'  helloworld:8000

     Expected output:

     Welcome to helloworld stage: prod, version: v3, ip: 192.168.0.6

     The output indicates that the requests are routed to the pod with the specified IP address 192.168.0.6, which is as expected.

Description of CRD fields

VirtualService

• HTTPRoute

  The headerToDynamicSubsetKey field is added to the HTTPRoute resource for ASM.

  Field	Type	Description
  headerToDynamicSubsetKey	HeaderToMetadataSubsetKey[]	Used to configure the mapping of request headers to dynamic subset keys. Each element in the array maps a header to a key.

• HeaderToMetadataSubsetKey

  HeaderToMetadataSubsetKey specifies the mapping from request headers to dynamic subset keys and the default value to be used if a request does not carry the specified header.

  Field	Type	Description
  header	string	The name of the request header.
  key	string	The name of the dynamic subset key. The part that is enclosed by percent signs (%) defines a built-in attribute of workloads.
  defaultValue	string	This default value is used if a request does not carry the header mapping to the key. If you do not specify the default value and a request does not carry the header mapping to the key, the system considers that the header is missing and does not perform matching from the perspective of header.

DestinationRule

In ASM, the dynamicSubset field is added to the trafficPolicy structure.

• TrafficPolicy

  Field	Type	Description
  dynamicSubset	DynamicSubsetLB	Used to configure dynamic grouping rules.

• DyunamicSubsetLB

  Field	Type	Description
  defaultSubset	map[string]string	Used to configure the default subset. When a request does not match any dynamic subset and the fallback policy is DEFAULT_SUBSET, the subset declared by this field is selected.
  subsetSelectors	SubsetSelector[]	Used to configure dynamic grouping rules. Each element in the array indicates a separate grouping rule.
  fallbackPolicy	DynamicSubsetLB_FallbackPolicy	Specifies the fallback policy used when the dynamic subsets are not matched. If you do not specify this field, the default value is NO_FALLBACK.

• SubsetSelector

  Field	Type	Description
  keys	string[]	The list of grouping dimensions. The values are mapped to workload tags. For example, the value version indicates that the tag named version of workloads is used as the grouping dimension. In addition to tags, built-in attributes of workloads can also be used as dimensions. For more information, see Built-in attributes of workloads.
  fallbackPolicy	DynamicSubsetLB_FallbackPolicy	Specifies the fallback policy used when no dynamic subsets are matched. If this field is not specified, the fallback policy specified in DyunamicSubsetLB is used.

• DynamicSubsetLB_FallbackPolicy

  DynamicSubsetLB_FallbackPolicy specifies the enumerated values of the fallback policy for dynamic subset load balancing. The following table describes the supported values.

  Value	Description
  NO_FALLBACK	No fallback is performed.
  ANY_ENDPOINT	Requests will be routed to any endpoint of the service if no subnet is matched.
  DEFAULT_SUBSET	Requests will be routed to the default subset if no subnet is matched.

Built-in attributes of workloads

References

• You can enable control-plane log collection and log-based alerting to detect and resolve potential risks in a timely manner. For more information, see Enable control-plane log collection and log-based alerting.

• You can install the asmctl diagnostics tool to detect potential configuration issues in ASM. For more information, see Install and use asmctl.

• You can add audit alerts for changes to ASM resources, such as virtual services and destination rules. This way, alerts are sent to alert contacts in a timely manner when important resources change. For more information, see Configure audit alerts for operations on ASM resources.

• You can configure traffic management rules, such as virtual services and destination rules, to isolate a version of an application or an application with specific characteristics into an independent runtime environment, which is known as a lane. In addition, you can configure traffic shifting, which routes traffic to the baseline version or an application with specific characteristics when the destination version or application is unavailable.

• If you want to obtain the lowest service call latency, you can use the zone aware routing feature to allow a client to call a destination service deployed in the same zone first. For more information, see Verify the zone aware routing feature on the topology of an ASM instance.