×
Community Blog Application Traffic Replication through a Kubernetes Ingress Controller

Application Traffic Replication through a Kubernetes Ingress Controller

This document introduces how to replicate application traffic between different Kubernetes clusters of Container Service.

Simulation Test

Generally, when a system is reconstructed or new features are released, we need to run stress tests to evaluate the carrying capacity of the new system in advance. Traditionally, we simulate all kinds of online test data in an offline environment to test the new system. However, this method may not effectively simulate the actual access traffic online, particularly various kinds of abnormal traffic in the simulated normal traffic.

In this case, we can replicate the online application traffic to a specified offline environment to run a simulation test on the new system. In another case, if our online system encounters a performance bottleneck, but we cannot quickly locate the problem, we can also use the traffic replication method to replicate the real application traffic to an offline environment to locate the problem.

Let's see how to replicate application traffic between different Kubernetes clusters of Container Service.

1

Deploying Basic Applications

Assume that you have applied for two different Kubernetes clusters on the Container Service console. We name them Kubernetes Product Cluster and Kubernetes Stage Cluster.

  1. Deploy an application in the Kubernetes Product Cluster and expose external service access through Ingress.
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: nginx-deployment
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          # The current configuration is the old-version image.
          - image: registry.cn-hangzhou.aliyuncs.com/xianlu/old-nginx
            imagePullPolicy: Always
            name: nginx
            ports:
            - containerPort: 80
              protocol: TCP
          restartPolicy: Always
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx-service
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: nginx
      type: NodePort
    ---
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: nginx-ingress
    spec:
      rules:
      # This configuration uses the default cluster domain name (or you can use a customized domain name and resolve it through DNS resolution).
      - host: nginx.c37bf6b77bded43669ba2fb67448b4146.cn-hangzhou.alicontainer.com
        http:
          paths:
          - path: /
            backend:
              serviceName: nginx-service
              servicePort: 80

  2. After application deployment, run the following commands to test the access to the application:
      # Check the Ingress configuration of the application.
      kubectl get ing nginx-ingress
    NAME            HOSTS                                                                  ADDRESS         PORTS     AGE
    nginx-ingress   nginx.c37bf6b77bded43669ba2fb67448b4146.cn-hangzhou.alicontainer.com   47.110.199.44   80        8m
     
      # Test the domain name for accessing the application.
      curl http://nginx.c37bf6b77bded43669ba2fb67448b4146.cn-hangzhou.alicontainer.com
    old

  3. Deploy a same application in the Kubernetes Stage Cluster and expose external service access through Ingress.
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: nginx-deployment
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          # The current configuration is the new-version image.
          - image: registry.cn-hangzhou.aliyuncs.com/xianlu/new-nginx
            imagePullPolicy: Always
            name: nginx
            ports:
            - containerPort: 80
              protocol: TCP
          restartPolicy: Always
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx-service
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: nginx
      type: NodePort
    ---
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: nginx-ingress
    spec:
      rules:
      # This configuration uses the default cluster domain name (or you can use a customized domain name and resolve it through DNS resolution).
      - host: nginx.c41eb6ca34a3e49f7aea63b8bc9e8ad98.cn-beijing.alicontainer.com
        http:
          paths:
          - path: /
            backend:
              serviceName: nginx-service
              servicePort: 80

  4. After application deployment, run the following commands to test the access to the application:
     # Check the Ingress configuration of the application.
     kubectl get ing nginx-ingress
    NAME            HOSTS                                                                 ADDRESS        PORTS     AGE
    nginx-ingress   nginx.c41eb6ca34a3e49f7aea63b8bc9e8ad98.cn-beijing.alicontainer.com   39.106.233.1   80        1m
    
     # Test the domain name for accessing the application.
     curl http://nginx.c41eb6ca34a3e49f7aea63b8bc9e8ad98.cn-beijing.alicontainer.com
    new

Configure Traffic Replication

Assume that we hope to replicate all access traffic (100%) of the application in the Kubernetes Product Cluster to the corresponding application in the Kubernetes Stage Cluster. That is, we need to replicate and forward all requests for the domain name nginx.c37bf6b77bded43669ba2fb67448b4146.cn-hangzhou.alicontainer.com to the domain name nginx.c41eb6ca34a3e49f7aea63b8bc9e8ad98.cn-beijing.alicontainer.com.

2

Note: Because Kubernetes Stage Cluster Ingress is used only to receive replicated traffic, it does not need to be configured or modified. We need to configure only Kubernetes Product Cluster Ingress.

  1. Configure the traffic replication percentage and domain name of the application that receives the replicated traffic.

    Run the following command to modify the ConfigMap configuration of the Kubernetes Ingress Controller. Add the https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/">http-snippet configuration option and configure the expected traffic replication percentage and domain name of the target application that receives the replicated traffic.

      ~ kubectl -n kube-system edit cm nginx-configuration

    Configure the following key-value pair:

     http-snippet: |
       split_clients "$date_gmt" $mirror_servers {
          100%    nginx.c41eb6ca34a3e49f7aea63b8bc9e8ad98.cn-beijing.alicontainer.com;
       }
    
    # Configuration description:
    # (1) The value range of the traffic replication percentage is (0, 100]. The total percentage must be less than or equal to 100%.
    # (2) Multiple different target applications can be configured to receive the replicated traffic.

  2. Configure traffic replication.

    Use the https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/">configuration-snippet and https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/">server-snippet annotations to add the application traffic replication configuration to the source Ingress. (The YAML reference is provided as follows.)

    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: nginx-ingress
      annotations:
        nginx.ingress.kubernetes.io/configuration-snippet: |
            mirror /mirror;
        nginx.ingress.kubernetes.io/server-snippet: |
            location = /mirror {
                internal;
                set $shadow_service_name    "nginx-product-service";
                proxy_set_header X-Shadow-Service  $shadow_service_name;
                proxy_pass http://$mirror_servers$request_uri;
            }
    spec:
      rules:
      - host: nginx.c37bf6b77bded43669ba2fb67448b4146.cn-hangzhou.alicontainer.com
        http:
          paths:
          - path: /
            backend:
              serviceName: nginx-service
              servicePort: 80

    Then, the application traffic replication based on a Kubernetes Ingress Controller has been configured.

Test Traffic Replication

We can try to access the domain name of the application in the Kubernetes Product Cluster: nginx.c37bf6b77bded43669ba2fb67448b4146.cn-hangzhou.alicontainer.com. Then, we can see that each time we request access to the domain name of the application in the Kubernetes Product Cluster through Ingress, the same request is replicated and forwarded to the corresponding application in the Kubernetes Stage Cluster.

3

At the same time, because we have configured proxy_set_header X-Shadow-Service $shadow_service_name during the traffic replication configuration, each request received by the target application in the Kubernetes Stage Cluster contains an X-Shadow-Service header to indicate the online application that this traffic image comes from.

To learn more about Alibaba Cloud Container Service for Kubernetes, visit https://www.alibabacloud.com/product/kubernetes

1 1 1
Share on

You may also like

Comments

Raja_KT March 21, 2019 at 2:23 pm

Nice one. We can copy and paste and make necessary changes.

Related Products

  • Dedicated Host

    DDH is a solution for security and regulation implementation and flexible resource deployment. It offers dedicated resources in Alibaba Cloud for industries such as government departments, enterprises, and financial institutions.

    Learn More
  • Elasticsearch

    Alibaba Cloud Elasticsearch is based on the open-source Elasticsearch engine and provides commercial X-Pack plug-ins. Designed for scenarios such as search and analytics, Alibaba Cloud Elasticsearch features enterprise-level access control, security monitoring, and automatic updates.

    Learn More
  • Managed Security Service

    Identify vulnerabilities and improve security management of Alibaba Cloud WAF and Anti-DDoS and with a fully managed security service

    Learn More
  • DirectMail

    A simple and efficient email service to help you send transactional notifications and batch email

    Learn More