Implement local development and testing by using KT-Connect and ASM - Alibaba Cloud Service Mesh

KT-Connect ("KT" is short for "Kubernetes Toolkit") is a utility tool to help you work with Kubernetes development environment more efficiently. KT-Connect proxy can use the traffic management feature in the Alibaba Cloud Service Mesh (ASM) console to debug local applications and accelerate local development and testing. This topic describes how to implement local development and testing by using KT-Connect and ASM.

Background

KtConnect is an open source tool designed to help developers simplify development processes in Kubernetes clusters. It allows you to deploy proxy pods in Kubernetes clusters to redirect traffic intended for specific services to local applications. This helps enable mutual access between local environments and Kubernetes clusters. The core features of KT-Connect include:

Supports accessing Kubernetes clusters from local applications.
KT-Connect makes all cluster resources accessible from local applications without any mode modification.
Supports local resolution of internal domain names for Kubernetes services.
Directly resolve service names to the IP addresses of the clusters where services reside, providing a true cloud-native experience for local development.
Supports redirecting traffic specified for cluster services to services in local environments.
Route requests to specified service in cluster to a local application, making all local applications can be directly accessed from cluster services.
Supports the collaboration in testing environment between multiple users without any interference.
You can configure routing rules to redirect only specified requests to local applications without affecting environment availability.
Supports Windows/MacOS/Linux development environments.
Multiple operating systems with the same usage method enables developers to access Kubernetes cluster services.

Prerequisites

An ingress gateway is deployed. For more information, see Create an ingress gateway.
An application is deployed in the cluster that is added to the ASM instance. For more information, see Deploy an application in an ACK cluster that is added to an ASM instance.
To install Docker, see the referenced document.

Preparations

This example demonstrates the following scenario:

Deploy a helloworld application v1 in the cluster and configure a traffic policy for the application.
Use Docker to deploy a helloworld application v2 in the local environment. Run the ktctl command to deploy a proxy pod in the cluster to enable connecting to the cluster from the local environment.
Update the routing policy to route traffic to the application v1 in the cluster and the application v2 in the local environment based on request headers.

Install and configure ktctl.
Create a namespace in the cluster.
```
kubectl create ns mesh-demo
```
Enable automatic sidecar injection for the namespace.
```
kubectl label ns mesh-demo istio-injection=enabled
```

Procedure

Step 1: Deploy applications and configure routing policies

Deploy applications in the mesh-demo namespace.

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld
  labels:
    app: helloworld
    version: v1
    stage: online
spec:
  replicas: 1
  selector:
    matchLabels:
      app: helloworld 
      version: v1
      stage: online
  template:
    metadata:
      labels:
        app: helloworld 
        version: v1
        stage: online
    spec:
      containers:
      - name: helloworld 
        env:
        - name: PODIP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: STAGE
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['stage']
        - name: VERSION 
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['version']
        command: ["/http-echo"]
        args:
          - "-text"
          - "Welcome to helloworld stage: $(STAGE), version: $(VERSION), ip: $(PODIP)"
        image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/asm-http-echo:1.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
  name: helloworld
  labels:
    app: helloworld
    service: helloworld
spec:
  ports:
  - port: 8000
    name: http
    targetPort: 5678
  selector:
    app: helloworld
EOF

Deploy a routing policy.

kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: helloworld-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - 'helloworld.mesh.com'
    port:
      name: http
      number: 80
      protocol: HTTP
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: helloworld
spec:
  host: helloworld
  subsets:
  - name: v1
    labels:
      version: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: helloworld
spec:
  gateways:
  - helloworld-gateway
  hosts:
  - helloworld.mesh.com
  http:
  - route:
    - destination:
        host: helloworld
        subset: v1
EOF

Verify whether the policy takes effect.

export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
curl ${INGRESS_HOST} -H 'host: helloworld.mesh.com'

Expected output:

Welcome to helloworld stage: online, version: v1, ip: 172.23.16.246

The output shows that the stage is online and the version is v1, which is as expected.

Step 2: Deploy a local development environment

Deploy a helloworld application v2 using Docker.

docker run -itd --rm \
    --name local-container \
    -p 5678:5678 \
    registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/asm-http-echo:1.0 \
    -text "Welcome to helloworld stage: local, version: v2, ip: 127.0.0.1"

Deploy the KT-Connect proxy in the cluster. In this way, a proxy pod with version=hzall label is deployed in the mesh-demo namespace.

ktctl -n mesh-demo mesh helloworld --expose 5678 --mode manual

Expected output:

4:22PM INF Using cluster context cluster-4KvcBF (kubernetes)
4:22PM INF KtConnect 0.3.7 start at 59150 (darwin amd64)
4:22PM INF Fetching cluster time ...
4:22PM INF Using manual mode
4:22PM INF Successful create config map helloworld-kt-mesh-hzall
4:22PM INF Deploying shadow pod helloworld-kt-mesh-hzall in namespace mesh-demo
4:22PM INF Waiting for pod helloworld-kt-mesh-hzall ...
4:22PM INF Waiting for pod helloworld-kt-mesh-hzall ...
4:22PM INF Pod helloworld-kt-mesh-hzall is ready
4:22PM INF Forwarding pod helloworld-kt-mesh-hzall to local via port 5678
4:22PM INF Port forward local:17982 -> pod helloworld-kt-mesh-hzall:22 established
4:22PM INF Reverse tunnel 0.0.0.0:5678 -> 127.0.0.1:5678 established
4:22PM INF ---------------------------------------------------------
4:22PM INF  Now you can update Istio rule by label 'version=hzall' 
4:22PM INF ---------------------------------------------------------

View the proxy pod.

kubectl get pod -n mesh-demo

Expected output:

NAME                         READY   STATUS    RESTARTS   AGE
helloworld-5cbdxxxxx-xxxxx   2/2     Running   0          116m
helloworld-kt-mesh-hzall     2/2     Running   0          106m

Where, helloworld-kt-mesh-hzall is the deployed proxy pod.

Step 3: Update the routing policy and verify connections

Update the routing policy. Forward requests with the x-env=local request header to the helloworld application v2 in the local environment, and forward all other requests to the helloworld application v2 in the cluster.

kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: helloworld
spec:
  host: helloworld
  subsets:
  - name: v1
    labels:
      version: v1
  - name: local
    labels:
      version: hzall
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: helloworld
spec:
  gateways:
  - helloworld-gateway
  hosts:
  - helloworld.mesh.com
  http:
  - match:
    - headers: 
        x-env:
          exact: local
    route:
    - destination:
        host: helloworld
        subset: local
  - route:
    - destination:
        host: helloworld
        subset: v1
EOF

Access a local application:
```
curl ${INGRESS_HOST} -H 'host: helloworld.mesh.com' -H 'x-env: local'
```
Expected output:
```
Welcome to helloworld stage: local, version: v2, ip: 127.0.0.1
```
The output shows that the stage is local and the version is v2. This indicates that the bidirectional connectivity between services in local environments and clusters is successfully configured, and the local service can be accessed from the cluster.

Access an application in the cluster.

curl ${INGRESS_HOST} -H 'host: helloworld.mesh.com'

Expected output:

Welcome to helloworld stage: online, version: v1, ip: 172.23.16.246

(Optional) Step 4: Clean up the environment

Delete the test resources for the created cluster.
```
kubectl delete ns mesh-demo
```
Stop the local application.
```
docker stop local-container
```