:Connect ACK and on-premises K8s using SAG vCPE

Scenario

An enterprise has a Kubernetes cluster deployed in its private data center and an ACK cluster deployed in an Alibaba Cloud region. The enterprise wants to use an SAG vCPE to connect its on-premises Kubernetes cluster with its cloud-based ACK cluster, enabling resource access between them.

You can deploy the SAG vCPE image on an instance (a physical server or a virtual machine) within your data center. This instance acts as an SAG vCPE device that connects to Alibaba Cloud. Once connected, you can use Cloud Connect Network and Cloud Enterprise Network to enable communication between resources in your data center and your Alibaba Cloud VPC. The following figure shows the architecture for this scenario.

Procedure

1. Create an SAG vCPE instance: In the Smart Access Gateway console, create an SAG vCPE instance, which you will use to manage the SAG vCPE device.

2. Deploy the SAG vCPE image: Choose a host in your data center and deploy the SAG vCPE image on it. This host will act as the SAG vCPE device to connect your data center network to Alibaba Cloud.

3. Configure the Alibaba Cloud network: In the Smart Access Gateway console, configure the network settings for the SAG vCPE device to ensure it can connect to Alibaba Cloud.

4. Configure the data center network: Configure the network in your data center to enable communication between the data center and Alibaba Cloud ACK resources.

5. Verify connectivity: Verify the network connectivity between the hosts in the cloud and the data center, and between the pods in the cloud and the data center.

Step 1: Create an SAG vCPE instance

Create an SAG vCPE instance in the Smart Access Gateway console. This instance is used to manage your SAG vCPE device.

1. Log on to the Smart Access Gateway console.

2. On the Smart Access Gateway page, choose Purchase SAG > Create SAG (vCPE).

3. On the buy page, configure the following parameters and click Buy Now to complete the payment.

   Parameter	Description	Example
   Area	The area where the SAG vCPE instance is deployed.	Chinese mainland
   Instance name	The name of the SAG vCPE instance. This parameter is optional. The name must be 2 to 128 characters in length, start with a letter or a Chinese character, and can contain digits, periods (.), underscores (_), or hyphens (-).	Demo
   Instance Type	The instance type of the SAG vCPE.	SAG-vCPE
   Edition	The edition of the SAG vCPE.	Basic
   Deployment method	The deployment method for the SAG vCPE. The default is Active-Active. In Active-Active mode, one SAG vCPE instance can connect to two SAG vCPE devices by default. You can configure two devices in an active-standby setup to connect your local network to Alibaba Cloud, improving network availability. This tutorial uses only the primary device.	Active-Active
   Peak bandwidth	The peak bandwidth for network communication, in Mbit/s.	50 Mbps
   Quantity	The number of SAG vCPE instances to create.	1
   Subscription duration	The subscription duration. You can select the Auto Renewal checkbox to enable automatic renewal.	1 Month
   Resource Group	The resource group that will contain the SAG vCPE instance.	N/A

4. Return to the Smart Access Gateway console. In the top navigation bar, select the region where you created the instance.

5. In the left-side navigation pane, click Smart Access Gateway.

6. On the Smart Access Gateway page, click the ID of the instance you created.

7. On the instance details page, click the Device Management tab. View and record the serial number and key of the primary SAG vCPE device. You will need this information to associate the SAG vCPE instance with the device.

   Select the primary device tab. In the left pane, click the Device Information menu to view the serial number. Next to the Key field, click the Show link to view the key.

Step 2: Deploy the SAG vCPE image

To enable network communication between your data center and the ACK cluster, you need to select a host in your data center to deploy the SAG vCPE image. After deployment, this host functions as an SAG vCPE device, connecting your data center network to Alibaba Cloud.

1. Select a host in your data center.

   To ensure the SAG vCPE image runs correctly, the selected host must meet the following requirements:

   • Supports one of the following operating systems:

     • (Recommended) CentOS 7.6 (64-bit) or later.

     • Ubuntu 18.04 (64-bit) or later.

   • Supports kernel version 3.10.0-957.21.3.el7.x86_64 or later.

   • Has a dedicated network interface card (NIC) that can connect to the internet.

   • Supports remote logon.

   • Is not running any business-critical systems.

2. Log on to the host and run the following command to download the script to the /root directory of the host.

   Note

   • You can download the script to a custom directory. If you do, make sure to update the path when you run the script.

   • Do not modify the content or name of the script after downloading.

   Choose the appropriate command based on whether your host is deployed in or outside the Chinese mainland.

   • Host is deployed in the Chinese mainland

     wget -O /root/sag_vcpe_v2.3.0_deployment.sh https://sdwan-oss-shanghai.oss-cn-shanghai.aliyuncs.com/vcpe_vm/sag_vcpe_v2.3.0_deployment.sh

   • Host is deployed outside the Chinese mainland

     wget -O /root/sag_vcpe_v2.3.0_deployment.sh https://sdwan-oss-shanghai.oss-accelerate.aliyuncs.com/vcpe_vm/sag_vcpe_v2.3.0_deployment.sh

3. Run the following command to grant execute permissions to the script.

   chmod +x /root/sag_vcpe_v2.3.0_deployment.sh

4. Run the script.

   /root/sag_vcpe_v2.3.0_deployment.sh -n sag**** -k X8==**** -t idc  -w eth0

   The following table describes key parameters. For a complete list of script parameters, see Script parameter descriptions.

   Parameter	Description
   -n	The serial number of the SAG vCPE device.
   -k	The key of the SAG vCPE device.
   -t	The platform where the host for the SAG vCPE image is located. Valid values: aliyun (default): The SAG vCPE image is deployed on an Alibaba Cloud Elastic Compute Service (ECS) instance. aws: The SAG vCPE image is deployed on an AWS EC2 instance. ens: The SAG vCPE image is deployed on an Alibaba Cloud Edge Node Service (ENS) instance. If your SAG vCPE image is deployed on an on-premises server, set this parameter to any string of English characters other than aliyun, aws, or ens.
   -w	The name of the WAN port's NIC. You can run the ifconfig command to view the host's NIC names.

   When you run the script, the system automatically checks if the deployment environment meets the requirements:

   • If some required components are not installed, the system displays the following prompt. Enter yes to have the system automatically install these components. The script then begins to deploy the SAG vCPE image.

     Missing dependency packages python3 telnet vim traceroute sshpass mtr docker inotify wget netifaces==0.10.9, are you sure want to install these packages? [yes|no]
     [no] >>>

   • If the deployment environment meets the requirements, the system starts deploying the SAG vCPE image directly. When the deployment is complete, the script displays the following message:

     2021-10-27 01:48:08 INFO: Install complete, please wait for few minutes to check state
     ==============================================================================================
     [root@xxx ~]#

5. After the deployment is complete, run the docker ps command to check if the following two containers are present in the system.

   [root@xxx ~]# docker ps
   CONTAINER ID   IMAGE        COMMAND                  CREATED       STATUS       PORTS     NAMES
   b5xxx          vsag-core    "sh -c 'mkdir -p /li…"   7 days ago    Up 7 days              vsag-core
   2fxxx          centos:7     "/sbin/init"             7 days ago    Up 7 days              vsag-manager-base

   • If the system contains the vsag-core and vsag-manager-base containers, the deployment is successful.

   • If the system does not contain these two containers, the deployment has failed. In this case, submit a ticket for assistance.

Step 3: Configure the Alibaba Cloud network

After deploying the SAG vCPE image, you must configure the network for the SAG vCPE device in the Smart Access Gateway console to connect it to Alibaba Cloud.

1. Configure on-premises route synchronization.

   1. Log on to the Smart Access Gateway console and select the target region from the top navigation bar.

   2. On the Smart Access Gateway page, find the target instance and click Network Settings in the Actions column.

   3. On the Network Settings tab, in the Method to Synchronize with On-premises Routes section, click Add Static Route.

   4. In the Add Static Route dialog box, enter the private CIDR block of your data center, and then click OK.

   5. Click Add Static Route again. In the Add Static Route dialog box, enter the pod CIDR block of the Kubernetes cluster in your data center, and then click OK.

      After the routes are added, on the Network Configuration tab, under the Method to Synchronize with On-premises Routes section, select Static Route. You can view the added route entries, such as 192.168.0.0/16 and 10.18.0.0/16.

2. Attach a Cloud Connect Network instance.

   Cloud Connect Network is a key component of SAG. SAG uses Cloud Connect Network to connect your network to Alibaba Cloud.

   1. Create a Cloud Connect Network instance. For more information, see Create a CCN instance.

      Note

      The Cloud Connect Network instance must be in the same region as the SAG vCPE instance.

   2. In the left-side navigation pane of the Smart Access Gateway console, click Smart Access Gateway.

   3. On the Smart Access Gateway page, find the target instance and click Network Settings in the Actions column.

   4. On the Network Settings tab, select Network Instance Details.

   5. In the Associated Instances Under Current Account section, click Attach Network, select the Cloud Connect Network instance you created, and then click OK.

   6. Click the Device Management tab to view the VPN Status and Controller Status of the SAG vCPE device.

      After attaching the Cloud Connect Network instance, if the VPN Status and Controller Status of the SAG vCPE device are both Normal, it indicates that the SAG vCPE device is connected to Alibaba Cloud.

3. Configure Cloud Enterprise Network.

   Connect the SAG vCPE instance and the VPC to a Cloud Enterprise Network (CEN). This allows the instances to learn routes from each other, enabling communication between the SAG vCPE device and resources in the VPC.

   1. In the left-side navigation pane of the Smart Access Gateway console, click CCN Instance.

   2. On the CCN Instance page, find the target Cloud Connect Network instance and click Bind CEN Instance in the Actions column.

   3. In the Bind CEN Instance panel, select the Cloud Enterprise Network instance to bind, and then click OK.

      You can choose a Cloud Enterprise Network instance in one of two ways. In this topic, Create CEN is selected.

      • Existing CEN: If you have already created a CEN, you can select it from the drop-down list to bind.

      • Create CEN: If you do not have a CEN, enter a name in the text box. The system creates a new CEN instance and automatically binds it.

        Note

        The CEN instance name must be 2 to 100 characters in length, start with a letter or a Chinese character, and can contain digits, underscores (_), or hyphens (-).

   4. Attach the VPC instance you created to this Cloud Enterprise Network. For more information, see Attach a network instance.

Step 4: Configure the data center network

To enable communication between your data center and Alibaba Cloud resources, you must also configure your data center's network. Contact your data center network administrator for the specific commands required for your environment.

1. Configure a static route in the data center.

   Configure a route in your data center network that directs traffic destined for the Alibaba Cloud ACK cluster's CIDR block (and the pod CIDR block, if needed) to the IP address of the host where the SAG vCPE is installed. This host handles communication with Alibaba Cloud services.

   ip route add 10.77.0.0/16 via 192.168.11.210

   Note

   This static route example is for reference only. The command may vary depending on the device manufacturer and model.

2. Configure the firewall in the data center.

   Allow traffic between the private CIDR blocks of the Alibaba Cloud ACK cluster and the data center.

Step 5: Verify connectivity

1. Verify host connectivity.

   1. Log on to an ECS instance within your VPC. For more information, see Connection methods overview.

   2. Run the ping command to access a host in your on-premises data center to verify the network connection between your cloud and on-premises hosts.

      This output confirms that resources in the Alibaba Cloud VPC can communicate with resources in the on-premises data center.

      [root@<hostname> ~]# ping 192.168.11.236
      PING 192.168.11.236 (192.168.11.236) 56(84) bytes of data.
      64 bytes from 192.168.11.236: icmp_seq=1 ttl=61 time=9.52 ms
      64 bytes from 192.168.11.236: icmp_seq=2 ttl=61 time=9.30 ms
      64 bytes from 192.168.11.236: icmp_seq=3 ttl=61 time=9.24 ms

2. Verify pod network connectivity.

   1. Deploy test containers on an ACK cluster on Alibaba Cloud and a Kubernetes cluster in an on-premises IDC. The following is a YAML example.

      apiVersion: apps/v1 # for versions before 1.8.0 use apps/v1beta1
      kind: Deployment
      metadata:
        name: nginx-deployment-basic
        labels:
          app: nginx
      spec:
        replicas: 2
        selector:
          matchLabels:
            app: nginx
        template:
          metadata:
            labels:
              app: nginx
          spec:
          #  nodeSelector:
          #    env: test-team
            containers:
            - name: nginx
              image: nginx:1.7.9 # Replace with your actual <image_name:tags>.
              ports:
              - containerPort: 80
              resources:
                limits:
                  cpu: "500m"

   2. Access the container in the ACK cluster and run the ping command to access the IP address of a pod in the on-premises Kubernetes cluster. This verifies pod network connectivity between the ACK cluster and the on-premises Kubernetes cluster.

      This output confirms that pods in the Alibaba Cloud ACK cluster can communicate with pods in the on-premises Kubernetes cluster.

      root@nginx-deployment-basic-8xxx-pjns8:/# ping 10.18.1.131
      PING 10.18.1.131 (10.18.1.131): 48 data bytes
      56 bytes from 10.18.1.131: icmp_seq=0 ttl=59 time=10.106 ms
      56 bytes from 10.18.1.131: icmp_seq=1 ttl=59 time=9.930 ms
      56 bytes from 10.18.1.131: icmp_seq=2 ttl=59 time=9.926 ms
      56 bytes from 10.18.1.131: icmp_seq=3 ttl=59 time=9.909 ms
      56 bytes from 10.18.1.131: icmp_seq=4 ttl=59 time=9.888 ms
      56 bytes from 10.18.1.131: icmp_seq=5 ttl=59 time=9.859 ms

Summary

By using an SAG vCPE with Cloud Connect Network and Cloud Enterprise Network, you can quickly set up a network that connects your on-premises and cloud environments, for seamless communication. To minimize potential issues, plan your network architecture in advance to avoid network conflicts that could impact your production environment.