Container Service for Kubernetes:Use SAG vCPE to connect an ACK cluster with an on-premises Kubernetes cluster

Scenarios

An enterprise deploys a Kubernetes cluster in a data center and creates an ACK cluster in an Alibaba Cloud region. The enterprise wants to use SAG vCPE to connect the on-premises Kubernetes cluster with the ACK cluster. This way, resources in the on-premises cluster can communicate with resources in the ACK cluster.

You can deploy the SAG vCPE image on an instance in the data center. The instance can be a physical server or a VM. This way, the instance can be used as a vCPE device that allows you to connect private networks to Alibaba Cloud. After you connect the SAG vCPE device to Alibaba Cloud, you can enable resources in the data center and Alibaba Cloud VPCs to communicate with each other by using Cloud Connect Network (CCN) and Cloud Enterprise Network (CEN). The scenario in the following figure is used as an example.

Flowchart

1. Create an SAG vCPE instance in the SAG console. Then, you can use the instance to manage an SAG vCPE device.

2. Select a host in the data center and deploy the SAG vCPE image on the host. The host can be used as the SAG vCPE device that allows you to connect private networks to Alibaba Cloud.

3. Plan CIDR blocks for the SAG vCPE device in the SAG console. This way, the device can connect to Alibaba Cloud.

4. Plan CIDR blocks for the data center. This way, resources in the data center and ACK cluster can communicate with each other.

5. Verify the connectivity between the hosts on Alibaba Cloud and the data center, and the connectivity between pods in the ACK cluster and the on-premises Kubernetes cluster.

Step 1: Create an SAG vCPE instance

You must create an SAG vCPE instance in the SAG console. Then, you can use the SAG vCPE instance to manage an SAG vCPE device.

1. Log on to the SAG console.

2. On the Smart Access Gateway page, choose Purchase SAG > Create SAG (vCPE).

3. On the buy page, set the following parameters and click Buy Now to complete the payment.

   Parameter	Description	Example
   Area	Select the region where you want to deploy the SAG vCPE instance.	Chinese mainland
   Instance Name	Enter the name of the SAG vCPE instance. You can leave this parameter empty. The name must be 2 to 128 characters in length, and can contain digits, periods (.), underscores (_), and hyphens (-). It must start with a letter.	Demo
   Instance Type	Select an instance type.	SAG-vCPE
   Edition	Select an edition for the SAG vCPE instance.	Basic Edition
   Deployment Method	Select a method to deploy the SAG vCPE instance. By default, Active-Active is selected. In this mode, one SAG vCPE instance can be associated with two SAG vCPE devices. You can deploy two SAG vCPE devices in active-active mode and connect on-premises networks to Alibaba Cloud. This improves network availability. In this example, only one device is used.	Active-Active
   Peak Bandwidth	Specify the bandwidth limit for network communication. Unit: Mbit/s.	50 Mbps
   Quantity	Specify the number of SAG vCPE instances that you want to create.	1
   Subscription Duration	Select a subscription duration. You can select Auto-renewal to enable automatic renewal upon expiration.	1 Month
   Resource Group	Select the resource group to which the SAG vCPE instance belongs.	N/A

4. Return to the SAG console. In the top navigation bar, select the region where you created the SAG vCPE instance.

5. In the left-side navigation pane, click Smart Access Gateway.

6. On the Smart Access Gateway page, click the ID of the SAG vCPE instance.

7. On the instance details page, click the Device Management tab, view and record the serial number and key of the active SAG vCPE device. The serial number and key are used to associate the SAG vCPE instance with an SAG vCPE device.

   

Step 2: Deploy the SAG vCPE image

To connect an on-premises Kubernetes cluster with an ACK cluster, you must select a host in the data center that is used to deploy the SAG vCPE image. After you deploy the SAG vCPE image, the host can be used as an SAG vCPE device and allows you to connect resources in the data center to Alibaba Cloud resources.

1. Select a host in the data center.

   To ensure that the SAG vCPE image runs as expected, the host that you select must meet the following requirements:

   • The host supports the operating systems of the following versions:

     • (Recommended) CentOS 7.6 64-bit or later.

     • Ubuntu 18.04 64-bit or later.

   • The host supports the kernel version 3.10.0-957.21.3.el7.x86_64 or later.

   • The host has an independent network interface controller (NIC) that is used to connect the host to the Internet.

   • You can remotely log on to the host.

   • No service system is running on the host.

2. Log on to the host and run the following command. The command is used to download the script to the /root directory of the host.

   Note

   • You can also specify a custom path and download the script to the corresponding directory. In this case, make sure that you select the custom path when you run the script.

   • After you download the script, do not modify its content or name.

   The commands vary based on whether the host is deployed within the Chinese mainland. You must run a suitable command to download the script.

   • The host is deployed within the Chinese mainland:

   wget -O /root/sag_vcpe_v2.3.0_deployment.sh https://sdwan-oss-shanghai.oss-cn-shanghai.aliyuncs.com/vcpe_vm/sag_vcpe_v2.3.0_deployment.sh

   • The host is deployed outside the Chinese mainland:

     wget -O /root/sag_vcpe_v2.3.0_deployment.sh https://sdwan-oss-shanghai.oss-accelerate.aliyuncs.com/vcpe_vm/sag_vcpe_v2.3.0_deployment.sh

3. Run the following command to make the script executable:

   chmod +x /root/sag_vcpe_v2.3.0_deployment.sh

4. Run the script:

   /root/sag_vcpe_v2.3.0_deployment.sh -n sag**** -k X8==**** -t idc  -w eth0

   The following table describes some of the parameters. For more information about the script parameters, see Descriptions of the script parameters.

   Parameter	Description
   -n	The serial number of the SAG vCPE device.
   -k	The key of the SAG vCPE device.
   -t	The service provider of the host on which you want to install the SAG vCPE image. Valid values: aliyun (default): deploys the SAG vCPE image on an Alibaba Cloud Elastic Compute Service (ECS) instance. aws: deploys the SAG vCPE image on an Amazon Elastic Compute Cloud (EC2) instance. ENS: deploys the SAG vCPE image on an Edge Node Service (ENS) instance. If you want to deploy the SAG vCPE image on an on-premises server, set the value to a string of letters except aliyun, aws, or ens.
   -w	The name of the NIC for the WAN port. You can view the NIC name of the host by running the ifconfig command.

   When you run the script, the system automatically checks whether the deployment environment meets the requirements.

   • If specific components are not installed in the deployment environment, the following prompt appears. You can enter yes. Then, the system automatically installs the components. After the components are installed, the system starts to deploy the SAG vCPE image.

   • If the deployment environment meets the requirements, the system automatically starts to deploy the SAG vCPE image. After the image is deployed, the following prompt appears.

5. After the SAG vCPE image is deployed, run the docker ps command to check whether the system contains the following containers.

   

   • If the system contains the vsag-core container and the vsag-manager-base container, the SAG vCPE image is deployed.

   • If the system does not contain the containers, the SAG vCPE image fails to be deployed. In this case, submit a ticket to contact Alibaba Cloud technical support.

Step 3: Configure networks on the Alibaba Cloud side

After the SAG vCPE image is deployed, you must plan CIDR blocks for the SAG vCPE device in the SAG console. This allows the SAG vCPE device to connect to Alibaba Cloud.

1. Select a method to advertise routes to Alibaba Cloud.

   1. Log on to the SAG console. In the top navigation bar, select the region.

   2. On the Smart Access Gateway page, find the SAG vCPE instance and click Network Configuration in the Actions column.

   3. In the Method to Synchronize with On-premises Routes section of the Network Configuration tab, click Add Static Route.

   4. In the Add Static Route dialog box, enter the CIDR block of the data center and click OK.

   5. Click Add Static Route again. In the Add Static Route dialog box, enter the pod CIDR block of the on-premises Kubernetes cluster and click OK.

      The following figure shows the page that is displayed after you specify the CIDR blocks.

2. Associate the SAG vCPE instance with a CCN instance.

   CCN is an important component of SAG. SAG connects your private networks to Alibaba Cloud through CCN.

   1. Create a CCN instance. For more information, see Create a CCN instance.

      Note

      The SAG vCPE instance and CCN instance must be deployed in the same region.

   2. In the left-side navigation pane of the SAG console, click Smart Access Gateway.

   3. On the Smart Access Gateway page, find the SAG vCPE instance and click Network Configuration in the Actions column.

   4. On the Network Configuration tab, click Network Instance Details.

   5. In the Associated Instances Under Current Account section, click Attach Network. In the dialog box that appears, select a CCN instance and click OK.

   6. Click the Device Management tab to view the VPN status and controller status of the SAG vCPE device.

      If the VPN status and controller status of the SAG vCPE device are normal after you associate the SAG vCPE instance with the CCN instance, the SAG vCPE device is connected to Alibaba Cloud.

3. Configure a CEN instance.

   You must perform the following operations to connect the SAG vCPE instance to CEN and attach the Alibaba Cloud VPC to a CEN instance. Then, the SAG vCPE instance and the Alibaba Cloud VPC can learn routes from each other. This way, the SAG vCPE device can communicate with the resources in the Alibaba Cloud VPC.

   1. In the left-side navigation pane of the SAG console, click CCN.

   2. On the CCN page, find the CCN instance and click Bind CEN Instance in the Actions column.

   3. In the Bind CEN Instance panel, select the CEN instance you want to associate and click OK.

      You can use one of the following methods to select a CEN instance. Create CEN is selected in this example.

      • Existing CEN: If you have already created a CEN instance, you can select an existing CEN instance from the drop-down list.

      • Create CEN: If no CEN instance is available, enter an instance name. Then, the system creates a CEN instance and automatically attaches the CCN instance to the CEN instance.

        Note

        The instance name must be 2 to 100 characters in length, and can contain digits, underscores (_), and hyphens (-). It must start with a letter.

   4. Attach the Alibaba Cloud VPC to the CEN instance. For more information, see Attach a network instance.

Step 4: Configure networks for the data center

To connect resources in the data center to resources on Alibaba Cloud, you must plan CIDR blocks for the data center. For more information about the commands that are used to plan the CIDR blocks, consult the network administrators of the data center.

1. Example on how to add static routes for the data center.

   Add routes for the data center. You must set the next hop of the CIDR block of the ACK cluster that you want to access to the IP address of the SAG vCPE device. You must also add the pod CIDR block if you want to access the pod. The SAG vCPE device is used to connect the on-premises network to Alibaba Cloud.

   ip route add 10.77.0.0/16 via 192.168.11.210

   Note

   The route in this example is provided only for reference. Route configurations may vary based on the manufacturer of the device.

2. Configure security group rules for the data center.

   Configure security group rules to allow the CIDR blocks of the ACK cluster and the data center to communicate with each other.

Step 5: Verify the connectivity

1. Verify the connectivity between hosts.

   1. Log on to an ECS instance in the VPC. For more information, see Overview.

   2. Run the ping command to verify the connectivity between the ECS instance and a host in the data center.

      The output in the following figure indicates that resources in the VPC can communicate with resources in the data center.

2. Verify the connectivity between pods.

   1. Deploy a test container in the ACK cluster and the on-premise Kubernetes cluster. The following YAML template is provided as an example:

      apiVersion: apps/v1 # for versions before 1.8.0 use apps/v1beta1
      kind: Deployment
      metadata:
        name: nginx-deployment-basic
        labels:
          app: nginx
      spec:
        replicas: 2
        selector:
          matchLabels:
            app: nginx
        template:
          metadata:
            labels:
              app: nginx
          spec:
          #  nodeSelector:
          #    env: test-team
            containers:
            - name: nginx
              image: nginx:1.7.9 # Replace this field with <image_name:tags> that you use. 
              ports:
              - containerPort: 80
              resources:
                limits:
                  cpu: "500m"

   2. Enter the container in the ACK cluster and run the Ping command to verify the connectivity to the pod in the on-premise Kubernetes cluster.

      The output in the following figure indicates that the pod in the ACK cluster can communicate with the pod in the on-premise Kubernetes cluster.

Summary

You can use SAG vCPE with CCN and CEN to connect on-premises networks to Alibaba Cloud. We recommend that you plan CIDR blocks in advance to reduce potential risks. Otherwise, your workloads may be affected by network conflicts in production environments.