Deploy and configure the Terway plugin - Container Service for Kubernetes

The container network plugin in a hybrid cluster has two components: one that runs in your data center and another that runs on cloud compute nodes. This topic describes how to deploy and configure the Terway plugin in a hybrid cluster.

Prerequisites

For Scenario 2: The container network in the data center uses BGP networks and Scenario 3: The container network in the data center uses the host network, you must configure the Terway network parameters when you create a registered cluster.
- Select or clear the IPvlan checkbox as needed.
- Configure the pod virtual switch.
- Configure the Service CIDR block.
For more information, see Create an ACK One registered cluster.
The container network plugins for cloud and on-premises nodes are configured.

Scenario 1: The container network in the data center uses an Overlay network

If the container network in your data center uses an Overlay network, the cloud compute nodes can also use this mode. Ensure that the cloud compute nodes can pull the container images used by the container network plugin DaemonSet.

Common Overlay network modes include the following:

Flannel VXLAN mode
Calico IPIP mode
Cilium VXLAN mode

Scenario 2: The container network in the data center uses BGP networks

If the container network in your data center uses Border Gateway Protocol (BGP) networks, the cloud compute nodes must use the Terway network. For more information about how to enable network communication between cloud and on-premises containers, see Configure BGP on a VBR.

In this scenario, ensure that the following conditions are met:

The DaemonSet of the on-premises container network plugin, such as Calico in BGP route reflector mode, is not scheduled to cloud compute nodes.
The DaemonSet of the Terway plugin is not scheduled to on-premises compute nodes.

In a registered cluster, compute nodes that are added by scaling out a node pool are assigned the alibabacloud.com/external=true label. You can use this label as a filter condition to distinguish between cloud and on-premises nodes.

For the on-premises Calico plugin, use nodeAffinity to prevent it from being scheduled to nodes that have the alibabacloud.com/external=true label. You can use this method for any workload that runs on-premises and should not be scheduled to the cloud. Run the following command to update the Calico plugin.

cat <<EOF > calico-ds.patch
spec:
  template:
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: alibabacloud.com/external
                operator: NotIn
                values:
                - "true"
EOF
kubectl -n kube-system patch ds calico-node -p "$(cat calico-ds.patch)"

By default, the Terway plugin is scheduled only to cloud nodes that have the alibabacloud.com/external=true label.

Scenario 3: The container network in the data center uses the host network

If the container network in your data center uses the host network, you must ensure that the DaemonSet of the Terway plugin is not scheduled to on-premises nodes. By default, the Terway plugin is scheduled only to cloud nodes that have the alibabacloud.com/external=true node label.

Install and configure the Terway plugin

In Scenario 2 and Scenario 3, you must install and configure the Terway plugin on the cloud nodes of the hybrid cluster.

Step 1: Configure RAM permissions for the Terway plugin

Configure in the console

Create a Resource Access Management (RAM) user and grant the following custom policy to the RAM user. For more information, see Use RAM to grant access permissions to clusters and cloud resources.

View the custom policy

{
    "Version": "1",
    "Statement": [
        {
            "Action": [
                "ecs:CreateNetworkInterface",
                "ecs:DescribeNetworkInterfaces",
                "ecs:AttachNetworkInterface",
                "ecs:DetachNetworkInterface",
                "ecs:DeleteNetworkInterface",
                "ecs:DescribeInstanceAttribute",
                "ecs:AssignPrivateIpAddresses",
                "ecs:UnassignPrivateIpAddresses",
                "ecs:DescribeInstances",
                "ecs:ModifyNetworkInterfaceAttribute"
            ],
            "Resource": [
                "*"
            ],
            "Effect": "Allow"
        },
        {
            "Action": [
                "vpc:DescribeVSwitches"
            ],
            "Resource": [
                "*"
            ],
            "Effect": "Allow"
        }
    ]
}

Log on to the Container Service Management Console . In the navigation pane on the left, click Clusters.
On the Clusters page, click the name of your cluster. In the navigation pane on the left, click Configurations > Secrets.
On the Secrets page, click Create from YAML. Enter the following content to create a secret named alibaba-addon-secret.
Note
The component uses the AccessKey ID and AccessKey secret stored in this secret to access cloud services. If the alibaba-addon-secret secret already exists, you can skip this step.
```
apiVersion: v1
kind: Secret
metadata:
  name: alibaba-addon-secret
  namespace: kube-system
type: Opaque
stringData:
  access-key-id: <The AccessKey ID of the RAM user>
  access-key-secret: <The AccessKey secret of the RAM user>
```

Configure using onectl

Install onectl on your on-premises machine. For more information, see Use onectl to manage registered clusters.

Run the following command to configure RAM permissions for the Terway plugin.

onectl ram-user grant --addon terway-eniip

Expected output:

Ram policy ack-one-registered-cluster-policy-terway-eniip granted to ram user ack-one-user-ce313528c3 successfully.

Step 2: Install the Terway plugin

Install from the console

Log on to the Container Service Management Console . In the navigation pane on the left, click Clusters.
On the Clusters page, click the name of your cluster. In the navigation pane on the left, click Add-ons.
On the Add-ons page, click the Network tab. In the terway-eniip section, click Install.

Install using onectl

Run the following command to install the Terway plugin.

onectl addon install terway-eniip

Expected output:

Addon terway-eniip, version **** installed.