Alibaba Cloud DevOps:Implement CI/CD in a VPC environment

1.1 R&D assets

A complete CI/CD flow revolves around a set of R&D assets. Common R&D assets include:

1.2 R&D flow

These assets are linked into a complete CI/CD flow through a series of operations. The flow both starts and ends between the build cluster and Flow: the build cluster retrieves the build job from Flow (①). After the flow ends, the build cluster syncs the execution results back to Flow (⑩).

After the build cluster receives the job, it clones the code from the repository (②) and completes the build on the build machine. During the build, it pulls the dependency packages from the artifact repository (③). After the build succeeds, the flow branches by deployment type. For a host deployment, the build output is uploaded to the artifact repository (④). For a Kubernetes deployment, the build output (an image) is pushed to the image repository (⑤).

The deployment stage also splits into two paths:

• Host deployment: The Flow Runner is pre-installed on the host cluster and automatically retrieves the deployment job from Flow (⑥), and then downloads the artifact package from the artifact repository (⑦). The Runner runs the deployment commands on the host to complete the release.

• Kubernetes deployment: The build machine sends the deployment commands to the Kubernetes cluster (⑧). The Kubernetes cluster then pulls the image from the image repository (⑨) and completes the deployment.

2.1 Core R&D assets must not be exposed to the Internet

R&D assets such as repositories, artifact repositories, image repositories, Kubernetes clusters, and host clusters carry the most critical code and business systems of an enterprise. Exposing them to the Internet (by opening public endpoints or by managing IP address whitelists poorly) widens the attack surface: source code and artifacts can be pulled and tampered with, and the Kubernetes API Server and the hosts can be exposed to brute-force attacks. Your core requirement is therefore: R&D assets must not be exposed to the Internet and must be accessible only from within the VPC — the assets hold only private addresses and are either invisible from the Internet side or expose only the minimum necessary controlled entry points.

2.2 Run the R&D flow inside the VPC

Isolating the assets alone is not enough. If the build machines and the Runners on the hosts that run these flows sit on the Internet side, every code clone, image push, deployment command, job retrieval, and artifact download has to leave the VPC and come back. The traffic detours over the Internet, and the isolation described earlier loses most of its value. The complete requirement is therefore: the flows between R&D assets should also stay inside the VPC as much as possible and must not traverse the Internet — not only must the target assets be in the VPC, but the build compute resources that initiate the flows must also enter the VPC and share the same internal network as the assets.

2.3 The build cluster is the core of a CI/CD solution

Map these two requirements onto the relationship diagram in section 1. The build cluster initiates and runs every R&D flow, so it determines the answers to two key questions in the solution design:

1. The network location of the build cluster: Does the build cluster sit on the Internet (the Alibaba Cloud DevOps network), or inside the user VPC? The answer determines whether the requirement to run the R&D flow inside the VPC can be met.

2. How the build cluster connects to the R&D assets: Does each link between the build cluster and the R&D assets — repositories, artifact repositories, image repositories, Kubernetes clusters, and host clusters — go over the Internet or over the VPC internal network?

   The next section describes the two build cluster options that Alibaba Cloud DevOps provides, based on these two questions.

3.1 Alibaba Cloud DevOps default build cluster

The Alibaba Cloud DevOps default build cluster uses Alibaba Cloud DevOps network resources. The build machines run on the Alibaba Cloud DevOps side and can connect to the R&D assets (repositories, artifact repositories, and image repositories) only through the Internet. For host deployments, the Runner on the host cluster retrieves deployment jobs from Flow and downloads artifacts from Packages over the Internet.

• Advantages: Ready to use and simple to operate. You do not need to prepare your own machines, and you pay no additional fees for network resources.

• Limits: All links go over the Internet. The R&D assets must expose endpoints that are accessible from the Internet, and flows such as code cloning, image pushing, and artifact downloading also detour over the Internet.

  This option suits scenarios in which all target resources can be accessed securely through public endpoints and you want to get started quickly with zero O&M.

3.2 User VPC build cluster

The user VPC build cluster uses your own VPC network resources. The build machines are scheduled into your VPC and share the same internal network as your workload resources. They can connect to all of your R&D assets (image repositories, Kubernetes clusters, repositories, and artifact repositories) through the VPC internal network. For host deployments, the Runner on the host cluster retrieves the deployment jobs and the artifacts.

• Advantages: The R&D assets do not need to be exposed to the Internet, and the build flow can complete inside the VPC. The links are shorter and more secure, and the latency is lower.

• Trade-offs: The configuration is more complex. You must associate a VPC and plan the security groups and routes, and you must pay additional fees for the user-side network resources that you occupy (VPCs, vSwitches, and NAT Gateways). In addition, accessing the Alibaba Cloud DevOps site over the internal network through a VPC domain name is supported only by regional organizations in enterprise collaboration mode. Before you choose this option, confirm that your organization type and collaboration mode meet this requirement.

  This option suits scenarios with strict requirements for network isolation, compliance, or latency, or scenarios in which the R&D assets are accessible only from the internal network.

4.1 Step 1: Create a VPC build cluster

In Flow, go to the Build Clusters management page. Create a build cluster, set its type to VPC Build Cluster, and complete the following key configurations:

1. Basic information: Enter a name for the build cluster. The build cluster ID is generated automatically by the system. Copy it for later use.

2. Associate a user VPC and a security group: Select the user VPC (and the vSwitch) into which the build machines are deployed, and bind a security group. The security group must allow the necessary outbound ports between the build machines and the targets, such as the Alibaba Cloud DevOps site, ACR, and ACK. If access control is enabled for ACR or ACK, point its whitelist to this security group or to the VPC CIDR block.

3. Select an access point: Job distribution and result reporting between the VPC build cluster and the Alibaba Cloud DevOps site go through an access point. There are three cases, depending on the type of Alibaba Cloud DevOps organization in use and the domain name that you select.

   When you select an access point, start with the organization type. A central organization supports public network access only, which corresponds to scenario 1. A regional organization supports both public network access and VPC private access, which correspond to scenario 2 and scenario 3, and the VPC access capability belongs to enterprise collaboration mode. For security, choose the VPC domain name first: the traffic does not traverse the Internet, and you can narrow the security group whitelist by IP address. Choose the public domain name second. A central organization has only a public domain name, so there is nothing to select.

4.2 Step 2: Verify the connection to the Alibaba Cloud DevOps site

After you create the cluster and complete the network configuration on the VPC side, verify the connectivity with a minimal pipeline:

1. Create a pipeline. For the runtime build cluster, select the VPC build cluster that you created in step 1.

2. Configure the pipeline sources: Add one Codeup repository and one Packages software repository.

   • Codeup usually provides a codeup-demo sample repository. Select it directly.

   • A Packages software repository contains no artifact files by default. First, upload any file so that the pipeline source has content to download.

3. Add an Alibaba Cloud DevOps run command step to the job: You do not need to write your own verification commands. Even with a blank command, the pipeline automatically downloads the pipeline sources (the content of the repository and the artifact repository) when it runs. If this step succeeds, both the code cloning link and the artifact downloading link work.

   Run the pipeline. You can quickly locate network problems from the execution logs:

If the pipeline runs, all the links between the VPC build cluster and the Alibaba Cloud DevOps site are available: job retrieval, code cloning, and artifact pulling. If it fails, use the preceding table to locate the affected link, and then return to step 1 to check the access point and security group configurations.

4.3 Step 3: Verify the connections to ACR and ACK

Add two more steps to the pipeline from step 2 to verify the deployment-side links. Again, you do not need to write your own commands. Select the built-in pipeline steps of Alibaba Cloud DevOps:

1. Verify the ACR connection: Add the pipeline step Docker login ACR, select the service connection for ACR, and enter the address of the image repository. Enter the VPC access address of ACR (the VPC domain name), not the public address. If the step succeeds, the build machine can access ACR through the VPC internal network, and the image push link is available.

2. Verify the ACK connection: Add the pipeline step Kubectl Deploy, and select the target Kubernetes cluster (ACK). Before you use this step, upload the YAML file that you want to deploy to the repository (for example, deployment.yaml), and then enter the relative path of the YAML file in the repository in the step configuration. If the step succeeds, the build machine can access the ACK API Server through the VPC internal network and complete the release, and the kubectl deployment link is available.

   If a step fails, use the following table to locate the problem:

If both steps succeed, the network connectivity of the entire build and deployment flow is verified. You can migrate your real build jobs and deployment jobs to run on this VPC build cluster.