Container Service for Kubernetes:Use ACK One GitOps and Container Registry to build CI/CD pipelines

Limits

• This best practice is suitable only for applications that are created by using ACK One GitOps.

• This best practice is suitable only for applications that are orchestrated and managed by using Kustomize or Helm.

Prerequisites

• The Fleet management feature is enabled. For more information, see Enable Fleet management.

• The kubeconfig file of the Fleet instance is obtained in the ACK One console and a kubectl client is connected to the Fleet instance.

• ACK One GitOps is enabled for your ACK One Fleet instance. For more information, see Log on to the GitOps system.

• The latest version of the ArgoCD CLI is downloaded and installed through ArgoCD.

• The Kruise Rollouts component is installed in the Staging and Production clusters and the kubectl-kruise component is installed.

Example

The sample application used in this example is named echo-server. The echo-server project and the echo-web-server project that contains multiple Deployments are forked as business code repositories. The gitops-demo project is forked as the Deployment repository. Make sure that the repositories are forked to your account because you need to propose application changes back to the Deployment repository. You can modify image.repository and image.tag in the values.yaml file based on the environment. For more information about any updates on the best practice, see echo-server example.

image:
  repository: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/echo-server
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "v1.0"

Step 1: Create a CI pipeline based on Container Registry

Create an image repository in Container Registry, bind it to the forked echo-server project, and create a build rule. For more information, see Use a Container Registry Enterprise Edition instance to build an image. You can turn on Allow Anonymous Users to Pull Public Artifacts from the overview page of the Container Registry Enterprise Edition instance or follow the steps in Use the aliyun-acr-credential-helper component to pull images without using a secret to pull images without using a Secret.

In this best practice, you need to add a build rule to allow Container Registry to automatically build Docker images and push the images to the image repository when new tags that start with release- are detected. You can configure the regular expression in the build rule on demand.

Step 2: Configure credentials that are used by ACK One GitOps to access the image repository and Deployment repository

ACK One GitOps automatically listens for image changes in the image repository. When image changes are detected, ACK One GitOps pulls the latest image and synchronizes the tag of the latest image to the Deployment repository (Git repository). Then, changes are synchronized from the latest image to the applications. You can configure credentials for ACK One GitOps to allow the Container Registry repository to interact with the Git repository.

1. Connect to ACK One GitOps. For more information, see Log on to the GitOps system.

2. Add a Git repository. For more information, see Add Git repositories to ACK One GitOps.

3. Create a GitOps application. For more information, see Use GitOps to manage applications.

4. Configure credentials that are used to access the image repository so that ACK One GitOps can listen for image changes in the image repository.

   Run the following command to create a Secret named acr in the argocd namespace of the ACK One Fleet instance:

   kubectl -n argocd apply -f - <<EOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: acr
   type: Opaque
   stringData:
     acr: <your_username>:<your_password>  # Replace <your_username>:<your_password> with the credentials of your image repository. 
   EOF

5. Configure credentials that are used to access the Git repository so that ACK One GitOps can propose image changes back to the Git repository.

If you have already specified the corresponding credentials (username and password) or private key certificate when you add the Git repository to ACK One GitOps, ACK One GitOps has the permissions to propose container image changes to the Git repository.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  annotations:
    argocd-image-updater.argoproj.io/write-back-method: git

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  annotations:
    argocd-image-updater.argoproj.io/write-back-method: git:secret:argocd/git-creds

kubectl -n argocd create secret generic git-creds \
--from-literal=username=<your_username> \
--from-literal=password=<your_password>

Step 3: Configure application image auto update and deploy the application in multiple clusters

You can add the following annotations to enable application image auto update. For more information, see Settings related to application image auto update.

metadata:
  annotations:
    argocd-image-updater.argoproj.io/image-list: echoserver=demo-test-registry.cn-hangzhou.cr.aliyuncs.com/cidemo/echo-server,webserver=demo-test-registry.cn-hangzhou.cr.aliyuncs.com/cidemo/echo-web-server
    argocd-image-updater.argoproj.io/echoserver.helm.image-name: image.echoServer.repository
    argocd-image-updater.argoproj.io/echoserver.helm.image-tag: image.echoServer.tag
    argocd-image-updater.argoproj.io/echoserver.update-strategy: latest
    argocd-image-updater.argoproj.io/webserver.helm.image-name: image.echoWebServer.repository
    argocd-image-updater.argoproj.io/webserver.helm.image-tag: image.echoWebServer.tag
    argocd-image-updater.argoproj.io/webserver.update-strategy: latest
    argocd-image-updater.argoproj.io/write-back-method: git:secret:argocd/git-creds

• echoserver in the value of the argocd-image-updater.argoproj.io/image-list annotation specifies an alias, which indicates the address of the image repository. You can also specify multiple image repository addresses and separate them with commas (,). Replace it with the actual image repository address.

• The annotations are for applications orchestrated by using Helm. For more information about the annotations for applications orchestrated by using Kustomize, see Annotations for applications orchestrated by using Kustomize.

Step 4: Roll back the application

Step 5: Test the CI/CD pipelines

1. Push branches and tags that match the build rule of the Container Registry CI pipeline to trigger Container Registry to build and push images. Run the following command to push new tags. Specify the tag names based on your business requirements.

   git clone https://github.com/{xxx}/echo-web-server.git
   cd echo-web-server
   git push origin HEAD:release-v3

2. Check whether a new image is built.

   1. Log on to the Container Registry console.

   2. In the top navigation bar, select a region.

   3. In the left-side navigation pane, click Instances.

   4. On the Instances page, click the Enterprise Edition instance that you want to manage.

   5. In the left-side navigation pane of the management page of the Enterprise Edition instance, choose Repository > Repositories.

   6. On the image repository page, click the image repository that you want to manage. In the left-side navigation pane, click Build. In the Build Log section, check whether a new image is built.

3. After a new image is built, run the following command to view the log of argocd-image-updater:

   kubectl -nargocd logs argocd-server-<xxxxx> -c argocd-image-updater -f

   Expected output:

   time="2023-07-19T07:35:41Z" level=info msg="Successfully updated image 'demo-test-registry.cn-hangzhou.cr.aliyuncs.com/cidemo/echo-web-server:v1.0' to 'demo-test-registry.cn-hangzhou.cr.aliyuncs.com/cidemo/echo-web-server:v3-5a7147', but pending spec update (dry run=false)" alias=echowebserver application=echo-web-server 
   time="2023-07-19T07:35:41Z" level=info msg="Committing 1 parameter update(s) for application echo-web-server" application=echo-web-server

4. Check whether the file manifests/helm/echo-server/.argocd-source-${appname}.yaml is generated in the Deployment repository on GitHub. If the file is generated, image changes are proposed back. In this best practice, the file is generated in the Dev, Staging, and Production clusters, respectively. The following figure shows the content of the file in the Dev cluster.

   

5. The image used by the Deployment in the Dev cluster is updated to v3-5a7147. You need to manually synchronize the change to the Staging and Production clusters and trigger canary releases so that the images used by the Deployments in the Staging and Production clusters can be updated to v3-5a7147.

Settings related to application image auto update

Specify the container images to be automatically updated

You can add the following annotation to specify one or more container images to be automatically updated for ACK One GitOps:

argocd-image-updater.argoproj.io/image-list: <image_spec_list>

image_spec_list specifies a list of container images that are separated by commas (,). Specify the description of each container image in the following format:

<alias_name>=]<image_path>[:<version_constraint>

alias_name specifies the alias of the container image. The alias is a string and can be used only in the image-list annotation. In this example, specify the image echoserver=demo-test-registry.cn-hangzhou.cr.aliyuncs.com/cidemo/echo-server:v1.0 and set its alias to echoserver.

argocd-image-updater.argoproj.io/image-list: echoserver=demo-test-registry.cn-hangzhou.cr.aliyuncs.com/cidemo/echo-server:v1.0

Image tag filter conditions

The image echoserver=demo-test-registry.cn-hangzhou.cr.aliyuncs.com/cidemo/echo-server:v1.0 is used as an example to demonstrate how to configure image tag filter conditions. Application image auto update is triggered only when new tags that meet the tag filter conditions of the echo-server image in the image repository are pushed.

You can use the following regular expression to specify the image tags that are allowed:

argocd-image-updater.argoproj.io/<image_name>.allow-tags: <match_func>

match_func specifies a standard regular expression. The following code block is an example:

argocd-image-updater.argoproj.io/echoserver.allow-tags: regexp:^v[1-9].*

Configure a container image update policy

The following table describes the container image update policies that you can configure. The default container image update policy is semver.

Add the following annotation to specify a container image update policy:

argocd-image-updater.argoproj.io/<image_name>.update-strategy: <strategy>

In the following code block, the semver update policy is specified:

argocd-image-updater.argoproj.io/echoserver.update-strategy: semver

Annotations for applications orchestrated by using Helm and Kustomize

• Annotations for applications orchestrated by using Helm

The application file may contain multiple container images. For example, the values.yaml file of the gitops-demo application contains the image.echoServer.repository and image.echoServer.tag settings. In this case, add the following annotations:

annotations:
  argocd-image-updater.argoproj.io/image-list: echoserver=demo-test-registry.cn-hangzhou.cr.aliyuncs.com/cidemo/echo-server:v1.0
  argocd-image-updater.argoproj.io/echoserver.helm.image-name: image.echoServer.repository
  argocd-image-updater.argoproj.io/echoserver.helm.image-tag: image.echoServer.tag
  argocd-image-updater.argoproj.io/echoserver.update-strategy: latest
  argocd-image-updater.argoproj.io/write-back-method: git

• Annotations for applications orchestrated by using Kustomize

To configure container image auto update for applications orchestrated by using Kustomize, you need to first specify the aliases (including tags) of the container images to be updated. Then, you need to specify the addresses of the base images (excluding tags). To do this, add the following annotations:

annotations:
  argocd-image-updater.argoproj.io/image-list: <image_alias>=<image_name>:<image_tag>
  argocd-image-updater.argoproj.io/<image_alias>.kustomize.image-name: <original_image_name>