Alibaba Cloud ACK One GitOps: Streamline Multi-Cluster Application Delivery

This article provides an overview of how Alibaba Cloud's ACK One GitOps facilitates continuous deployment and management of applications across multip...

By Jing Cai

Overview of GitOps

The core of GitOps for application distribution is to use the Git repository as the only source for application deployment, and continuously synchronize the application states in the Git repository to the Kubernetes cluster. According to the results of the CNCF microsurvey [1] on GitOps usage trend evaluation published in late 2023, the data shows that GitOps has become the top choice of most developers for fast, consistent, and secure delivery. CNCF announced the graduation of Argo [2] in December 2022, which marks that Argo has been a stable and mature project and allows more Kubernetes-based users to use Argo CD to implement GitOps CD.

Benefits of GitOps:

Rapid delivery and continuous deployment of applications

GitOps improves the deployment speed through automated building and deployment and automatically synchronizes the application states in Kubernetes clusters and Git repositories to keep states consistent.

High reliability

GitOps uses the Git repository as the only source of application templates and provides version control, fast rollbacks, and auditing, enabling greater deployment consistency.

High security

No permissions of Kubernetes clusters are required for developers to use GitOps. Only permissions of Git repositories are required. The automated process is more secure and reduces the risk of errors in manual operations.

Ease of learning

Git is easy to be accepted by developers and integrated with no additional learning costs.

ACK One GitOps

ACK One GitOps provides GitOps continuous delivery capability for multi-cloud, multi-cluster, and hybrid cloud applications. Through the fully managed and open-source Argo CD project, ACK One GitOps integrates ACK One multi-clusters and Alibaba Cloud Resource Access Management (RAM) and single sign-on (SSO) to support out-of-the-box Argo CD capabilities. It provides a complete and secure GitOps CD experience for applications among clusters and allows you to implement continuous hybrid cloud application deployment across clusters in a fast, consistent, and secure manner.

The following figure shows the ACK One GitOps capabilities:

The developer uses Argo CD UI, CLI, or Go SDK to create an application or ApplicationSet and deploys the application.
The developer updates the new image to the image repository and the new tag is updated to yaml in the Git repository after the Argo CD Image Updater detects the image update.
Argo CD regularly synchronizes the application states of the Git repository to cloud clusters and on-premises clusters (Secret management in GitOps is implemented based on KMS).
The state changes during application synchronization are notified in real time by DingTalk.

Benefits of ACK One GitOps:

ACK One GitOps is integrated with open-source Argo CD and provides a CLI and a UI that offer the same user experience as the CLI and the UI provided by Argo CD.
ACK One GitOps is O&M-free and provides out-of-the-box features.
ACK One GitOps provides a separate Argo CD console that is integrated with Alibaba Cloud RAM user and SSO of the RAM role and supports Argo CD multi-tenancy permission management.
ACK One GitOps supports hybrid cloud application distribution across clusters. Argo CD is automatically enabled for the clusters that are associated with the Fleet instances of ACK One. As the target clusters, the associated clusters use GitOps for application distribution.
ACK One GitOps supports Argo CD ApplicationSet to improve the user experience of application distribution across clusters.
ACK One GitOps publishes multi-cluster applications more securely, supports Secret management in GitOps, and accesses sub-clusters at the ServiceAccount level.

Rapid Deployment of Hybrid Cloud and Multi-Cluster Applications

There are three steps to implement fast application deployment among clusters in hybrid cloud scenarios:

Register the IDC cluster to the cloud through ACK One Registered Clusters [3].
Manage cloud ACK clusters in multiple regions and on-premises IDC clusters through ACK One Fleet [4].
Implement fast and automated application deployment among cloud and on-premises clusters through ACK One GitOps.

Argo CD is automatically enabled for the clusters that are associated with the Fleet instances of ACK One. As the target clusters, the associated clusters use GitOps for application distribution. This simplifies the application distribution process across clusters.

At present, some ACK One customers are using ACK One to manage dozens of hybrid cloud clusters and on-premises hybrid cloud clusters, and using GitOps to fast deploy thousands of applications (Argo CD Application). The Argo CD ApplicationSet is used to improve the management efficiency of applications across clusters. The following is a yaml example of ApplicationSet from the Argo CD community [5], which can easily deploy multiple applications to multiple clusters:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: cluster-git
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    # matrix 'parent' generator
    - matrix:
        generators:
          # git generator, 'child' #1
          - git:
              repoURL: https://github.com/argoproj/argo-cd.git
              revision: HEAD
              directories:
                - path: applicationset/examples/matrix/cluster-addons/*
          # cluster generator, 'child' #2
          - clusters:
              selector:
                matchLabels:
                  argocd.argoproj.io/secret-type: cluster
  template:
    metadata:
      name: '{{.path.basename}}-{{.name}}'
    spec:
      project: '{{index .metadata.labels "environment"}}'
      source:
        repoURL: https://github.com/argoproj/argo-cd.git
        targetRevision: HEAD
        path: '{{.path.path}}'
      destination:
        server: '{{.server}}'
        namespace: '{{.path.basename}}'

In addition, consistent deployment of applications across clusters is also extremely necessary in many scenarios. One case is the construction of a hybrid cloud disaster recovery system based on ACK One [6]. It is necessary to ensure that the services of applications in the cloud ACK clusters and on-premises IDC clusters are consistent. You can use GitOps to implement consistent application deployment across clusters. In addition, automated deployment can avoid the risks of errors in manual deployment.

Multi-tenancy Permission Management

When users from multiple teams use the GitOps system together, multi-tenancy permission control is often required. ACK One GitOps integrates Alibaba Cloud RAM user and SSO of the RAM role. Multi-tenancy permission management mainly includes the following aspects:

Permission management for ACK One Fleet supports Alibaba Cloud RAM primary accounts or permission administrators to grant RAM users and RAM roles RBAC permissions on Fleet and sub-clusters (including Application resources).
Permission management supports Alibaba Cloud RAM primary account or permission management to grant RAM users and RAM roles Argo CD RBAC permissions [7] in the argocd-rbac-cm.
Permission management supports Alibaba Cloud RAM primary account or permission management. You can use Argo CD Projects to manage the RBAC permissions of RAM users and RAM roles on target clusters, repositories, and applications.

The following figure shows an example of a multi-tenancy configuration:

The administrator creates one project for team-one and one project for team-two.
- The project of team-one binds with Application 1 and team-one Git Repo.
- The project of team-two binds with Application 2 and team-two Git Repo.
When team-one wants to access Application 2 of team-two, it will be rejected by Argo CD because team-one has not been granted the permissions for Application 2.

The following example of an Argo CD project shows how an administrator assigns a RAM user or role the read-only permission on an application, which can be created by using the Argo CD UI:

27*02 is the ID of the RAM user or role.
27*02 can only execute get on application 1 in the demo project (create and delete cannot be executed).

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: demo
  namespace: argocd
spec:
  destinations:
  - name: '*'
    namespace: '*'
    server: '*'
  roles:
  - groups:
    - "27***02"
    name: test
    policies:
    - p, proj:demo:application1, applications, get, demo/*, allow
  sourceRepos:
  - https://github.com/AliyunContainerService/gitops-demo.git

The following example shows how to configure a RAM user or role in argocd-rbac-cmConfigMap to have all operation permissions on projects:

data:
  policy.csv: |
    p, role:project-admin, projects, *, *, allow
    
    g, "27***02", role:project-admin     # Bind the ArgoCD role: project-admin permission to RAM User/Role "27***02".
scopes: '[uid]'

Secret Management in GitOps

It is necessary to securely and efficiently manage sensitive information such as token authorization, user names, user passwords, and private keys in multi-cluster GitOps. ACK One GitOps provides a solution that combines KMS to implement Secret management [8] in GitOps.

Here is a brief introduction to the solution based on ACK Secret Manager:

Install the ACK Secret Manager components first in the sub-cluster where you want to use the Secret.
Add secrets in KMS Secrets Manager.
Add an ExternalSecretyaml to the application in the Git repository and reference the Secret in the Deployment, as shown in the yaml example on the left of the following figure.
Use ACK One GitOps to synchronize the application that contains ExternalSecret to the sub-cluster. The installed ACK Secret Manager components will create a Secret based on the ExternalSecret for Deployment use.

ACK One: Building a Hybrid Cloud Zone-Disaster Recovery System

References

[1] Results of the CNCF Microsurvey
https://www.cncf.io/blog/2023/11/07/cncf-gitops-microsurvey-learning-on-the-job-as-gitops-goes-mainstream/
[2] CNCF Announced the Graduation of Argo in December 2022
https://www.cncf.io/announcements/2022/12/06/the-cloud-native-computing-foundation-announces-argo-has-graduated/
[3] ACK One Registered Clusters
https://www.alibabacloud.com/help/en/ack/distributed-cloud-container-platform-for-kubernetes/user-guide/overview-9
[4] ACK One Fleet
https://www.alibabacloud.com/help/en/ack/distributed-cloud-container-platform-for-kubernetes/user-guide/fleet-management-overview
[5] From the Argo CD Community
https://argo-cd.readthedocs.io/en/stable/operator-manual/applicationset/Generators-Matrix/
[6] Construction of a Hybrid Cloud Disaster Recovery System Based on ACK One
https://www.alibabacloud.com/help/en/ack/distributed-cloud-container-platform-for-kubernetes/use-cases/construction-of-hybrid-cloud-disaster-recovery-system-based-on-ack-one
[7] Argo CD RBAC Permissions
https://www.alibabacloud.com/help/en/ack/distributed-cloud-container-platform-for-kubernetes/user-guide/manage-users-based-on-gitops?spm=a2c4g.11186623.0.0.43be734fVamflQ#6f2facdfddgxr
[8] A Solution that Combines KMS to Implement Secret Management in GitOps
https://www.alibabacloud.com/help/en/ack/distributed-cloud-container-platform-for-kubernetes/user-guide/secret-management

Community

Alibaba Cloud ACK One GitOps: Streamline Multi-Cluster Application Delivery

Overview of GitOps

Benefits of GitOps:

ACK One GitOps

Rapid Deployment of Hybrid Cloud and Multi-Cluster Applications

Multi-tenancy Permission Management

Secret Management in GitOps

Related Articles

References

Read previous post:

Read next post:

Alibaba Container Service

You may also like

Comments

Alibaba Container Service

Related Products

Alibaba Cloud Flow

DevOps Solution

ACK One

Function Compute