By Jing Cai
Argo Workflows is an open-source, cloud-native workflow engine for orchestrating jobs on Kubernetes. It easily automates and manages complex workflows on Kubernetes. It is also suitable for various scenarios such as scheduled tasks, machine learning, ETL and data analysis, model training, data flow pipeline, and CI/CD.
Kubernetes Jobs only provides basic task execution without defining step dependencies and order, workflow templates, a visual interface, or workflow-level error handling. This makes it inadequate for business scenarios such as batch processing, data processing, scientific computing, and continuous integration.
As a graduate project of CNCF, Argo Workflows is used in many scenarios, with Continuous Integration (CI) being one of its key application areas.
CI is an important part of the software development lifecycle, and a good CI system can greatly improve your delivery efficiency. Based on Alibaba Cloud EventBridge and ACK One Serverless Argo Workflows, you can build an event-driven automated CI Pipeline to deliver your applications easily, quickly, efficiently, and cost-effectively, enabling automatic builds and deliveries upon committing code.
EventBridge is a serverless event bus service provided by Alibaba Cloud. You can connect Alibaba Cloud services, custom applications, and SaaS applications to EventBridge in a standardized and centralized manner. It routes events among these applications using the standardized CloudEvents 1.0 protocol, helping you build loosely coupled and distributed event-driven architectures. Using EventBridge as an event-driven engine offers the following advantages:
1. Availability: It enables high-throughput event publishing and consumption, auto-scaling to handle sudden traffic spikes, disaster recovery deployment within the same region with strong data reliability, and multi-region deployment with low network latency.
2. Usability: It supports fully managed service with no need for deployment, operation, or maintenance, a wide range of event sources, including official Alibaba Cloud event sources and custom event sources, and seamless migration to the cloud with full compatibility with CloudEvents 1.0.
3. Security: It supports granting permissions from Alibaba Cloud accounts to RAM users, permissions across accounts, and different access permissions and HTTP/HTTPS protocols to different RAM users.
ACK One fully manages Argo Workflows, which improves its performance, stability, observability, and O&M (Operation and Maintenance) capabilities. This article will introduce how to build an event-driven automated CI Pipeline based on EventBridge and ACK One Serverless Argo Workflows, achieving a CI system driven by Git events for building and pushing Docker Image and running pre-build tests.
An event-driven automated CI Pipeline consists of two parts:
After the image is built, you can use the CD system (such as ACK One GitOps) to synchronize the new Image tag to the Kubernetes cluster.
EventBridge contains event sources, event buses, event rules, and event targets:
• Event Source: Publish event messages generated by applications such as Alibaba Cloud services and custom applications to an event bus. In this article, the event source is connected to GitHub.
• Event Bus: Store received event messages and route them to event targets according to event rules.
• Event Target: Consumers of the events. In this article, the event target is the ACK One Serverless Argo Workflows.
CI Pipeline based on the workflow cluster primarily involves three steps: Clone and switch branches of the Git code; run test cases for Go projects; and build and push the image to ACR EE. BuildKit Cache is used to accelerate image builds.
The main steps for CI driven by Git events are as follows:
(Note that the namespace needs to match the Workflow resource.)
The Apps Code Repo example is the echo-server project. If you use this sample code, you need to fork it into your own Git repository. If it is a private Git repository, you can refer to Clone Private Git Repository in CI Pipeline to modify the WorkflowTemplate accordingly.
Below are the configuration steps for the CI Pipeline driven by GIt events for the specified branch (release-v1) of the echo-server project.
1) Activate EventBridge and grant permissions.
2) Manage custom event buses.
3) Integrate Github.
https://www.alibabacloud.com/help/en/eventbridge/use-cases/integrate-github
4) Edit event rules (optional).
• Event pattern: The following settings only trigger changes from the release-v1 branch.
{
"source": [
"github.event"
],
"data": {
"body": {
"ref": [
"refs/heads/release-v1"
]
}
}
}
5) Event target:
• Service Type
: Select Container Service for Kubernetes
.
• KubeConfig File
: Enter the KubeConfig file of the ACK One workflow cluster.
• YAML Configuration
: Template.
• Variables
: Add the workflowName parameter, configured as the event id as follows.
{
"workflowName": "$.id"
}
6) Template
: Fill in the Workflow CI CR.
• You need to specify name and namespace instead of generateName, and declare the default namespace.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
name: ci-go-v1-eb-${workflowName}
namespace: default
labels:
workflows.argoproj.io/workflow-template: ackone-ci
spec:
arguments:
parameters:
- name: repo_url
value: https://github.com/ivan-cai/echo-server.git
- name: repo_name
value: echo-server
- name: target_branch
value: release-v1
- name: container_image
value: "test-registry.cn-hongkong.cr.aliyuncs.com/acs/echo-server"
- name: container_tag
value: "v1.0.0"
- name: dockerfile
value: ./Dockerfile
- name: enable_suffix_commitid
value: "true"
- name: enable_test
value: "true"
workflowTemplateRef:
name: ci-go-v1
clusterScope: true
1. Commit the GitHub code to trigger the automation process (according to the preceding configuration, you need to modify the release-v1 branch code to trigger the process).
You can view event traces in Event Tracking.
2. View the execution topology of the newly created Workflow in the ACK One workflow cluster or the workflow console.
3. View the pushed images in ACR EE.
As a fully managed Argo workflow service, ACK One Serverless Argo Workflows helps you implement a larger-scale, faster-running, and lower-cost CI Pipeline. Combined with EventBridge, which offers advantages in availability, usability, and security, you can deliver your applications simply, quickly, efficiently, and cost-effectively, enabling an automated CI system that triggers builds and deliveries upon committing code.
Best Practices for Configuring Throttling Rules for Applications in ASM
155 posts | 29 followers
FollowAlibaba Container Service - July 24, 2024
Alibaba Container Service - August 30, 2024
Alibaba Cloud Native Community - March 11, 2024
Alibaba Container Service - July 1, 2024
Alibaba Cloud Native - November 3, 2022
Alibaba Developer - September 7, 2020
155 posts | 29 followers
FollowVisualization, O&M-free orchestration, and Coordination of Stateful Application Scenarios
Learn MoreServerless Application Engine (SAE) is the world's first application-oriented serverless PaaS, providing a cost-effective and highly efficient one-stop application hosting solution.
Learn MoreAn enterprise-level continuous delivery tool.
Learn MoreProvides a control plane to allow users to manage Kubernetes clusters that run based on different infrastructure resources
Learn MoreMore Posts by Alibaba Container Service