Create large numbers of efficient and cost-effective CI pipelines based on Serverless Argo - Container Service for Kubernetes

Kubernetes clusters for distributed Argo workflows (workflow clusters) provided by Distributed Cloud Container Platform for Kubernetes (ACK One) are a fully-hosted Argo Workflows service that complies with the specifications of open source Argo Workflows. Workflow clusters are intended for large-scale computing-intensive jobs. Workflow clusters integrate Elastic Container Instance to support on-demand expansion and auto scaling. Workflow clusters also support preemptible instances to help you greatly reduce computing costs. This topic describes the benefits of the Continuous Integration (CI) pipeline solution based on workflow clusters, the issues that the solution resolves, the architecture, and the best practices.

Benefits

Argo Workflows is developed on top of Kubernetes. Both Argo Workflows and Kubernetes have been tested based on years of practices. With the auto scaling and concurrency capabilities, Argo Workflows is capable of handling large numbers of pipelines with faster speeds and lower costs. This allows developers to focus on business development and explore business value.

Argo Workflows can be seamlessly integrated with Argo CD, Argo Rollout, and Argo Event in the Argo ecosystem to empower CI scenarios.

You can create large numbers of cloud-native, efficient, and cost-effective CI pipelines based on Argo Workflows.

Compared with Jenkins, Argo Workflows provides the following benefits.

Comparison item	Argo Workflows	Jenkins
Whether is Kubernetes-native	Yes. Argo Workflows has advantages in managing Kubernetes containers: Auto container recovery from failures. Auto scaling. Role-Based Access Control (RBAC) permission management. It can be used with the Single Sign-On (SSO) capability of Argo to support multi-tenant isolation scenarios.	No.
Auto scaling, concurrency, and performance	Argo Workflows is usually used to handle large numbers of pipelines and supports auto expansion. Argo Workflows allows you to run multiple pipelines concurrently, which is more efficient.	Jenkins is suitable for handling small numbers of pipelines. The performance of Jenkins is compromised when the number of pipelines to be handled increases. Jenkins is weak in auto expansion. The concurrency level of Jenkins is low. Handling multiple pipelines can be time-consuming.
Costs	Supports auto scaling to minimize the costs. Supports preemptible elastic container instances to reduce costs.	Idle computing resource may exist in Jenkins, which cause resource waste.
Community and ecosystem	The Argo community is continuous growing and seamlessly integrated with Argo CD, Argo Rollout, and Argo Event in the ecosystem to empower CI scenarios.	The Jenkins community is mature and comes with rich resources and plug-in, which greatly reduce the technical gap for beginners. However, the O&M cost of plug-in updates and permission management keeps growing over time. Developers need to spend their energy on plug-in maintenance instead of developing features and exploring business value.

Issues resolved

Allows you to create Kubernetes-native CI pipelines.
The performance is not compromised when the number of pipelines increases.
Supports auto expansion and high concurrency, and allows you to quickly run pipelines. This reduces idle computing resources and avoids resource waste.
Supports RBAC permission management and can be used with the SSO capability of Argo to support multi-tenant isolation scenarios.
Supports preemptible elastic container instances to greatly reduce computing costs.

Architecture

Best practices

For more information, see Create CI pipelines for Golang projects in workflow clusters.

References

For more information about workflow clusters, see Overview of Kubernetes clusters for distributed Argo workflows (workflow clusters).