Create CD pipelines with Kubernetes, Ansible, and Jenkins

Date: Oct 27, 2022

Related Tags：1. Continuous Integration & Continuous Delivery Solution
2. CI/CD and the CI/CD Pipeline: An In-depth Overview


Abstract: CI/CD (CI full name Continuous Integration, continuous integration; CD full name Continuous Deployment, continuous deployment) The term is often used with other terms such as DevOps, Agile, Scrum, and Kanban, automation, etc.

What problem does CI/CD solve?

Sometimes people just see it as part of a workflow without really understanding what it is or what it means to adopt it. Young DevOps engineers often take CI/CD for granted, they may not have seen a "traditional" software release cycle, and as a result, don't get the benefits of CI/CD.

CI/CD stands for Continuous Integration, Continuous Delivery and Deployment. A team that does not implement CI/CD must go through the following stages when creating a new software product:

A product manager (on behalf of the customer) provides the features that the product should have and the behavior it should achieve. This segment must be as comprehensive and specific as possible.

Developers, with the help of business analysts, write code, run unit tests and commit the results to a version control system such as git.

After the development phase, the project will be moved to QA (Quality Assurance). Run several tests against the product, such as user acceptance tests, integration tests, performance tests, etc. During this time, no changes will be made to the codebase until the QA phase is complete. If there are any bugs, they are returned to the developers to fix, and the revised product is then handed over to QA again.

After QA is over, the operations team deploys the code to production.

But the above workflow has many disadvantages:

First, it takes a long time between when a product manager makes a request and when a product is ready to go into production.

It's not easy for developers to fix bugs in code written a month or more ago. Remember that bugs are only discovered after the development phase ends and the QA phase begins.

When urgent code changes are required (such as serious bugs that require hotfixes), the QA phase is often shortened because it needs to be deployed as quickly as possible.

With little communication and collaboration between different teams, when bugs happen, people start blaming each other. Each person only cares about his own part, ignoring the common goal.

CI/CD solves the above problems by introducing automation. Every time a code change is pushed to the version control system, it is tested and then deployed to the stagingUAT environment for further testing before it is deployed to production for users to consume. Automation ensures that the entire process is fast, reliable, repeatable, and reduces the chance of errors.

That being the case, what is CI/CD?

Some related books have been introduced very clearly. How, why and when to implement CI/CD in your infrastructure. However, we always prefer a little less theory and a little more practice. That being the case, we'll briefly describe the automated steps that should be performed after committing code changes:

Continuous Integration (CI): QA is not included in the first step. In other words, it is not concerned with whether the code provides the features requested by the client. Instead, it focuses on ensuring code quality. Through unit testing, integration testing, developers can be notified of any code quality issues. We can further enhance the code quality with code coverage and static analysis.

User Acceptance Testing: This is the first part of the CD process. During this phase, the code is automatically tested to ensure that it meets the client's expectations. For example, a web application might work without any errors, but the client expects visitors to go through a landing page with promotions before getting to the home page. The current code will take the visitor directly to the main page, which deviates from the actual requirements of the customer, and the UAT test can point out such problems. In a non-CD environment, this work needs to be done manually by QA testers.

Deployment: This is the second part of the CD process. It involves making changes to the server/container that will host the application so that it reflects the updated version. This work should be done in an automated fashion, preferably through a configuration management tool such as Ansible, Chef, or Puppet.



What is a pipeline?

A pipeline is an advanced term with a simple concept. When you achieve a common goal, you may need to execute many scripts in a certain order, and these scripts are collectively called "pipes".

For example, in Jenkins, a pipeline may contain one or more stages that must all complete for a successful build to occur. Using multiple stages enables you to visualize the entire process, understand how long each stage takes, and clearly see where the build fails.



Lab: create pipeline for golang app

In this lab, we will build a continuous delivery (CD) pipeline. We are using a very simple application written in Go language. For simplicity, we only run one type of test on the code. The prerequisites for this experiment are as follows:

A running Jenkins instance. Can be cloud instances, virtual machines, bare metal or docker containers. It must be publicly accessible from the network so that the repository can connect to Jenkins via a webhook.

Image registry: You can use Docker Registry, a cloud-based product like ECR or GCR, or even a custom registry.

ECR: https://aws.amazon.com/ecr/

GCR: https://cloud.google.com/container-registry/

A GitHub account. Although we used GitHub in this example, the process can be done with minor modifications as with other repositories such as Bitbucket.