Use Alibaba Cloud container service Jenkins to implement continuous integration & build Docker image

Continuous integration (CI), as an important step for agile development, aims to achieve quick iteration of products while maintaining high quality. For every code update, you should run automated tests to check the accuracy of the code and functions. Only code that passes the automated tests can be used for subsequent delivery and deployment.[XM1] This article mainly introduces how to use Jenkins, one of the most popular CI tools, in combination with Alibaba Cloud Docker for automated testing, building images, and pushing.
The next demonstration shows how to achieve automated testing and build Docker images using Alibaba Cloud container service Jenkins to realize high-quality CI.
Scenario: After each code delivery to the nodejs project in GitHub, Alibaba Cloud container service Jenkins will automatically trigger the unit test. If the code passes the test, the process continues building the image and pushing the image to the target image repository. Then an e-mail notification will be sent with the final results. The procedure can be outlined in the figure below: The slave-nodejs is the slave node used to perform the unit test, build the image and push the image.
1 About Jenkins
Jenkins is an open-source CI tool developed based on the Java language. It monitors and triggers continuous and repeated jobs and features open source, support of multiple platforms and plug-in extensions, easy installation and interface-based management. Jenkins describes each step of work with jobs and the nodes are the environment for executing the projects. The master node is the default execution environment of Jenkins jobs, and the installation environment of the Jenkins application.
1.1 Master/slave
The master/slave concept in Jenkins is equivalent to the server and agent concepts. The master node provides web interfaces for users to manage jobs and slave nodes. Jobs can run on the master nodes or are allocated to slave nodes. One master node can be correlated to multiple slave nodes to provide services for different jobs or different configurations of the same job. You can configure multiple slave nodes to prepare independent unit tests for different projects and build their environments.
Tips: Jenkins job and project mentioned in this article both refer to one building unit and execution unit of Jenkins.
2 Use container service to deploy Jenkins application and slave nodes
Different applications require different dependencies for building and testing. The best practices are using different slave nodes containing corresponding runtime dependencies and tools for the testing and building. Through the slave images and example templates provided by Alibaba Cloud container service for Python, NodeJS, and Go among other environments, you can easily and quickly generate a Jenkins application and various slave nodes, configure node information in the Jenkins application and build desired execution nodes in the project to achieve the entire CI process.
2.1 Create Jenkins orchestration templates
Create a new template based on the content below and create an orchestration template.
jenkins:
image: 'registry.aliyuncs.com/acs-sample/jenkins:latest'
ports:
- '8080:8080'
- '50000:50000'
volumes:
- /var/lib/docker/jenkins:/var/jenkins_home
privileged: true
restart: always
labels:
aliyun.scale: '1'
aliyun.probe.url: 'tcp://container:8080'
aliyun.probe.initial_delay_seconds: '10'
aliyun.routing.port_8080: jenkins
links:
- slave-nodejs
slave-nodejs:
image: 'registry.aliyuncs.com/acs-sample/jenkins-slave-dind-nodejs'
restart: always
labels:
aliyun.scale: '1'

2.2 Create a Jenkins application and slave nodes using templates
Of course, you can directly use the Jenkins example templates that Alibaba Cloud container service provides to create a Jenkins application and slave nodes. After successful creation, the Jenkins application and slave nodes will show in the service list.
 Open the access endpoint provided by the container service, and you will be able to use the Jenkins application you just deployed.
 
3 Implement automated testing and automated building and pushing of images
3.1 Configure the slave container to the slave node of the Jenkins application
Open the Jenkins application and enter the system settings interface. Select Manage Nodes to configure corresponding parameters, as shown in the figure below.
Note 1: The label is the unique identifier of a slave node.
Note 2: The slave container and the Jenkins container run on the Alibaba Cloud platform at the same time. So you need to fill in a container node IP address inaccessible to external networks to isolate the testing environment.  Note 3: Use the Jenkins account and password in the Dockerfile when the slave-nodejs image is created as the credential.
URL of image Dockerfile: https://github.com/AliyunContainerService/jenkins-slaves/tree/master/jenkins-slave-dind-nodejs
3.2 Create a project to achieve automated testing
Create a new item, and select to build a freestyle software project.
Fill in the project name and select the project running nodes. In this example, the slave-nodejs-ut prepared as mentioned above is filled in.
 Configure the source code management and code branch. In this example, the source code is managed by GitHub.
 Configure the building trigger. In this example, the automated triggering of project execution is implemented using GitHub Webhooks & Services in combination.
 At last, add the Jenkins service hook in GitHub to complete the automated triggering implementation.
On the GitHub project homepage, click Settings tab, and select Webhooks & Services > Add Service in the left menu bar. Then select Jenkins (Git plugin) in the drop-down box.
Input ${Jenkins IP}/github-webhook/, eg. in the Jenkins Hook URL dialog box.
http://jenkins.cd****************.cn-beijing.alicontainer.com/github-webhook/
 The building step of execute shell type is added. You can compile shell scripts to execute the test.
 In this example, the commands are as follows:
pwd
ls
cd chapter2
npm test

3.3 Create a project to achieve automated building and pushing of images
Create a new item, and select to build a freestyle software project.
Fill in the project name and select the project running nodes. In this example, the slave-nodejs-ut prepared as mentioned above is filled in.
Configure the source code management and code branch. In this example, the source code is managed by GitHub.
(The above three steps are similar to that in Section 2, so I am not providing the illustrations here.)
Add the following trigger and set execution of automated image building only after the unit test is successful. At last, compile the shell scripts for building and pushing images. In this example, the commands are as follows:
cd chapter2
docker build -t registry.aliyuncs.com/qinyujia-test/nodejs-demo .
docker login -u ${yourAccount} -p ${yourPassword} registry.aliyuncs.com
docker push registry.aliyuncs.com/qinyujia-test/nodejs-demo

4 Automatically re-deploy the application
4.1 First deployment of application
Deploy the image created in Section 3 to the container service using the orchestration template and create the nodejs-demo application. The example is as follows:
express:
image: 'registry.aliyuncs.com/qinyujia-test/nodejs-demo'
expose:
    - '22'
    - '3000'
restart: always
labels:
    aliyun.routing.port_3000: express

4.2 Re-deploy automatically
Select the just-created nodejs-demo application and create the trigger. Add a line in the shell scripts in Section 3.3 and the URL is the created trigger link provided in the text above.
curl ‘https://cs.console.aliyun.com/hook/trigger?triggerUrl=***==&secret=***’

Change the example commands in Section 3.3 to the following:
cd chapter2
docker build -t registry.aliyuncs.com/qinyujia-test/nodejs-demo .
docker login -u ${yourAccount} -p ${yourPassword} registry.aliyuncs.com
docker push registry.aliyuncs.com/qinyujia-test/nodejs-demo
curl ‘https://cs.console.aliyun.com/hook/trigger?triggerUrl=***==&secret=***’

By now, Jenkins will automatically trigger the re-deployment of nodejs-demo application after the image is pushed.
5 Configure e-mail notifications on pushing results
If you want to push the results of the unit testing or image building via e-mail to related developers or project execution initiators, you can perform the following configurations.
First, on the Jenkins homepage, select System Management > System Settings to configure the e-mail address of the system administrator.
 Install the Extended Email Notification plug-in, configure the SMTP server information among others, and set the default list of e-mail recipients, as shown in the figure below.
 The above is the parameter settings of Jenkins application system. Below are the required configurations for the Jenkins project for the e-mail result notifications.
After you add the building in the Jenkins project, select Editable Email Notification type and fill in the e-mail recipient list.

 Add the e-mail sending trigger.
6 Summary
With the help of the Docker images and Compose templates, the container service can quickly create automated testing and building environments to simplify CI environment establishment and extension. The monitoring, logging, and storage functions provided by the container service can further offer convenient, uniform environment management with complete functions.
Thank @Mo Yuan’s great support.
To learn more about the slave nodes available in Alibaba Cloud container service, visit https://github.com/AliyunContainerService/jenkins-slaves. To learn more about the container service, visit https://www.aliyun.com/product/containerservice. [XM1]usually instead of automatic we use 'automated', ie. automated testing or automated tests, but automatic works, maybe they use this term more frequently in Ali group