Automated Web App Deployment from GitHub Push Using Alibaba Cloud EDAS

By Amit Maity, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.

Overview

Alibaba Cloud Enterprise Distributed Application Service (EDAS) is providing PaaS platform for hosting applications and micro services with high availability, monitoring and release management capabilities. To extend the EDAS capabilities, we can build continuous integration/deployment solution using EDAS and Jenkins.

In this tutorial, I'll demonstrate how a sample website can be hosted in EDAS platform and can be website upgraded automatically as soon as the new version of code is committed into GitHub.

Architecture

For this article, we'll be using a web application written in Java and war build of this application is deployed in EDAS. Also, source code of this application is maintained in GitHub repository.

Any changes of the GitHub repository will be pushed to Jenkins and build of Jenkins pipeline project will be automatically triggered to generate new war file with latest version of the source code and to deploy the war file using EDAS API into same EDAS application.

Prerequisites

This tutorial can easily be understood and followed if you are familiar with the below technologies and concepts,

Alibaba Cloud Environment
If you do not have access, get 1 year free access with $10 credit using the below link
Alibaba Cloud Free Account
Alibaba Enterprise Distributed Application Service
Click here to know more about EDAS
Jenkins
Click here to know more about Jenkins
Python Programing Knowledge

Server Setup

To develop and run the complete solution demonstrated here, you will need a host server (Alibaba Cloud ECS or any other Linux Server) with Jenkins, Python 2.7 and Docker pre-installed. Alternatively, refer below links to download and install Jenkins, Python 2.7 and Docker in this host.

Download/Clone Sample Web Application from GitHub repository into a directory of your choice in Jenkins host.

Sample Web Application Source Code

Purchase and Activate following services in your cloud account,

Object Storage Service (OSS)
Enterprise Distributed Application Service (EDAS)
Computing Server (ECS)

Configure OSS, ECS and EDAS

Please decide a single region (Preferably China- Beijing) for configuring all the required Alibaba cloud services before you proceed. This will help you to avoid any technical challenges of integrating multi region services.

Create a new bucket in OSS console to store web application java war file, which will be deployed as EDAS application. Make sure bucket access policy is set to public read. Otherwise, EDAS instance may not be able to fetch the war file for deployment.
Keep a note of bucket name and OSS end point. Later you'll have to update the configuration file of this solution with bucket name and OSS end point.
Create ECS Instance with minimum 1 GB memory and CentOS 7.4 OS in Alibaba cloud console. Choose region same as OSS region. Also, create a new VPC network for this host and allocate public IP.
Once ECS instance is up and running, install EDAS agent after connecting to the host using VNC. Below is the command to be executed for installing agent. Please change the region part depending on the region of ECS host. In this example"e cn-beijing" is the region of our ECS.
```
wget -q -O /root/install.sh http://edas-bj.oss-cn-beijing-internal.aliyuncs.com/intl/install.sh && sh /root/install.sh -full
```
Setup EDAS cluster using Products->EDAS->Resources->Clusters. Enter the following details in the create cluster form. Assign VPC network of ECS instance to this EDAS cluster.
Region: Same as ECS host
Cluster Name: As your wish
Type: ECS
Network Type: VPC
VPC Network: VPC network of ECS host
After cluster is created, open cluster details page and add cluster host
Add cluster host form, select import ECS option and click on Synchronize ECS button to get the list of ECS host. Select the host and follow the remaining steps to complete cluster setup.
Now, create EDAS application from EDAS->Application Management->Applications with the following configuration option,
Cluster: Name of the cluster created in previous step
Runtime Environment: Use default value
Java Environment: JDK8
Note: you may have to wait for few mins after entering all the details in the form for next step button to be enabled.

In next step, choose deployment method as war and select instance from the list of cluster host. Use Deploy Now option to deploy the sample web application war file by using https://warfiles.oss-cn-beijing.aliyuncs.com/webapptest.war url.
After application is created, make a note of application id as it needs to be updated in the configuration file.
After successful deployment, check the EDAS application is running or not as shown below. If it's in running status, then try accessing website using url http://< public IP of ECS Host>:8080 (default http port). You should be able to see below webpage if everything is working as expected.

Sample Web Application Page

Jenkins Setup

Assuming, you have already configured Linux host for running Jenkins and able to access Jenkins home page with admin privilege. Also, all the suggested plugins are installed during Jenkins initial setup. If you are struggling to get the Jenkins up and running, then refer Jenkins Install link for guidance. Also, please ensure the firewall rules are properly configured in your linux host network to allow inbound internet traffic on port 443.

Let's now follow the remaining configuration steps to setup Jenkins for automated build and deployment.

For easier demonstration for this tutorial, disable Jenkins security configuration. But, this is not recommended for production environment.
Jenkins->Manage Jenkins->Configure Global Security
Disable security as below
(Warning: you should avoid this step for production and setup appropriate security settings to avoid unauthorized access)
Verify if below Jenkins plugins are already installed or not from plugin manager. If not, then install them.
Jenkins->Manage Jenkins-> Manage Plugins
Click on available tab and enter git in filter field. Select following plugins and install. If any of these plugins not appearing in the search results, then it may have been already installed.
Git plugin
GitHub
GitHub Integration Plugin
Update the following configurations in Jenkins before setting up the build pipeline project.
Jenkins->Manage Jenkins->Configure systems
Add GitHub Server as below

Update GitHub Pull Requests with github project url
Create Pipeline project for automated build.
Jenkins->New Item
Enter Project Name and select Pipeline type and then click OK

Update below configurations of this project as follows,
Jenkins->WebAppBuild->Configure
Enter GitHub repository url in GitHub project

In Build Triggers section select GitHub hook trigger for GITScm polling and Poll SCM option

In Advanced Project Options, select Pipeline definition as follows,

Definition: Pipeline script from SCM

SCM: Git

Repositories:

Repository URL: https://github.com/ username>/.git
Credentials: GitHub credentials
Script Path: Jenkinsfile

Note that Jekinsfile is already available in https://github.com/itexpertshire/webapptest/ Create a fork of my project in your github account and write your own pipeline script by modifying Jenkinsfile.

Walkthrough of Jenkinsfile:

pipeline {

agent {

docker {

image 'openkbs/jre-mvn-py3'

args '-u 0 --name=mycontainer -v /home/bitnami/GitHub/webapptest/jenkins/scripts:/root/.m2'

}

}

stages {

stage('Build') {

steps {

sh 'mvn -B -DskipTests clean package'

}

}

stage('Python Library Install') {

steps {

sh './jenkins/scripts/pythoninstall.sh'

}

}

stage('Deliver') {

steps {

sh './jenkins/scripts/deliver.sh'

}

}

}

}

1. Pipeline script will be executed in a Docker container. Docker image openkbs/jre-mvn-py3 is used here for this purpose. mvn, jre and python is already installed in this docker image.

Argument –u 0 is used for running the docker container with root privilege.

To use Jenkins stage scripts inside Docker container, Github project is cloned in a folder and local jenknins folder is mapped to docker container directory path using below syntax

-v /home/bitnami/GitHub/webapptest/jenkins/scripts:/root/.m2

Here github project is cloned into GitHub folder in Jenkins server.

According to your installation, replace /home/bitnami/GitHub/webapptest/jenkins/scripts path with the local path in your Jenkins host pointing to Jenkins pipeline stage scripts location. Don't forget to mention absolute path here.

2. Build stage, invokes mvn tool to build the war file of the Github project source code

3. Python Library Install stage will install required python libraries in docker for calling EDAS API to deploy the web application code into EDAS platform.

Following python libraries need to be installed,

aliyun-python-sdk-core==2.12.0

oss2

configparser

4. Deliver.sh script will execute python script EDAS.py which will upload the java war file generated from 'Build' stage into OSS and then deploy into EDAS application. Detail explanation of EDAS.py is given in later section.

GitHub Setup

Assuming, your Jenkins is up and running and can accept http/https request from external applications.

In your forked project, configure webhook settings.

GitHub->{Your Project}->Settings->Webhooks

Add webhook records as shown below. Url should be replaced with your Jenkins host public ip address.

After saving the record, webhook url should be marked as verified with green icon. If not, then verify the configurations in Jenkins.

EDAS Python App Walkthrough

Below is the walkthrough of python script which will use EDAS API to deploy the java application.

a. edas.cfg

This file maintains all required configuration values for running the python script. Update the configuration parameters in this file according to your cloud account/service details.

AccessKeyId = <Cloud Account Access Key>

AccessKeySecret = < Cloud Account Access Secret Key >

Endpoint = <EDAS End points url> i.e. edas.cn-beijing.aliyuncs.com

Region = <EDAS region>  i.e. cn-beijing

AppId = <EDAS application id>

OSSEndpoint = <OSS End Point url> i.e. oss-cn-shenzhen.aliyuncs.com

OSSBucketName = <OSS Bucket Name> i.e. warfiles

b. common.py

This script is for parsing the required configuration parameter values from edas.cfg and return the values to main python scripts EDAS.py for invoking EDAS application deployment API.

# Read config.cfg and parse the configuration parameters

parser = ConfigParser.ConfigParser()

parser.read(cfg_fn)

# Assign each of the configuration parameters values to separate varaiables

accessKeyId = parser.get("Base", "AccessKeyId")

accessKeySecret = parser.get("Base", "AccessKeySecret")

endpoint = parser.get("Base", "Endpoint")

.

.

.

return accessKeyId,accessKeySecret,endpoint,region . . .

c. EDAS.py

Core logic of invoking EDAS API is used in this script. Alibaba cloud has offered /pop/v5/changeorder/co_deploy api for deploying application into EDAS platform. This API accepts war file location in url format. So, war file should be uploaded into Alibaba OSS bucket and OSS url of the war file will be passed as parameter to EDAS api. Read access policy of OSS bucket should be set to public.

Script will take local war file location as input parameter and will upload the war file into OSS bucket using the upload_war function. resumable_upload api of OSS python library is used for uploading the file into bucket. After successful upload it' generate the complete OSS url of the war file.

def upload_war(warfilepath):

auth = oss2.Auth(accessKeyId, accessKeySecret)

bucket = oss2.Bucket(auth, ossendpoint, ossbucketname)

oss2.resumable_upload(bucket, os.path.basename(warfilepath), warfilepath)

warurl = "https://"+ossbucketname+"."+ossendpoint+"/"+os.path.basename(warfilepath)

"deploy" function is defined in this script to call api for deploying the war file.

def deploy(warurl):

EDAS api accepts request in POST method and api version has to be set to '2017-08-01'

request = CommonRequest()

request.set_accept_format('json')

request.set_method('POST')

request.set_protocol_type('https') # https | http

request.set_domain(endpoint)

request.set_version('2017-08-01')

Set all the required api parameters as follows,

RegionId : Region where application is setup
AppId : application id of the EDAS application
PackageVersion : Unique numeric value
DeployType: url
WarUrl: OSS url of war file
GroupId: all

request.add_query_param('RegionId', region)

request.add_query_param('AppId',appid )

request.add_query_param('PackageVersion',packageversion)

request.add_query_param('DeployType','url')

request.add_query_param('WarUrl',warurl)

request.add_query_param('GroupId','all')

Pass the deployment api name - /pop/v5/changeorder/co_deploy in http request

request.set_uri_pattern('/pop/v5/changeorder/co_deploy')

Run and Test

Before demonstrating the real time deployment after committing change to Github, let's first ensure following checks are passed.

1. EDAS application is accessible over internet by browsing http url of the application – http:// IP of EDAS host instance>:8080

2. Github webhook is verified

3. Manually trigger deployment of your Jenkins pipeline project for the first time to ensure pipeline is executed without any error. (If the error is due to permission denied, then run below commands)

sudo chmod 755 /var/run/docker.sock

sudo gpasswd -a tomcat docker

newgrp docker

Now let's clone the forked github repository in a directory of Jenkins host. Let's assume, it's cloned in a directory named GitHub.

Update edas.cfg file ./GitHub/webapptest/jenkins/scripts
Update ./GitHub/webapptest/Jenkinsfile file with the absolute path of python scripts which is cloned in your Jenkins host.
Navigate to website source code folder and modify the index.jsp and then commit the changes to github repository
Open Jenkins pipeline project in a browser. You should see the build of the pipeline project is already started. Wait for complete execution of the project.
Open EDAS web url in your browser and verify the changes in the website is reflecting or not.

Community

Automated Web App Deployment from GitHub Push Using Alibaba Cloud EDAS

Overview

Architecture

Prerequisites

Server Setup

Configure OSS, ECS and EDAS

Jenkins Setup

GitHub Setup

EDAS Python App Walkthrough

Run and Test

Read previous post:

Read next post:

Alibaba Clouder

You may also like

Comments

Alibaba Clouder

Related Products

EDAS

ECS(Elastic Compute Service)

OSS(Object Storage Service)