Cloud O&M undoubtedly involves a series of resource deployments. For example, you need to create a VPC, a VSwitch, a security group, and an ECS instance in sequence to use an ECS instance. Then, you need to create a server load balancer, database, and multiple cloud server instances to create a cluster.
As the business scale continues to grow and the IT systems and environments become increasingly complex, it is not advisable to create resources one by one manually. Many people are resorting to tools that support automated resource deployment and configuration.
Based on the concept of Infrastructure as Code (IaC), this article will share how to use automated orchestration tools to implement automated deployment and improve the efficiency of cloud O&M.
In terms of the resource deployment on the cloud, when the on-cloud O&M is performed in a manual or semi-manual manner, most of the work involved is done by selecting the specifications of the specific resource in the console. CLI (such as aliyun-cli) or SDKs are call interfaces directly to create resources. However, as enterprises expand their cloud business continually, they must solve five problems no matter what solutions they choose:
The reason behind these problems is that the deployment of resources is not automated. However, these problems also urge us to think about ways to solve these pain points so the entire resource deployment process can be automated.
Before automated deployment, it's better to check the cloud service resources that need to be created, such as VPCs, VSwitches, and ECS instances. They are cloud infrastructure compared to applications, such as Web services. If you think of the Infrastructure as the code that defines information, such as products, specifications, and quantities, can this code manage the entire infrastructure?
This is the concept of Infrastructure as Code (IaC), which regards Infrastructure configuration as software programming. In Kief Morris' book Infrastructure as Code, he defines IaC as:
"Infrastructure as Code is a way to use new technologies to build and manage dynamic infrastructure. It regards infrastructure, tools, and services as well as infrastructure management as a software system and adopts software engineering practices to manage changes to the system in a structured and secure manner."
With the concept of IaC, O&M personnel can deploy and manage infrastructure quickly:
When it becomes more convenient to deploy and manage the infrastructure, the preceding pain points of manual or semi-manual O&M can be addressed:
There are many automatic IaC deployment tools, including third-party resource orchestration tools and cloud-native resource orchestration tools provided by cloud service providers. There are four automatic configuration and orchestration tools introduced below:
Here are some suggestions on how to select an automatic configuration and orchestration tool:
How can orchestration tools be used for automated deployment and management?
For O&M personnel, the threshold for the use of an IaC automated deployment tool is not high, and the procedure is very simple, which is mainly about writing and using templates. The following describes the considerations for using a template and how to make better use of the tools and improve O&M efficiency.
After creating templates, users can use the corresponding automated deployment tools to convert the templates into real resources. The preceding orchestration tools can parse resource dependencies and create resources in sequence. In addition, users can create instances in parallel on resources that are independent of each other.
When templates are implemented in the entire infrastructure after O&M, DevOps becomes easier. We can use version management tools, such as Git, to manage templates that describe the current infrastructure. Alibaba Cloud Apsara DevOps, AWS CodePipeline, and Jenkins can create a pipeline that starts from code submission triggering to manual card points and goes to resource stack deployment. As such, the entire infrastructure management becomes more agile and automated.
Figure 1: A Flowchart of Infrastructure Changes
After each template change, the branch content from the local repository is pushed to a remote repository for review.
If the review fails, the template is modified, and the review is re-initiated. If the review passes, the pipeline is triggered automatically.
The pipeline triggers the manual review and notifies the superior administrator to check the change. If not, terminate; if agreed, proceed to the next step.
If the template is submitted for the first time, create the resource stack, namely, create the infrastructure. Otherwise, update the resource stack, namely, update the infrastructure.
The IT infrastructure does not remain unchanged. As businesses change, we may have to deal with scaling, and the whole architecture may change as well. Based on the IaC philosophy, we only need to describe the latest infrastructure configurations without worrying about how to make changes. Even so, we need to know what will happen before changes. The change set of Alibaba Cloud ROS and AWS CloudFormation as well as the execution plan of Terraform allow us to understand the changes in advance.
For example, due to business changes, an ECS instance is added to the Alibaba Cloud platform based on the architecture in Figure 1, and Server Load Balancer is used for load balancing between the two ECS instances. After compiling a new template, you can use the change set to detect changes. The following figure shows a change example from Alibaba Cloud ROS:
After the confirmation, the change can be executed. Then, the automated orchestration tool will update the entire infrastructure and decide which resources to add, change, or delete according to the changes in the template.
Despite the use of automated orchestration tools to deploy resources, some personnel may still modify the attributes of some resources in the infrastructure in non-standard ways, such as through the console or API. The consequence is that the actual situation of the resource is different from the resources defined in the template. An excellent automated orchestration tool can detect the differences between the actual attributes of infrastructure and the attributes defined in templates. It can also correct templates or actual resources based on the different results, so the template is consistent with the infrastructure. The differences between the resources defined in the template and the actual resources can be detected easily using the deviation detection capability of Alibaba Cloud ROS and AWS CloudFormation. The deviation correction function can also be used to keep the template content consistent with the actual resources.
With the full migration of IT infrastructure, the manual on-cloud O&M is no longer sustainable, causing pain points, such as low deployment efficiency, poor replicability, poor consistency, management difficulties, and difficulties in implementing DevOps. Automated orchestration tools, such as Alibaba Cloud ROS, AWS CloudFormation, Terraform, and Pulumi, can define infrastructure using templates based on the IaC concept. At the same time, these tools are capable of standardizing and automating the entire deployment process. With capabilities, such as change set, deviation detection, and the pipeline, the DevOps for IT infrastructure management is truly implemented. Operations and maintenance team members can focus on and use relevant tools to improve operations and maintenance efficiency and unleash productive forces.
Wang Binxin is engaged in the R&D of Alibaba Cloud elastic computing resource orchestration tools. He is also the producer of Alibaba Cloud Lingyun Moment and PyCon China and works with open-source projects and writing in his spare time.
Disclaimer: The views expressed herein are for reference only and don't necessarily represent the official views of Alibaba Cloud.
Alibaba Clouder - February 15, 2021
Alibaba Cloud ECS - September 7, 2021
Alibaba Clouder - March 11, 2020
Alibaba Clouder - September 7, 2020
afzaalvirgoboy - October 29, 2019
Alibaba Cloud Community - September 15, 2021
An online computing service that offers elastic and secure virtual cloud servers to cater all your cloud hosting needs.Learn More
High Performance Computing (HPC) and AI technology helps scientific research institutions to perform viral gene sequencing, conduct new drug research and development, and shorten the research and development cycle.Learn More
Apsara Stack Agility Elastic Compute Service (Alibaba Cloud ZStack) is a light-weight hybrid cloud solution.Learn More
A HPCaaS cloud platform providing an all-in-one high-performance public computing serviceLearn More
More Posts by Alibaba Clouder