By Fengchong (Senior Technical Expert of Alibaba Cloud Serverless)
With the popularization of modern applications and the deepening of enterprise cloud migration, more cloud resources are required in business. During enterprise cloud migration, there are usually 2 teams involved, a platform team and an infrastructure team. The Platform Team focuses on the business, based on your business scenario abstraction for R&D personnel shielding infrastructure. The Infrastructure Team pays attention to safety and cost for the Platform Team planning different sub-account permission management, strategy, and network configuration.
The inevitable result of this hierarchical management leads to completely different lifecycles of applications and infrastructure. Therefore, infrastructure administrators, platform administrators, and R&D personnel focus on different perspectives on cloud resources.
When using the platform, developers only need to focus on code, data, configuration, and other program-related content.
Therefore, as the boundaries of responsibilities vary, the Platform Team faces an even greater challenge. The team should explain the business to the R&D personnel and explain the use of cloud products to the infrastructure team, which undoubtedly increases a lot of communication costs and reduces the iterative efficiency of the business.
If an automated pipeline can be established to allow different teams to complete their boundaries in a self-service manner, productivity will be improved significantly. This requires several important concepts to connect Dev and Ops:
Service: It describes the code and program and only describes the information related to the program, such as function configuration and log collection path.
If you want to achieve efficient self-service operations, the rationalization solution is to template the concepts above and use the following workflow to complete them:
Different teams of the enterprise can complete the construction of infrastructure by themselves through this borderline templated processing method, improving production efficiency while ensuring the isolation of permissions and protecting the infrastructure.
Serverless Devs  is a management tool for the entire lifecycle of Serverless applications. Its model specification contains the concepts of applications and services but currently lacks internal support for the environment. The code and infrastructure are jointly maintained under an
s.yaml. The limitations of this mode in multiple environments are mainly three points:
s.yamlfor different environments are relatively high. When updating the environment, you need to re-initiate the deployment. When you connect to CI/CD services, you need to re-initiate a complete release and launch operation. (However, under normal circumstances, environmental changes, such as upgrading and updating permissions, are safe for the program and do not need to initiate a launch.)
If the layered templating scheme described in the preface of this article is adopted, the problems above can be solved successfully.
How can we define an environment template in Serverless Devs?
The environment template is aimed at infrastructure or cloud resources. Serverless Devs cannot work without the operation of cloud resources. The traditional method is to directly use cloud product SDKs in components. However, corresponding component codes need to be developed when supporting new resources. Therefore, the development efficiency caused by resource expansion is reduced, and the code is increasingly difficult to maintain. A better way is to use the IaC of infrastructure (i.e. code) to create cloud resources.
Serverless Devs used Pulumi  to encapsulate Pulumi Stack through a single component in previous practice. However, in practice, it was found that using GPLs to define IaC still requires good abstraction at the model level. Therefore, it is not very good to extend Pulumi to component developers in terms of ecological maturity and flexibility.
Currently, the most powerful tool of IaC ecology is Terraform , which has become the de facto standard. Terraform HCL is a DSL, which can be compatible with any ecosystem, and it has lots of Providers. If Alibaba Cloud products are connected to POP, Terraform providers can be automatically generated, which has high reliability and access convenience.
If you use Terraform IaC to define an environment template and connect it well within the Serverless Devs system, this broadens the user domain immensely. You can write Terraform files to define your infrastructure and use Serverless Devs to connect all workflows.
Follow the component development specifications of Serverless Devs and encapsulate operations in multiple environments into env commands. The following workflows can be implemented using the env command:
Let's assume the business scenario requires:
As a platform administrator, you need to develop:
Environment templates use IaC to define resources. Currently, only Terraform templates are supported. The code directory of the environment template must contain two types of files.
IaC File: It is the .tf file of Terraform. The core elements of the IaC file are:
Write IaC to define the variable, resource, and output of the environment template
Complete code example: https://github.com/devsapp/fc/blob/main/examples/multi-envs/infra/main.tf
Define a policy for the preceding resources, maintain the principle of minimum permissions, and only release the necessary write permissions. Terraform depends on the read permissions of many resources when creating resources. Therefore, we recommend adding these common read permissions:
Example of complete code: https://github.com/devsapp/fc/blob/main/examples/multi-envs/infra/policy.json
Publish an environment template using the s env apply-template:
s env apply-template --name testing --description 'it is a demo' --code ./infra
The following list describes the fields in the format:
|Name||True||Environment Template Name|
|Description||False||Environment Template Description|
|Code||False||Template Code Directory|
After the operation succeeds, the variables, outputs, status, policy, text content, and version of the current template are returned.
The environment needs permissions to operate the corresponding cloud resources to grant Function Compute access to the cloud resources in role-playing mode. Therefore, you need to:
s env init command to enter an interactive operation. Enter the environment name, region, role, environment template, and template parameters to deploy the environment:
After the execution, the
env/fc-env-testing.yaml description file is created in the
. s directory. You can view and edit the file.
The developer writes
s.yaml and associates the infrastructure configuration with the specified environment. You can use the following methods:
environment.outputsto associate the output defined in the environment template.
environment.nameto automatically associate the service with the environment.
s deploy --env to deploy the function to a specified environment:
s deploy --env fc-env-testing
After the command is executed, the component determines whether the environment has been deployed. If the environment is in the ready state, the service is deployed to the environment. Otherwise, the component deploys the environment before the service is deployed.
This article analyzes the application and infrastructure management challenges encountered by enterprises when going to the cloud and proposes adopting a hierarchical templating method to organize the workflows of different teams. It defines a modern application through environment, service, and pipeline. Environmental templates, service templates, and pipeline templates are used to shield the complexity of infrastructure and improve operational security. The core is that a pipeline is needed to connect the entire workflow and let the various stages of DevOps be self-service and completed safely.
The author hopes that Serverless Devs can serve as this pipeline. The ideas of its applications, components, and plug-ins provide developers with a good foundation for building modern applications. Serverless Devs already has the abstraction of services and service templates and needs to expand the capabilities of environments and pipelines.
This article focuses on how to use Serverless Devs to manage multiple environments. The key challenge is to decouple code and infrastructure and use IaC to complete the definition of infrastructure. In the IaC ecosystem, Terraform is the most suitable one to be introduced. Therefore, Terraform HCL is used to define environment templates. The Resource Orchestration Service (ROS) of the environment is completed through the Terraform service in the backend. This way, you can use Serverless Devs to complete the complete workflow of publish environment template → deploy environment → deploy an application to the specified environment.
There is still a long way to go to realize the vision above. The main Roadmap for future planning is listed below:
This article introduces the use of multi-environment features of Serverless Devs. Part 2 of this 2-part series
will explain some common problems in detail.
Alibaba Cloud Serverless - August 23, 2022
Alibaba Cloud Community - August 26, 2022
Alibaba Cloud Community - November 25, 2021
Alibaba Cloud Community - December 21, 2021
Alibaba Cloud Serverless - February 17, 2023
Alibaba Cloud Native Community - August 25, 2022
Alibaba Cloud Function Compute is a fully-managed event-driven compute service. It allows you to focus on writing and uploading code without the need to manage infrastructure such as servers.Learn More
An online computing service that offers elastic and secure virtual cloud servers to cater all your cloud hosting needs.Learn More
Accelerate software development and delivery by integrating DevOps with the cloudLearn More
An enterprise-level continuous delivery tool.Learn More
More Posts by Alibaba Cloud Serverless