80% of Software Environment Management Problems Are Rooted Here


The root cause of 80% of software environment management problems is here. AppStack, the cloud-native application management platform, is an application delivery platform based on OAM. Enterprises can use AppStack in the cloud to use declarative methods such as application orchestration, placeholders, and variables. Define, implement a set of orchestration and multi-environment differentiated deployment, and implement one-click pull-up and one-click rollback of environments based on versions and baselines.

The final state of software delivery is to provide a stable and predictable system. To do this, we need to ensure: First, the consistency of software products; Second, the consistency of the operating environment.

In the third lecture, we shared how to ensure the consistency of software products. In this lecture, we will talk about how to ensure the consistency of the environment .

The goal of operating environment consistency is that the environment is predictable, stable, and low-cost. Among them, low cost is more critical, because the cost of environmental resources is generally relatively high.

We can divide the runtime environment into three parts: artifacts, execution engines, and orchestration rules.

To ensure the consistency of the product, the first is to ensure the consistency of the code and its dependencies; the second is to ensure the consistency of the built environment; the last is to ensure the consistency of the build script. Ensuring the consistency of the environment also includes three points:

1. Application consistency, such as consistent container images;
2. Consistency of the context required for container operation, such as consistent data configuration, etc.;
3. The consistency of the orchestration rules needs to ensure that the orchestration executes the same rules, such as the same container deployment rules, the same node distribution rules, and the same backup rules.
To ensure these three points, a consistent runnable environment will be formed after the deployment is completed. But in reality, there are still many other problems in the environment, such as:

There are many configurations such as monitoring in the configuration file. For users, they do not know the specific function of the configuration, and they do not know how to set it when they need to be modified.

When testing, the dependent environment often has problems, which consumes a lot of waiting and troubleshooting time. For example, if there are frequent problems with the dependent API, it may take a long time for the relying party to troubleshoot and repair, so that the test work cannot be continued.

Up a new environment is time-consuming. It is very painful to build a new environment. For example, an international team often has to build new sites, and it takes a whole day to build a new environment. It is very painful.

The app doesn't work locally. For example, the application cannot run normally due to a lack of a certain resource.

The configuration environment needs to be careful, and there may be configuration omissions. You need to be very careful every time you configure the environment, especially when multiple people cooperate in the environment configuration, there will be configuration conflicts, which will cause the program to fail to run. You need to check and solve the whole link.

Here we briefly list five common problems, all of which are rooted in the lack of a clear definition of the environment: unclear about the specific content of the environment, and vague cognition of the process of building the environment.

A clear definition of the environment, including what artifacts the environment contains, how those artifacts are deployed, etc. So what is the end state of environmental management?

When the artifacts are the same, the running context is the same, and the resource data is the same, the environment should be consistent based on the same orchestration rules. Meanwhile, the environment can be defined and declared by software. This is what we think of as the end state of environmental management, or in simple terms, a software-defined immutable environment. Software-defined immutable environments can be incorporated into version management to ensure that the environment is firstly defined and then has a clear version.

software environment management:Stages of Environmental Management

Stage 1: Instructions
The first stage of environmental management is documentation, which I believe many people have experienced. When we are working on a project or product, we will write various documents such as deployment instructions, documentation, and upgrade documents for the entire product. However, documentation is difficult to guarantee practical, and it is not necessarily up-to-date and accurate. Every time we go to the field to deliver, we encounter some problems that are not described in the documentation. At this time, you have to call to confirm whether there is anything missing. Using documents or manuals to manage the environment has great drawbacks, so we thought of using commands to manage it.

Stage 2: Command
In the second stage, we wrote various shell and Python scripts by means of commands, and combined related commands together. When we were doing the privatized delivery of a product, the initial approach was to use scripts to manage the environment, because it was too painful to open a new environment, and it took a lot of time to change parameters, find packages, configure IP, etc., which was too efficient. Low. So we wrote scripts to manage and replaced documents with scripts.

But there are also many problems with managing environments with scripts. We have to deal with a variety of environments. The same task may have different command combinations in different scenarios, so there will be more and more scripts, and the cost of maintaining scripts will also increase.

Stage 3: Declaration
In order to solve the maintenance and stability of the command script, we entered the third stage, declarative - expressing the environment through configuration and defining the environment. Declarative descriptions provide a deterministic representation of the environment.

Taking k8s as an example, let's take an example to see how to make environmental declarations

The above picture shows the simple architecture of K8S. The upper left corner is the K8S master, and the lower-left corner is the node. The master has several components: scheduler, ControllerManager, API server, and Etcd. Etcd is a distributed storage, and the configuration information is in Etcd. A node is a physical machine or a virtual machine. There will be multiple pods on each Node, and many containers will run on it

We know that the smallest unit of K8S is a pod, which contains containers and various network storages, which are described by pod declarations. After the declaration is applied, the specific things are handled in the controller.

Separation of concerns via sidecar

We write an application. A lot of code in this application is actually not related to business but contains a lot of code for service governance, such as logging, monitoring, current limiting, and circuit breaker. These codes account for a large proportion, and the maintainers and application developers of these codes are not necessarily the same group of people. For code related to service governance, many companies will have a dedicated team responsible for maintenance and upgrades, similar to Alibaba's middleware team. Traditionally, we need to upgrade and redeploy applications to upgrade service governance capabilities. However, in the cloud-native era, application developers want to focus only on application business code, and put the code related to service governance in other containers. The business container and service governance container are usually arranged in the same pod, but the management of the service governance container, Development and release have nothing to do with application developers, these are sidecar containers. The sidecar container realizes the separation of concerns. Application developers, middleware developers, and operation and maintenance-related developers can all have their own concerns.
Let's take two roles as an example. One is the business developer, who is concerned with the application container, so when it is released, his focus is on the application container. The second is the SRE of the enterprise. His focus is often on the various service management containers of the sidecar, such as whether the log level and sampling rate of the log agent are reasonable.
With sidecars, the concerns of business developers and SREs are separated. Another advantage of this separation is that the sinking of middleware is managed in a sidecar manner. Once the corresponding middleware needs to be upgraded, the business code does not need to be changed or released, only the sidecar release is required.
We mentioned earlier that a consistent environment needs to have 3 components: the same artifacts, the same runtime context, and the same orchestration rules. The essence of the same running context is that the configuration inside should be consistent. At first we used documents to tell us how to manage the environment, then we used scripts, and finally we used environment declarations.
However, defining the environment in a declarative way is not perfect. For a specific example, some corresponding configurations are required when the application is running. Middleware, basic resources, CPU, storage, etc. all need to be configured. This will face a big problem - there are too many environment-related configurations.

Related Articles

Explore More Special Offers

  1. Short Message Service(SMS) & Mail Service

    50,000 email package starts as low as USD 1.99, 120 short messages start at only USD 1.00

phone Contact Us