Local Development - Alibaba DevOps Practice Guide Part 9

Part 9 of this 27-part series describes the software development and delivery processes, including development, debugging, testing, integration, and delivery.

This article is from Alibaba DevOps Practice Guide written by Alibaba Cloud Yunxiao Team

When developing a requirement, you need to write code and perform personal verification first. After you verify the function works as expected, you can submit the code and go to the integration environment for further verification and acceptance. Coding and verification occupy most of the time in the requirement delivery. Therefore, it is crucial to improve the efficiency of this part.

The Problem

What factors reduce the efficiency of development and debugging?

Given the following system, modify application A and application D to develop a certain requirement. Here, applications refer to a group of independent processes that can provide services and an optional load balance, such as a service in Kubernetes and its backend deployment.

Now, let's learn about the problems you may encounter when testing these two applications locally.

Difficulty Starting the Entire System Locally

We usually develop an application in a complex system. This application may be at the frontend of the system or in the middle of the system. Sometimes, all relevant applications need to be started to verify the entire process from end to end.

For example, application A in the figure above is the frontend application, and application D is in the middle, while the section in the black box contains applications involved to test this requirement. If they are Java applications, a development engine is already overloaded with five startup processes running on it. However, in many cases, the number of applications that need to be fully started far exceeds this number.

Unstable Dependent System

Since the entire system cannot be started locally, the local depends partially on the public test environment. Although the code should be deployed to the test environment after the local test meets expectations, as mentioned before, some bugs will inevitably occur, resulting in an unavailable test environment. This is also the value of the test environment, which refers to early problem detection. Once the dependent system is unavailable, applications cannot be tested normally.

Test Environment Connectivity in Cloud-Native Development Mode

In the Kubernetes-based infrastructure, most applications in the entire system do not need to be exposed to the public network through Ingress. If your test environment is a standalone Kubernetes cluster, it is impossible for the local to access the applications in the cluster. Then, there is no dependency on the public test environment, such as the dependencies of A > C, D > E, and D > F in the preceding figure.

There is another dependency; the dependency of upstream applications on local applications, such as the C > D dependency in the preceding figure. However, since C is in the public test environment, not all requests from C to D can be played locally. Therefore, it requires a mechanism to ensure that only requests based on specific rules are routed to locally developed application D.

Connectivity between External Dependent Systems and the Development Environment

Some test processes need to accept callbacks from external dependent systems, such as those from WeChat or Alipay. However, local applications usually do not have public network addresses, causing some difficulties with debugging.

Middleware Isolation

Message-oriented middleware, such as RocketMQ, is often used in distributed systems. If a public test environment is used, it means RocketMQ is also shared. Should RocketMQ messages be consumed by the test environment or a personal development environment? This is also a problem to be solved.

Efficient Local Development

We should try to use a verification method with faster feedback for efficient development in the whole process. This way, we can find problems early and gradually carry out more integrated and realistic tests.

Generally, a development process goes through the following three phases:

A coding and unit test ensures correctness at the level of small logical units.
An integration test targeting a single application may need to perform an HTTP-level mock on the dependent application.
A complete integration test combines with the public test environment.

Based on the preceding three phases, we can use the following methods to solve the preceding problems:

Use appropriate testing tools, such as Junit, for each language to conduct unit testing
Use HTTP Mock tools, such as moco, to solve the problem of local isolation verification and complete the integration testing of a single application
Use tools, such as kt-connect and virtual-environment, to resolve the connectivity issues between local and test environments and the coloring and routing of HTTP request process in the cloud-native infrastructure
Use tools, such as ngrok, to resolve the problem of calling local applications by external dependencies
Use a backbone stable environment as the public test environment to improve the stability
Use the coloring isolation capability of middleware to ensure the coloring and routing of processes, such as messages, other than HTTP requests

Method 1 and 4 are mature technologies. Details will not be described here. Method 5 and 6 are explained in detail in the following articles related to the test environment. This article mainly explains methods 2 and 3.

Integration Test Solution for a Single Application

For example, the test scope for application D is shown in the orange box in the following figure:

The dependency, such as the persistence of an application, uses a real one (generally the local database). However, external applications (applications E and application F) use an HTTP-based test substitute. Thus, all dependencies are stable, and it is easy to modify the behavior of the test substitute to test in specific scenarios.

Application D depends on two applications:

org-service (application F) provides the capability of querying organization information.
user-service (application E) provides the capability of querying user information.

Configure the access addresses for the two applications in the configuration item of application D:

...
org-service-host: org-service
user-service-host: user-service
...

We use the Docker compose + moco solution to explain how to use a local test substitute.

First, create the following directory structure:

├── Dockerfile
├── docker-compose.yml
├── moco-runner.jar
└── services
    ├── org-service
    │   └── config.json
    └── user-service
        └── config.json

Dockerfile:

FROM openjdk:8-jre-slim

ARG SERVICE

ADD moco-runner.jar moco-runner.jar
COPY services/${SERVICE}/config.json config.json

ENTRYPOINT ["java", "-jar", "moco-runner.jar", "http", "-c", "config.json", "-p", "8080"]

docker-compose.yml:

version: '3.1'
services:
  service-f:
    ports:
    - 8091:8080
    build:
      context: .
      dockerfile: Dockerfile
      args:
        SERVICE: org-service
  service-e:
    ports:
    - 8092:8080
    build:
      context: .
      dockerfile: Dockerfile
      args:
        SERVICE: user-service

services/org-service/config.json:

[
  {
    "request": {
      "uri": "/"
    },
    "response": {
      "text": "org service stub"
    }
  },
  {
    "request": {
      "uri": {
        "match": "/orgs/[a-z0-9]{24}"
      }
    },
    "response": {
      "json": {
        "name": "some org name",
        "logo": "http://xx.assets.com/xxx.jpg"
      }
    }
  }
]

services/user-service/config.json:

[
  {
    "request": {
      "uri": "/"
    },
    "response": {
      "text": "user service stub"
    }
  },
  {
    "request": {
      "uri": {
        "match": "/users/[a-z0-9]{24}"
      }
    },
    "response": {
      "json": {
        "name": "somebody",
        "email": "somebody@gmail.com"
      }
    }
  }
]

Run the following commands to start two dependent applications:

docker-compose up --build

Verify the behavior of the local test substitute:

$ curl http://localhost:8092/users/111111111111111111111111
{"name":"somebody","email":"somebody@gmail.com"}
$ curl http://localhost:8091/orgs/111111111111111111111111
{"name":"some org name","logo":"http://xx.assets.com/xxx.jpg"}

Then, change the dependency configuration of application D to the local test substitute for the test:

...
org-service-host: localhost:8091
user-service-host: localhost:8092
...

Finally, we have obtained a stable integration test environment for a single application. If you want to modify the dependency behavior, modify the config.json of the application.

The Docker compose + moco solution is a way to implement a single-application integration test. You can choose the appropriate tools and solutions according to the specific situation of the project.

Mutual Access and Process Isolation between Local and Public Test Environments

After completing the integration test of a single application, you can have quality confidence in a single application. However, verification of a wider range still needs to be performed with the real dependency integration.

As shown in the preceding figure, you need to solve two problems to enable applications (application A and application D) to start locally on-demand and reuse other applications (application C) in the test environment:

How can the local call applications in the public test environment? How can application A call application C?
How can the public test environment be called to the local? How can application C call the local application D?

Problem 1: If the networks of the local environment and the test environment are directly accessible, modify the configuration item of local application A. If you are using cloud-native infrastructure, you will need tools to achieve access, such as Yunxiao kt-connect. For more information about these tools, please see the connect section in kt-connect

Problem 2: Three problems need to be solved:

The call chain initiated from application A in the test environment should eventually access application D in the test environment. The call chain initiated from application A in the local environment should eventually access application D in the local environment. The two call methods do not affect each other. The call chain needs to be colored to distinguish the two calls. The coloring method used here can add an additional header to the request.
The call chain routes according to this coloring sign, which is also called coloring mark.
A call chain runs through multiple applications. It is necessary to ensure that the coloring mark can be passed down automatically when different applications are called.

Alibaba has a complete set of solutions for coloring and routing related to the first two problems. These solutions apply to scenarios, such as the HTTP process, RPC, and asynchronous messaging. In the open-source field, kt-connect based on cloud-native infrastructure can be used, and its mesh function can route call chains with specific coloring rules.

kt-connect routes based on the VirtualService and DestinationRule of Istio. The basic principle is to create a service and deployment of shadow copies in the cluster. Then, submit the DestinationRule resource of application D. This way, the request that contains the "local-env: true" header is routed to the shadow copies of application D. Next, the shadow copies of application D forward the request to the local. In this process, in addition to submitting and updating Istio resources manually, other tasks can be finished using the ktctl mesh command.

Next, solve the third problem, which is coloring mark transmission. In other words, after local application A transmits the request containing the "local-env: true" header to application C in the test environment when application C continues to access application D, the request should also contain this header.

The general idea is to add an Interceptor at the entry of the Web layer and record the coloring mark into a ThreadLocal. Then, take the coloring mark out of ThreadLocal at the exit of the HttpClient layer and fill it in the Request object. There is a problem that needs to be noted. The coloring mark is placed in ThreadLocal. Once you face the multi-thread situation while processing a web request, you need to carefully pass the ThreadLocal value to the corresponding sub-thread. If all applications correctly transmit the coloring mark, the coloring mark will be transmitted throughout the comprehensive process.

You can start the application locally as needed and perform development debugging and testing using the kt-connect mesh solution and the comprehensive-process coloring mark solution.

Summary

A combination of the unit test, the single-application integration test, and the end-to-end integration test are used for local debugging and testing to improve the efficiency of obtaining feedback.
The key technologies to locally start applications on-demand for end-to-end integration tests are comprehensive-process coloring and routing. There are different implementation methods under different infrastructures.

Community

Local Development - Alibaba DevOps Practice Guide Part 9

The Problem

Difficulty Starting the Entire System Locally

Unstable Dependent System

Test Environment Connectivity in Cloud-Native Development Mode

Connectivity between External Dependent Systems and the Development Environment

Middleware Isolation

Efficient Local Development

Integration Test Solution for a Single Application

Mutual Access and Process Isolation between Local and Public Test Environments

Summary

Read previous post:

Read next post:

Alibaba Cloud Community

You may also like

Comments

Alibaba Cloud Community

Related Products

Cloud-Native Applications Management Solution

Alibaba Cloud Flow

DevOps Solution

Container Service for Kubernetes