Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Dubbo 3.0: Service Discovery with Support for Millions of Clusters and Scalable Microservice Architecture

Dubbo 3.0: Service Discovery with Support for Millions of Clusters and Scalable Microservice Architecture

This article focuses on the performance testing of the next-generation microservice framework Dubbo 3.0 in the address push link.

By Liu Jun (Lugui)

This article is a stress testing article on the address push performance of Dubbo. The article demonstrates the performance improvements of Dubbo 3.0 through comparisons, focusing on the newly introduced application-level address model. However, this article is not a reference baseline for the performance of the official version. Some of the comparative data was not collected due to the environment and time. This is a qualitative result of the detection stage, which doesn't generate much impact on the whole due to restrictions.

Introduction

This article focuses on the performance testing of the next-generation microservice framework Dubbo 3.0 in the address push link. It is also a phased summary on the implementation of Dubbo 3.0 in Alibaba at the performance level. This round tests the interface-level address discovery of Dubbo-2 and Dubbo-3 and the application-level address discovery of Dubbo-3. The stress testing data shows the stress testing scenario with millions of instance addresses:

  • Based on the interface-level address discovery model, the comparison between Dubbo3 and Dubbo2 shows that the permanent memory decreases by more than 50% in Dubbo-3, and the Full GC interval is lengthened significantly.
  • The newly introduced application-level service discovery model of Dubbo-3 can reduce resource usage significantly. The permanent memory is reduced by 40% compared with Dubbo-3 interface-level addresses. The incremental memory allocation is basically zero in application instance scaling scenarios. The Full GC is reduced to two times in the same time cycle (one hour).

As the core middleware supporting future business systems, Dubbo 3.0 will undoubtedly be of great help to business systems by improving the resource occupancy rate and stability.

Background

1. Introduction to the Next-Generation Service Framework Dubbo 3.0

Dubbo 3.0 is a converged product based on High-Speed Service Framework (HSF) and open-source Dubbo. It upgrades the cloud-native architecture comprehensively based on the two frameworks. It will be the main product for Alibaba and the open-source community.

Dubbo 3.0 was created during the migration to the cloud promoted by Alibaba. This also provides an opportunity for middleware products to fully embrace cloud business and provide internal and open-source products. Middleware products are expected to get rid of exclusive and open-source systems, which is also conducive to the realization of value maximization. The large-scale practical experience of the Alibaba e-commerce system can be shared with the community. Outstanding developers in the community can also participate in the project. HSF and Dubbo are very successful products for service frameworks. Internally, HSF supports all the previous Double 11 with excellent and time-tested performance. On the open-source side, Dubbo is the first open-source service framework in China with a wide user base.

Maintaining two highly homogeneous products at the same time poses huge challenges to development efficiency, business costs, product quality, and stability. For example, the interworking between Dubbo and HSF is a major obstacle. When some eco-companies in Alibaba, such as Kaola and Eleme are using Dubbo technology stacks, achieving smooth and sound interworking with HSF is a major obstacle. Besides, incompatible services lead to a high cost of community output and quality acceptance. The accumulated service-oriented experience and achievements cannot empower the community directly. After the second adaptation of Dubbo, its functionality and stability acceptance must be reverified. To completely solve the problems above, we have made a plan for the comprehensive development of Dubbo 3.0 combined with the overall cloud migration, open-source, and cloud product output strategies mentioned above.

1

2. Performance Stress Testing and Comparison of Address Push Procedures for Different Dubbo Versions

The following figure shows the basic working principles of the service framework. The Orange part is the address push procedure for key stress testing this time. The differences between the Consumers in different versions of Dubbo will be highlighted among millions of address push instances, especially the real-world performance of Dubbo 3.0.

2

For comparison, the following scenarios were selected for stress testing:

  • Dubbo-2: It is the reference baseline of this stress test.
  • The interface-level address discovery model of Dubbo-3: It is the same model as Dubbo-2.
  • The application-level address discovery model of Dubbo-3: It is introduced by the cloud-native version of Dubbo.

The Stress Testing Environment and Method

  • Stress Testing Data

This stress test simulates a 2.2 million interface-level address push of cluster instances. A single consumer process subscribes to 2.2 million addresses.

  • Stress Testing Environment

It is running on a Linux system at the condition of 8C and 16GB, and the heap memory is set to 10GB in the JVM parameter.

  • Stress Testing Method

The Consumer process subscribes 700 interfaces. As the registry, ConserverServer simulates the address change push continuously in a certain proportion for more than one hour. Consumer processes and machine metrics are collected during this process.

Analyses and Comparisons of Optimization Results

1. GC Time Consumption and Distribution

3
Interface-Level Address Model of Dubbo-2

4
Interface-Level Address Model of Dubbo-3

5
Application-Level Address Model of Dubbo-3

2. Incremental Memory Allocation

6
Interface-Level Address Model of Dubbo-2

7
Interface-Level Address Model of Dubbo-3

8
Application-Level Address Model of Dubbo-3

3. OLD Area and Permanent Memory

9
Interface-Level Model of Dubbo-2

10
Interface-Level Model of Dubbo-3

11
Application-Level Model of Dubbo-3

4. Consumer Loads

12
Interface-Level Model of Dubbo-3

13
Application-Level Model of Dubbo-3

Detailed Comparisons and Analyses

1. Dubbo-2 Interface-Level Model vs. Dubbo-3 Interface-Level Model

On a scale of 2.2 million addresses, Dubbo 2 quickly occupies the entire heap memory, and most of them cannot be released. However, the frequently triggered GC makes the entire Dubbo process unresponsive. Therefore, our stress testing data collection did not last long.

In the same way, the interface-level address model is kept unchanged. After optimization, Dubbo-3 only has three times of Full GC within one hour. In addition, the memory that cannot be released during the continuous push process is reduced by about 1.7GB.

2. Dubbo-3 Interface-Level Model vs. Dubbo-3 Application-Level Model

After the switch to the application-level service discovery model of Dubbo-3, the overall resource usage decreases significantly, which is reflected in the following aspects:

  • When the application process goes online or offline, the incremental memory increases slightly. The interface-level MetadataData is basically fully reused, and the new parts only come from the configuration changes of newly expanded machines or some services.
  • The resident memory has dropped by nearly 40% to about 900M compared with the interface-level model of Dubbo-3.

The current application-level address push model still has room for further optimization in code implementation, such as Metadata reuse and URL object reuse. This part will be the focus of our follow-up exploration.

Summary

Dubbo 3.0 has achieved the full integration of Dubbo and HSF, and the cloud-native solution is also in progress. During the 2020 Double 11 Global Shopping Festival, Dubbo 3.0 supported the Koala business and passed the partial online pilot tests by other Alibaba e-commerce applications. In the future, we will focus on further Dubbo 3.0 improvements. We will implement the application-level service discovery, new service governance rules, and the next-generation Triple agreement. We will also achieve the non-functional objectives, including resource occupation, performance, and cluster size.

The performance stress testing of this push procedure is a phased acceptance during implementation and R&D. Application-level service discovery has reduced resource usage significantly, which has brought the feasibility of building truly scalable clusters in the new architecture in the future. This strengthens our confidence in the application-level service discovery architecture after the continuous improvement of interface-level and application-level models to take the lead in the overall performance of Dubbo 3.0. We will focus on the implementation of the migration solution to support smooth and transparent migration from old models to new models.

Architecture Microservices Trends Computing Stress testing Open Source cloud native Dubbo RPC framework
0 0 0
Share on

Read previous post:

An In-Depth Insight into Nacos 2.0 Architecture and New Model with a Supported gRPC Persistent Connection

Read next post:

Fluid Helps Improve Data Elasticity with Customized Auto Scaling

Alibaba Cloud Native Community

480 posts | 48 followers

You may also like

Comments

Alibaba Cloud Native Community

480 posts | 48 followers

Related Products

More Posts by Alibaba Cloud Native Community

See All