×
Community Blog Dubbo-Go 3.0: Enter the Cloud-Native Era with gRPC

Dubbo-Go 3.0: Enter the Cloud-Native Era with gRPC

This article focuses on the new communication protocol and application-level service registration and discovery model of dubbo-go 3.0.

By Li Zhixin and Yu Yu

1

After becoming open-sourced in 2011, Dubbo has been adopted by a large number of small and medium-sized companies and has also been the most popular RPC framework in China. In 2014, Dubbo was suspended for a period due to the adjustment of Alibaba's internal organizational architecture. After the launch of Spring Cloud, the integration of the two systems promoted the popularity of microservices.

However, this world is changing fast. Since the birth of container technologies represented by Docker and container orchestration technologies represented by Kubernetes, the cloud-native era has arrived. In the cloud-native era, the immutable infrastructure has brought immutable middleware infrastructure to existing middleware. The gRPC unified the underlying communication layer. The protobuf unified the serialization protocol and service mesh represented by Envoy and Istio unified the service control and data layers gradually.

The idea of dubbo-go is "Bridging the gap between Java and Go." While maintaining the interconnection between Go and Java applications, dubbo-go embraces the cloud-native era with the help of the Go language (the de facto, first cloud-native language.) The dubbo-go community strived to create three products in 2020:

  • Dubbo-go 1.5 was released, which is similar to dubbo 2.7.
  • The dubbo-go-proxy project in the Sidecar form will be released in the near future.
  • Dubbo-go 3.0 is in development.

In a word, dubbo-go 3.0 has a new communication protocol, serialization protocols, application registration models, and service governance capabilities! This article focuses on the new communication protocol and application-level service registration and discovery model of dubbo-go 3.0.

dubbo-go 3.0 vs. gRPC

There is a saying, “Progress is made by knowing yourself and your opponents.” The communication layer of dubbo-go 3.0 mainly uses gRPC for reference.

In short, the gRPC protocol is based on the HTTP2 protocol. It adds a specific protocol header starting with grpc-. It also uses a specific unpacking tool (protobuf) to serialize the data. Therefore, RPC calls are implemented.

2

As we all know, gRPC has almost no service governance capabilities. The existing Dubbo framework of Alibaba Cloud is equipped with RPC and service governance, which is as powerful as gRPC. However, since gRPC is used by everyone, the new communication protocol of dubbo-go 3.0 must be fully compatible with gRPC and services deployed by developers. It should also retain the existing Dubbo protocol and service governance capabilities and further introduce a series of new policies, such as mesh support and application-level service registration.

dubbo-go 3.0 vs. dubbo-go 1.5

Currently, gRPC has already been supported in dubbo 2.7. Developers can use the protoc-gen-dubbo to convert the protobuf IDL protocol into stub supported by the framework. Then, the RPC process of underlying gRPC conn can be used to transfer the existing service governance capabilities to gRPC from top to bottom. By doing so, the support for gRPC services is implemented.

The dubbo-go 1.5.x also supports gRPC stream calling. Similar to unary RPC, by generating the stubs supported by the framework, the streaming RPC capability is incorporated into the framework based on underlying gRPC stream calls. However, since dubbo 2.7.x and dubbo-go 1.5.x do not support stream calling, no upper-layer service governance is supported for gRPC stream calling.

3

The problem developers face is that they may not feel at ease when using dubbo-go 2.7 for gRPC transmission. The upcoming protocol of dubbo-go 3.0 will solve this problem from the root.

Three Levels of Protocol Compatibility

The author believes the support from a service framework provided to the third-party protocol can be divided into three levels: application level, protocol level, and transmission level.

If a framework encapsulates the interface beyond protocol SDK, the support can be considered as application-level. This framework needs to be in line with the interfaces of lower-level SDKs, while the extensibility is poor.

In terms of protocol level, support is provided by the framework from the configuration layer to the service governance layer. The protocol layer and the network transmission layer all use a fixed communication protocol. Such a framework can solve service governance problems, but the framework itself cannot be fully adapted to the third-party protocol. If so, its support for the third-party protocol will be weakened, such as the lack of dubbo-go 1.5 support for RPC stream calling mentioned above.

Efforts need to be made at the transmission layer to further support more third-party protocols. The specific fields of the third-party protocols and the frame models and data streams of the underlying protocols depended (such as HTTP2) must be understood. Then, data exchange modules that are completely consistent with the third-party protocol can be developed as the bottom layer of this framework. By doing so, the extensibility of the protocol is maximized. The required fields can be added optionally while being compatible with existing protocols. Thus, functions that cannot be implemented by existing protocols can be developed, such as the back-pressure policy supported in dubbo-go 3.0.

HTTP2-Based Communication Process

The transmission process of unary RPC calling in gRPC based on HTTP2 is listed below:

  • The client sends the Magic message: PRI * HTTP/2.0rnrnSMrnrn
  • The server receives it and checks the correctness
  • The client and the server send the setting frame to each other and then send an ACK to confirm it.
  • The client sends the Header frame that contains gRPC protocol fields with the End Headers as the Header ending mark.
  • The client sends the Data frame, which contains the request information of the RPC call with the End Stream as the Data ending mark.
  • The server calls the function to obtain the result.
  • The server sends the Header frame, which contains the gRPC protocol field, with the End Headers as the Header ending mark.
  • The server then sends the Data frame containing the response information returned by the RPC call.
  • The server sends the Header frame again, which contains the RPC status and message information, with the End Stream as the ending mark of the current RPC call.

The following figure shows the HTTP2 Header frame that contains the gRPC call information:

4

When the stream is called in gRPC, Data can be sent multiple times when the server returns the Data. After the call is completed, the Header is sent to terminate the RPC process, and the status information is reported.

The communication layer of dubbo-go 3.0 has the same communication process based on HTTP/2 protocol, ensuring the communication with gRPC underlying layer.

Expected Communication Architecture of dubbo-go 3.0

In addition to the communication protocol HTTP2, dubbo-go 3.0 will use the triple protocol (hereinafter referred to as the dubbo3 protocol) based on Google protobuf as the serialization protocol. This will lay the foundation for Dubbo to support more programming languages in the future. The current transmission model of dubbo-go 3.0 is shown in the following figure:

5

  • To support both unary RPC and stream RPC, the data stream structure is added on the server side and the client side to transfer data through asynchronous calls.
  • The original TCP communication capability is continuously supported.
  • The dubbo3 protocol is supported based on the HTTP2 protocol. The decoding process is compatible with the protobuf that gRPC uses. This ensures the connection to gRPC services.

Application-Level Service Registration and Discovery

1. An Introduction to Application-Level Service Registration and Discovery

Dubbo-go 3.0 uses the next-generation service registration and discovery system. The original "interface-level registration and discovery" is replaced by "application-level registration and discovery."

Interface-level registration and discovery uses the RPC service as the key and the instance list as the value for data organization in the registry. The new "application-level service registration and discovery" uses an Application as the key and a group of Instances deployed by the application as the value. This brings two differences:

  • The data mapping relationship changes from RPC Service :arrow_right: Instance to Application :arrow_right: Instance.
  • Less data is involved, and the RPC service and its configuration information are not included in the registry.

It can be concluded that the amount of data stored and pushed by the application-level model is proportional to the number of applications and instances. The number of applications or instances will only increase after the pressure of address push increases.

For an interface-level model, the data volume is positively correlated with the number of interfaces. Since one application usually publishes multiple interfaces, the data volume is generally dozens of times that of an application-level mode. Another key point is that the interface definition is more about the internal behavior on the business side. The interface-level model will cause opacity of the cluster size assessment. For the application-level model, the instance and application growth is usually planned on the O&M side with good controllability.

Industrial and Commercial Bank of China (ICBC) used to estimate these two models in production. The result showed that the application-level model can reduce the data volume to 1.68% of the original in the registry. The new model also made it easy for ZooKeeper to support 100,000-level services and nodes.

2. An Introduction of the Metadata Center Synchronization Mechanism

The decrease of data volume in the data center results in the loss of RPC service-related data in the registry. Only the data on application and instance level remains. To ensure that the missing data can still be perceived by the Consumer, a separate communication channel is established between the Consumer and the Provider. Currently, two solutions are available for metadata synchronization:

  • Built-in MetadataService
  • An independent metadata center that coordinates data through the refined metadata cluster

3. Compatibility with dubbo-go in Early Versions

The entire development process should be more transparent to the users of early-version dubbo-go, while avoiding the impact brought by the specified provider in terms of extensibility. For that goal, a set of mapping relationships between RPC services and application names has been designed. These relationships automatically complete the conversion from RPC services to provider application names on the consumer side.

6

This design allows dubbo-go 3.0 to maintain the interconnection with dubbo 2.6.x, dubbo 2.7.x, and dubbo 3.0.x.

Support for Unified Routing

In terms of routing, a subset of IP addresses is selected based on the specific routing rules from the list of all existing IP addresses. The routing process involves filtering according to the configured routing rules and the result obtaining from the intersection of all the routing rules. Similar to a pipeline, multiple routers form a routing chain, which filters out the final destination address set from all the address tables. Then, the chain selects the address to be accessed based on the load balancing policy.

1. Routing Chain

7

The logic of the routing chain can be simply understood as target = rn(...r3(r2(r1(src)))). For the internal logic of each router, multiple mutually disassociated address pools, such as addrs-pool-1 ... addrs-pool-n, are split by the addrs-in based on full address addrs-all in the router. The intersection is taken as the output address based on the defined rules. Likewise, the calculation of the entire routing chain is completed.

8

2. Failover

Failover fields can be configured in the routing rule configuration file. Failover can be triggered when an address cannot be found. Then, other subsets are selected, and the same process is executed. This procedure will be repeated until the address is found. Otherwise, an error indicating that the address cannot be found should be reported.

3. Backup Routing

In the configuration of the routing rules, a match without any conditions can be configured. The final result is that at least one subset is selected for address protection.

About the Authors

Li Zhixin (GitHub ID: LaurenceLiZhixin) is the Development Engineer of Alibaba Cloud's Cloud-Native Middleware Team and a developer in the dubbo-go community. Li is currently a student at Sun Yat-sen University, majoring in software engineering. Li is good at using the Go language, focusing on cloud-native technologies and microservices.

Yu Yu (Github ID: @AlexStocks) is the Director of the dubbo-go project and community and a programmer with more than ten years of experience in infrastructure research and development on the server. Yu Yu has participated in improving Muduo, Pika, Dubbo, Sentinel-go, and other well-known projects. Currently, Yu Yu is engaged in container orchestration and service mesh in the trusted-native department of Ant Financial.

0 0 0
Share on

Alibaba Developer

210 posts | 34 followers

You may also like

Comments

Alibaba Developer

210 posts | 34 followers

Related Products

  • Starter Package

    High-performance virtual machines with data transfer plan, starting from $2.50 per month

    Learn More
  • Link IoT Edge

    Link IoT Edge allows for the management of millions of edge nodes by extending the capabilities of the cloud, thus providing users with services at the nearest location.

    Learn More
  • Elastic High Performance Computing Solution

    High Performance Computing (HPC) and AI technology helps scientific research institutions to perform viral gene sequencing, conduct new drug research and development, and shorten the research and development cycle.

    Learn More
  • Quick Starts

    Deploy custom Alibaba Cloud solutions for business-critical scenarios with Quick Start templates.

    Learn More