SOFARegistry is a production-level, low-latency, and highly available service registry powered by Ant Group. SOFARegistry was originated from ConfigServer of Taobao. After more than ten years of business development of Ant Group, SOFARegistry has evolved into the fifth generation architecture. Currently, SOFARegistry not only provides full support to Ant Group and its numerous partners, but also embraces the open source community. Built on an AP architecture, SOFARegistry normally supports message push within one second, and very rarely up to several seconds when the server is extremely busy. It also adopts a layered architecture to support infinite horizontal scaling.
SOFARegistry adopts a layered architecture and partition-based data storage to break the single machine performance and capacity bottleneck, and to support the theoretical “infinite horizontal scaling”. It has been providing reliable services to the Ant Group production environment which has a massive number of nodes and services.
With the SOFABolt communication framework, SOFARegistry determines node status and pushes changes based on TCP persistent connections, notifying service online/offline status in seconds.
Unlike CP-architecture based registry products such as ZooKeeper, Consul, and Etcd, SOFARegistry adopts the AP architecture based on the service characteristics of service discovery, which significantly improves the availability of the registry in the case of failures caused by network partitioning. SOFARegistry takes many measures, such as multi-replica clusters, to prevent service unavailability arising from node failures.
SOFARegistry has four roles: Client, SessionServer, DataServer, and MetaServer, each with unique capabilities and responsibilities. They are combined to provide external services. The following figure shows their relationships and structures.
A client provides basic APIs to allow applications to access SOFARegistry. The client provides JAR packages to application systems, so that they can invocate the service subscription and publishing features of SOFARegistry.
The SessionServer grants clients access to SessionServer, and accepts service publishing and subscription requests from clients. It also serves as an intermediate layer to forward the published data to DataServer for storage. The SessionServer can be infinitely scaled up to support connection with large amounts of clients.
The DataServer is responsible for storing data published by clients. The data is stored by dataId through consistent hashing. DataServer supports multi-replica backup to ensure high availability of the data. DataServer can also be infinitely scaled up to support large amounts of data.
The MetaServer is responsible for maintaining the consistency list between SessionServer and DataServer within a cluster, and immediately notifies other nodes in the case of any node changes. The MetaServer ensures high availability and consistency through SOFAJRaft.