EventBridge event bus and EDA architecture analysis

As one of the top 10 strategic technology trends defined by Gartner, Event Driven Architecture (EDA) is gradually becoming the mainstream technology architecture. According to Gartner's estimate, 60% of new digital business solutions will use EDA, and EDA accounts for half of the technology stack that commercial organizations participate in.

Currently, more successful enterprises have realized that in order to maximize operational efficiency and customer experience, it is necessary to closely combine business and technical measures. The changes in operational events or business situations are the focus of attention for many enterprises nowadays, and these changes can bring practical and useful information to enterprise leaders. The main purpose of architecture design is to obtain insights from customer contacts, transactions, operations, and other aspects of information, which complement each other. Traditional technology has always had many limitations on the speed at which enterprises can obtain insights from events, such as batch ETLs (extraction, transformation, loading) used to record, collect, and process such events. Based on the above background, Alibaba Cloud EventBridge has emerged.

EventBridge is a specific implementation product driven by events and also the best practice for EDA.

What is Event Driven (EDA)

As early as 2018, Gartner's evaluation report listed Event Driven Model as one of the top 10 strategic technology trends, and Event Driven Architecture (EDA) will become the mainstream of microservices in the future. The report also makes the following assertions:

By 2022, the software model for event notification will become a solution for over 60% of new digital businesses;

By 2022, over 50% of commercial organizations will participate in the ecosystem of event driven digital business services.

I really like the sentence George Santayana said in 'The Life of Reason', 'This who fails to learn History is doomed to repeat it.'. Let's take history as a lesson and see why architecture evolved to event driven.

The above diagram is about the timeline of architecture evolution. Architecture itself has no distinction between advantages and disadvantages. It is a set of technical decisions that determine the development of all functions in subsequent projects (frameworks, coding specifications, documents, processes...). Therefore, we will not discuss the quality of selection here, but only discuss why certain frameworks are introduced and what problems this framework solves in software development.

Single architecture: In a single node service, all modules of a single application are encapsulated and run in a single process, and communication is completed through the same stack calls. This mode can easily lead to unclear structure and relationships, making it difficult to make changes and refactoring to the system. Just like an opaque, sticky, fragile, and stiff Big Ball of Mud!

Layered architecture: In classic layered architectures, layers are used in a very cautious manner. That is, a layer can only know the data of the layer below it. In subsequent practical applications, a more common approach is for a layer to access any layer below it. Layered architecture solves the problem of logical separation in single architecture, where each layer can be equivalently replaced, and is more standardized by layer differentiation. At the same time, a layer can be used by several different/higher level layers. Of course, there are also obvious drawbacks to layers. Layers cannot encapsulate everything, such as adding a field to the UI that may also need to be added to the DB, and adding extra layers can seriously damage system performance.

MVC architecture: The reason for the emergence of MVC architecture is actually very simple. As the complexity of business systems increases, the so-called "full stack engineer" no longer applies to most scenarios. In order to reduce the integration complexity of the front-end and back-end, MVC architecture has been promoted. Among them, Model represents business logic; View represents the view layer, such as a small component of the front-end UI; Controller provides coordination between View and Model, such as converting a user's operation into business logic. In addition, there are many extension architectures, such as Model View Presenter, Model View Presenter ViewModel, Resource Method Representation, Action Domain Responder, which won't be discussed in detail. Interested students can search on the wiki.

EBI architecture: i.e. Entity, Boundary, and Controller. The EBI architecture views system boundaries as complete connections, not just views, controllers, or interfaces. The entity of EBI represents the actual entity that holds data and terminates related behavior, similar to Alibaba Cloud's POP API. EBI is mainly a backend concept that complements MVC.

Onion architecture: Onion architecture is a low coupling, high cohesion architecture model. All applications are built around independent object models, with interfaces defined on the inner layer and implemented on the outer layer. The coupling direction converges towards the center, and all code can be compiled and run independently with the infrastructure.

SOA Architecture: SOA is the abbreviation for Service Oriented Architecture, which means Service Oriented Architecture. Each function is provided through an independent service, which defines a clear callable interface, and the orchestration and invocation between services can complete a complete business. In fact, this architecture is also the most mature and commonly used architectural pattern among current architectures.

After introducing all the architectural trends before, let's look back at what EDA architecture is.

EDA Event Driven Architecture is a system architecture model whose core capability lies in the ability to discover system "events" or important business moments (such as transaction nodes, site visits, etc.) and take necessary actions on corresponding events in real-time or near real-time. This pattern replaces the traditional "request/response" model, in which services must wait for a response before proceeding to the next task. The process of event driven architecture is provided and run by events.

The above diagram actually provides a good explanation of the EDA architecture model, but it is not yet clear enough. Therefore, we will compare it with the individual architecture to see the differences between them.

In the above comparison graph, we can actually see the difference between it and traditional architecture more clearly. In general traditional architecture, after the creation of an order operation occurs, a series of operations are actually completed through a system. The concept of event driven is to convert all operations into "event" concepts, and downstream determines which system is called to complete what operations by capturing a certain "event".

Looking back at 'events', the important part we just introduced is actually converting operations into certain types of events for distribution. How do we define this event?

Simply put, an event is actually a significant change in state that is triggered when a user takes a specific action. Taking 4S stores selling cars as an example:

When a customer purchases a car and its status changes from For Sale to Sold, it is an event;

After a successful transaction, deducting the amount from the account is an event;

After clicking on the reservation test drive, adding the reservation information to the designated user is an event;

Each event may trigger one or more options as a response.

In fact, the Cloud Native CNCF Foundation hosted the open source CloudEvents project in 2018, which aims to describe events in a unified and standardized format to enhance interoperability between different services, platforms and systems. Under the definition of this project, the general event specification is as follows:

Events are mainly composed of Json bodies, which describe the events that occur through different fields.

In summary, event driven is actually encapsulating important business moments into "events" and routing them to downstream systems through a certain EventBus.

I have understood the entire processing process of EDA architecture, but have not yet solved what this so-called "EventBus" is?

As shown in the figure above, the core logical architecture of EventBus is composed of both Event Producer and Event Consumer, and the intermediate links are decoupled through Bus. Is it very similar to a traditional MQ architecture? Don't worry, the complex parts of this architecture will be explained in the upcoming implementation practice section.

Practical Reflection on the Implementation of EDA Architecture

Before introducing the implementation practice, let's first take a look at a classic EDA architecture model:

This is a very classic EDA order architecture that mainly uses EventBridge and FC function calculations (if you are not familiar with FaaS, you can consider the FC node as a POD node of ECS or Kubernetes), driving collaboration among various businesses through events.

So the central node (EventBridge) in this area actually has three important capabilities:

1. For Event Capturing: Possess the ability to collect events;

2. For Routing: The ability to distribute event routing downstream through event content;

3. For Event Processing: The ability to desensitize or preliminarily filter and filter events.

Usually, achieving these three abilities is quite difficult, for example: Event Capturing may require familiarity with Dell Boomi, Snaplogic, MuleSoft, Dataflow, Apache Apex, etc. The Routing section may require knowledge of RocketMQ, RabbitMQ, ActiveMQ, Apache Kafka, and Event Processing may require knowledge of Apache Storm and Apache Flink. So the logical architecture mentioned earlier is actually very ideal, and in order to achieve EDA event driven completion, these core capabilities need to be included.

In fact, we can also glimpse some information from the architecture just now. The EDA architecture actually doesn't seem so simple, so what are its advantages and disadvantages?

Below is a brief list of the advantages of EDA architecture in practice:

Loose coupling: the event driven architecture is a highly loosely coupled and highly distributed architecture model. The event creator (source) only knows the event that occurred, but does not know how to handle the event, and also cares how many interested parties subscribe to the event;

Asynchronous execution: The EDA architecture is the most suitable execution tool in asynchronous scenarios, where we can keep the required events in the queue until the state is normal before execution;

Scalability: Event driven architecture can quickly partition services through routing and filtering capabilities, providing more convenient extension and routing distribution;

Agility: Event driven architecture can provide more agile and efficient deployment solutions by distributing events to anywhere.

Of course, the disadvantages are also obvious:

Complex architecture: Event driven architecture is complex, with multiple routing nodes, complex system formation, and multiple functional requirements;

Difficulty in routing distribution: Event routing distribution is difficult, and flexible event routing requires strong real-time computing power, which requires high requirements for the overall distribution system;

Unable to track: Event tracking is the guarantee of the entire EDA architecture, and it is often difficult to track the event processing status in the EDA architecture, requiring a large amount of customized development;

Poor reliability: Event driven systems typically have poor reliability due to the need for multiple system integration, and delivery cannot be guaranteed.

How to solve the dilemma in EDA scenarios?

In response to these issues faced by EDA scenarios, Alibaba Cloud has launched EventBridge, a serverless event bus service. Its mission is to serve as a hub for cloud events, connecting cloud products and applications, applications, and applications using the standardized CloudEvents 1.0 protocol, providing centralized event governance and driving capabilities, and helping users easily build loosely coupled and distributed event driven architectures; In addition, there are a large number of vertical SaaS services in the cloud market outside of Alibaba Cloud. EventBridge will assist customers in creating a complete, event driven, efficient and controllable cloud experience with excellent cross product, cross organization, and cross cloud integration and integration capabilities.

Alibaba Cloud has defined EventBridge, and its core values include:

• Unified event hub: Unified event interface, defined event standards, breaking cloud product event silos;

Event driven engine: Massive event sources with millisecond level triggering capability, accelerating EDA/Serverless architecture upgrades;

Open and Integrated: Provide rich cross product and cross platform connectivity capabilities, promoting the integration of cloud products, applications, and SaaS services.

Below is an introduction to EventBridge from the architectural and functional perspectives:

Architecture level

In response to complex architecture issues, EventBridge provides industry-wide management capabilities for Source, Buses, Rules, and Targets modules, while supporting both EventBus and EventStream modes, significantly reducing the difficulty of event driven architecture.

1) The N: N model of the classic EDA (Event Driven) scenario in the event bus model provides core capabilities such as multi event routing, event matching, and event transformation, helping developers quickly build event driven architectures.

2) The event flow model standard Streaming (1:1) streaming processing scenario, with no bus concept, is used for end-to-end data dumping, data synchronization, and data processing, helping to easily build end-to-end data pipeline services on the cloud.

Functional level

At the functional level, the core highlight applications of EventBridge include:

1) Event rule driven

For event based routing distribution, EventBridge is driven by event rules, supports 8 major event modes, and 4 re converters to meet all the requirements of routing distribution.

2) Event tracking

Exclusive event tracking and analysis/query capabilities are provided for events that cannot be tracked. Perfect full link event query and analysis capabilities for users.

3) DLQ/retry mechanism, full process triggering of events

In response to poor reliability, it supports DLQ/retry mechanism and triggers the entire event process, greatly ensuring event failures and delays caused by downstream systems of users.

4) Schema Registry

In response to the complexity of event management, it supports a Schema registry, supports the interpretation, preview, and upstream and downstream code generation capabilities of event information, and helps users complete the sending and receiving of events in low code. Solve a series of event management issues such as difficulties in cross departmental information communication and redundant business codes.

5) Meanwhile, based on the above functions, EventBridge supports docking with more than 85 Alibaba Cloud products and 847 event types.