This article aims to provide a clear understanding of the recent high-frequency words "Message-Driven, Event-Driven, and Streaming" in messaging field. Among them, Event-Driven Architecture (EDA) is one of the top ten technology trends according to Gartner, and EventBridge is one of the current focuses as the next-generation messaging middleware.
Alibaba Cloud RocketMQ 5.0 has been upgraded from message to the integration of message, event, and streaming. Based on this, Message-Driven, Event-Driven, and Streaming become high-frequency words in the messaging field recently. Since the three words are newly coined, many people may be confused.
The translation of these three words:
These three models are similar to asynchronous communication. The sender of the message will not wait for the consumer to respond to any data. Service decoupling is a common feature of the three models.
In the field of service communication, the following elements should also be considered in model selection:
After the background introduction, let's look at each model in detail.
The procedure in message-driven communication is when a Producer sends a message to a Consumer. The model is listed below:
Middleware is usually used in this model. Common Middleware includes RocketMQ, Kafka, and RabbitMQ. The function of the middleware is to cache the messages sent by the producer until the consumer is ready to receive them, thereby decoupling the systems on both ends.
The format of messages in the message-driven architecture is based on the needs of consumers; messaging can be one-to-one, many-to-many, one-to-many, or many-to-one.
A common example is the pushing of commodity orders. The upstream component is responsible for generating orders, and the downstream is responsible for receiving and processing orders. As such, the upstream does not need to care too much about the entire order lifecycle and focuses on generating orders quickly, so the performance of individual components can be improved.
The message-driven model provides light coupling between services (this coupling refers to Producer/Consumer SDK) and allows production and consumption to be extended according to demands.
First of all, please note that the event-driven model is an improvement on the message-driven model. It imposes strict restrictions on the message size and format. This message-based restriction encapsulation is called an Event.
The producer publishes events in the event-driven model to indicate system changes, and any interested and authorized service can subscribe to these events and use them as triggers to initiate certain logic/storage/tasks.
Messaging can be one-to-one, many-to-one, one-to-many, or many-to-many in an event-driven model. Typically, multiple targets execute different events based on filtering conditions.
The format of the event in this model is formulated by the producer according to the event standard protocol. Thanks to the standard restrictions and encapsulation, the producer does not need to care about which systems are consuming the events it generates.
Events are not commands. Events do not tell consumers how to process messages. Their role is only to inform consumers that an event occurs at a certain time. As immutable data, events are important and have the same data value as messages. Normally, when an event occurs and is executed, it is often accompanied by another event.
The event-driven model provides minimal coupling between services and allows production and consumption services to be extended according to demands. It allows for the addition of various new components without affecting existing services.
There is a pertinent example of the event-driven model. We take a customer that purchases a product as an event to demonstrate the application of the model in the event scenario:
The event-driven model can be applied anywhere!
It is possible to filter, transform, schema, and search events in the productization of EventBridge, with the standardized encapsulation for messages. These capabilities also extend to more specific functions and related features for event-driven scenarios.
Streaming is an ordered set of unbounded events or data. The execution operation is usually performed on a fixed event segment (e.g. 00:00 - 12:00) or a relative event (e.g. the past 12 hours).
Usually, the operation of a single event is implemented using the event itself, but the operations on the streaming are filtering, combining, splitting, mapping, etc.
The operations on the streaming can be stateless or stateful:
A simple example of streaming can be given here. For example, the logistics system will generate an event when an item passes through a logistics node, but if we want to know the complete logistics status event, there must be an aggregation of individual events of each logistics node. This aggregation event is a streaming event.
Kafka is the most typical streaming middleware. The location information of events is critical in streaming scenarios. Typically, the location information (e.g. offset) is managed by the consumer.
After defining Event and Streaming, let's discuss the specifications.
The event specifications are to clarify the relationship between event producers and consumers. Currently, there are two major parts: AsyncAPI and CloudEvents.
The following introduces some related concepts of CloudEvents. The core function of CloudEvents is to define a set of metadata about events transmitted between different components and how the metadata should appear in the message body.
The main ideas of CloudEvents are listed below:
Occurrence refers to the logical occurrence of an event. An event occurs based on a certain situation.
Event indicates the data record of the event and the context. Routing can be determined based on the information in the event, but the event itself does not contain routing information.
Producer: This is the instance or component that creates the event.
Source: This is the context in which events occur, which can be composed of multiple producers.
Consumer: Receives and consumes events
Intermediary receives messages containing events and forwards them to the next receiver, similar to a router.
Context: Context metadata is encapsulated in context attributes to determine the relationship between events and other systems.
Data: It can also be called payload.
EventFormat: The event format, like JSON
Message: A message encapsulates the event and passes it from source to destination.
Protocol: It can be an industry standard (such as HTTP), an open-source protocol (such as Kafka), or a vendor protocol (such as AWS Kinesis).
Protocol Binding describes how to send and receive events through a given protocol and how to put events into messages.
ApsaraDB - December 29, 2021
Alibaba Cloud Native Community - December 16, 2022
DavidZhang - July 5, 2022
Alibaba Cloud Storage - April 10, 2019
Hologres - June 16, 2022
Alibaba Clouder - June 17, 2020
EventBridge is a serverless event bus service that connects to Alibaba Cloud services, custom applications, and SaaS applications as a centralized hub.Learn More
An array of powerful multimedia services providing massive cloud storage and efficient content delivery for a smooth and rich user experience.Learn More
Alibaba Cloud Function Compute is a fully-managed event-driven compute service. It allows you to focus on writing and uploading code without the need to manage infrastructure such as servers.Learn More
Make identity management a painless experience and eliminate Identity SilosLearn More
More Posts by Alibaba Cloud Native Community