This topic describes the scenarios of Message Queue for Apache RocketMQ to help you better use Message Queue for Apache RocketMQ in your business.

For example, the business of an Internet e-commerce enterprise covers a variety of aspects such as registration, ordering, inventory, and logistics. The enterprise needs to cope with business peaks during promotional activities such as flash sales, anniversary celebrations, and regular special offers. These activities pose great challenges to the processing performance of various microservice applications in a distributed system.

Message Queue for Apache RocketMQ is an important component of distributed systems. It can cope with these challenges, for example, by implementing asynchronous decoupling for applications.

This topic first describes how Message Queue for Apache RocketMQ implements the following features in the scenario of user registration:

  • Asynchronous decoupling
  • Data consistency of distributed transactions
  • Sending and subscription of ordered messages

Then, this topic describes how Message Queue for Apache RocketMQ implements load shifting and cache synchronization for large-scale servers in the scenarios of flash sales and price synchronization for e-commerce enterprises.

Asynchronous decoupling

Traditional implementation

In a common scenario, after a user registers, the system needs to send email and SMS notifications to notify the user that the registration is successful. Traditional implementation of this requirement is classified into two modes:

  • Serial mode
    The following figure shows the registration process in serial mode. serial

    The preceding registration process consists of the following steps:

    1. The user enters an account and a password and submits registration information on the registration page. The registration information is written to the registration system.
    2. The registration system sends a request to the email notification system. After the email notification system receives the request, the email notification system sends an email notification to the user.
    3. The email notification system sends a request to the SMS notification system. After the SMS notification system receives the request, the SMS notification system sends an SMS notification to the user.

    After all the preceding steps are complete, a registration result is sent to the client and the user can log on with the registered account.

    If each step takes 50 milliseconds, the user needs to wait 150 milliseconds on the registration page before the user can log on.

  • Parallel mode
    The following figure shows the registration process in parallel mode. parallel

    The preceding registration process consists of the following steps:

    1. The user enters an account and a password and submits registration information on the registration page. The registration information is written to the registration system.
    2. The registration system sends requests to the email notification system and the SMS notification system at the same time. After the two notification systems receive the requests, the email notification system sends an email notification to the user and the SMS notification system sends an SMS notification to the user.

    After all the preceding steps are complete, a registration result is sent to the client and the user can log on with the registered account.

    If each step takes 50 milliseconds, the user needs to wait 100 milliseconds on the registration page before the user can log on. This is because the email notification and SMS notification are sent to the user at the same time.

The following section describes the implementation of Message Queue for Apache RocketMQ in the scenario of user registration.

Asynchronous decoupling

A user can have a better experience if the user can log on as soon as the registration system stores the account information of the user. The user does not need to check the SMS and email notifications at the earliest opportunity.

The registration system does not necessarily need to wait for the SMS and email notification systems to send notifications on successful registration. After data is written to the registration system, the registration system can put the required operations into Message Queue for Apache RocketMQ and immediately return the registration result to the user. The required operations are asynchronously completed in Message Queue for Apache RocketMQ. withRMQ

The preceding registration process consists of the following steps:

  1. The user enters an account and a password and submits registration information on the registration page. The registration information is written to the registration system.
  2. The registration system sends a message to Message Queue for Apache RocketMQ. Message Queue for Apache RocketMQ immediately sends a response to the registration system. The registration is complete. The user can log on.
  3. The email and SMS notification systems consume the registration message from Message Queue for Apache RocketMQ and send email and SMS notifications to the user. The registration process is complete.

In this case, the user needs to wait only 55 milliseconds on the registration page, during which the registration data is written to the registration system and Message Queue for Apache RocketMQ.

Asynchronous decoupling is a key feature of Message Queue for Apache RocketMQ. This feature is designed to reduce the response time and implement decoupling. This feature is applicable to operations that are time-consuming and do not require immediate responses. The operations can be put into message queues as messages. The message sender and receiver are decoupled by Message Queue for Apache RocketMQ. On the condition that the message format remains unchanged, the message sender and receiver do not communicate with or affect each other.

Data consistency of distributed transactions

In the registration process, users enter registration information on the web page of the registration system while notifications are sent by the email notification system. The data in the two systems must be consistent.

Processing of normal messages

As described in the preceding section, the registration system and the email notification system implement asynchronous processing by using message queues. After registration information is written to the registration system, the registration system sends a message that indicates successful registration to Message Queue for Apache RocketMQ. The email notification system consumes the registration message from Message Queue for Apache RocketMQ and sends an email to notify the user of successful registration. inconsistentstatus

The preceding registration process consists of the following steps:

  1. The registration system initiates registration.
  2. The registration system sends a message that indicates successful or failed registration to Message Queue for Apache RocketMQ.

    2.1 If the message is sent to Message Queue for Apache RocketMQ, the process goes to Step 3.

    2.2 If the message fails to be sent, the email notification system fails to receive the message that indicates successful or failed registration from Message Queue for Apache RocketMQ. In this case, the email notification system cannot send an email notification to the user. In this case, the status data in the email notification system and registration system is inconsistent.

  3. The email notification system receives the message that indicates successful registration from Message Queue for Apache RocketMQ.
  4. The email notification system sends an email notification that indicates successful registration to the user.

In this process, the systems are decoupled and the upstream system does not care about the processing result of the downstream system. However, the consistency between the status data in the email notification system and registration system is not ensured.

Processing of transactional messages

To ensure the consistency of the status data between different systems, you can use transactional messages that are provided by Message Queue for Apache RocketMQ. consistentstatus

The preceding registration process consists of the following steps:

  1. The registration system sends a half transactional message to Message Queue for Apache RocketMQ.

    1.1 If the half transactional message is sent to Message Queue for Apache RocketMQ, the process goes to Step 2.

    1.2 If the half transactional message fails to be sent, the registration system does not register the user but ends the process.
    Note After this step, the data in the registration system is consistent with the data in the email notification system.
  2. The registration system starts to register the user.

    2.1 If the registration succeeds, the process goes to Step 3.1.

    2.2 If the registration fails, the process goes to Step 3.2.

  3. The registration system sends the status of the half transactional message to Message Queue for Apache RocketMQ.

    3.1 If the half transactional message is submitted, a message that indicates successful registration is generated and the process goes to Step 4.

    3.2 If the half transactional message is rolled back, no message that indicates successful registration is generated and the process ends.
    Note After this step, the data in the registration system is consistent with the data in the email notification system.
  4. The email notification system receives the message that indicates successful registration from Message Queue for Apache RocketMQ.
  5. The email notification system sends an email notification that indicates successful registration to the user.
    Note After this step, the data in the registration system is consistent with the data in the email notification system.

For more information about distributed transactional messages, see Transactional messages.

Sending and subscription of ordered messages

Ordered messages in Message Queue for Apache RocketMQ are classified into two types:

  • Globally ordered messages: All messages under a specified topic are published and consumed in strict first-in-first-out (FIFO) order.
  • Partitionally ordered messages: All messages under a specified topic are partitioned by a sharding key. Messages in the same shard are published and consumed in strict FIFO order. This ensures that one message is consumed by one process.

    In the scenario of user registration, you can specify user ID as a sharding key for partitioning. Registration messages that are created, updated, or deleted in the same shard must be published and consumed in FIFO order.

For more information about ordered messages, see Ordered messages.

Load shifting

Load shifting is another frequently used feature of Message Queue for Apache RocketMQ. For example, you can use this feature in flash sales or group-buying activities.

In a flash sales or group-buying activity, traffic sharply increases due to large numbers of user requests. After the flash sales application processes huge access traffic, the notification system cannot bear the massive calls. In this case, some notifications may be missed due to issues such as system breakdown. To resolve these issues, Message Queue for Apache RocketMQ can be added between the application and notification system. Load shifting

The preceding flash sales process consists of the following steps:

  1. Users send a large number of requests to the flash sales processing system.
  2. Based on flash sales processing logic, the flash sales processing system sends requests that meet the flash sales conditions to Message Queue for Apache RocketMQ.
  3. The notification system consumes the flash sales messages from Message Queue for Apache RocketMQ and sends messages that indicate successful purchase to the users.
  4. The users receive the notifications on successful purchase.

Cache synchronization of large-scale servers

During Double 11, each venue has a wide variety of items, whose prices may change in real time. In this case, the cache technology cannot meet the requirements for accessing product prices and the network interface controllers (NICs) of cache servers are fully loaded. Multiple queries of product prices may slow down the opening of venue web pages.

In this case, a broadcasting mechanism is required. Originally, one message is consumed by only one server in a cluster. However, in broadcasting consumption mode of Message Queue for Apache RocketMQ, the message is consumed by all the consumers in the consumer cluster. This way, price information is synchronized to all servers, which is similar to the cache feature.

For more information about broadcasting consumption, see Clustering consumption and broadcasting consumption.