Message Queue for Apache RocketMQ provides asynchronous decoupling and load shifting for distributed application systems, and supports features for Internet applications, including massive message accumulation, high throughput, and reliable retry.

Network access

In addition to internet access, Message Queue for Apache RocketMQ supports private networks for application infrastructure, namely Virtual Private Cloud (VPC) access. You have full control over your VPC, which you can define and customize by specifying the IP address range and configuring route tables and network gateways. You can also launch Alibaba Cloud resources such as Elastic Compute Service (ECS), Relational Database Service (RDS), and Server Load Balancer (SLB) in your own VPC.

Scenarios

  • Load shifting

    Large activities, such as flash sales, red envelope snatching, and enterprise success, may cause high traffic pulses, and the system may become overloaded or even crash due to lack of proper protection, or the user experience may be affected because too many requests fail due to too many limits. Message Queue for Apache RocketMQ provides the load shifting feature to solve this problem.

  • Asynchronous decoupling

    As the core system of Taobao/Tmall primary sites, the transaction system can attract the attention of hundreds of downstream business systems, including logistics, shopping cart, credits, and stream computing analysis when each transaction order is created. The overall business system is large and complex. Message Queue for Apache RocketMQ supports asynchronous communication and application decoupling, ensuring the continuity of services on the primary site.

  • Sending and subscription of ordered messages

    Several scenarios need to ensure the sequence in daily life, such as the time-first principle of securities trading, order creation, payment, refund, and other processes in the trading system, as well as the handling of boarding messages of passengers on flights. Ordered messages in Message Queue for Apache RocketMQ are sent and received in the first-in-first-out (FIFO) order.

  • Consistency of distributed transactions

    Final data consistency must be ensured in scenarios such as transaction systems and payment envelopes. Message Queue for Apache RocketMQ distributed transactions can implement decoupling between systems and ensure final data consistency.

  • Big data analysis

    Data creates values during movement. Traditionally, data analysis is mostly based on the batch computing model and cannot be performed in real time. Message Queue for Apache RocketMQ together with Stream Compute of Alibaba Cloud allows you to conveniently analyze your service data in real time.

  • Distributed cache synchronization

    During the Tmall Double 11 Shopping Festival, the changes to the commodity prices in different activities need to be perceived in real time. A large number of concurrent accesses to the database results in slow page response. The centralized cache restricts the traffic for commodity changes due to bandwidth bottlenecks. To solve this problem, Message Queue for Apache RocketMQ provides a distributed cache to notify commodity data changes in real time.

For more information, see Scenarios.

Terms

  • Topic: The topic of the message. It is a level-1 message type. The producer sends messages to the topic.
  • Producer: A producer, also known as a publisher, produces and sends messages to a topic.
  • Consumer: A consumer, also known as a subscriber, receives and consumes messages from a topic.
  • Message: A message is a combination of the data that the producer sends to a topic and the attributes (optional).
  • Message attribute: A producer can define attributes for a message, including the message key and tag.
  • Group: A group is a type of producer or consumer that produces or consumes the same type of messages with the same publishing or subscription logic.

For more information about the concepts of Message Queue for Apache RocketMQ, see Terms.

Message publish/subscribe pattern

Message Queue for Apache RocketMQ supports the publish/subscribe pattern. A message producer can create a topic and send messages to the topic. A consumer can subscribe to a topic to receive messages from it. Communication can be in one-to-many (fan-out), many-to-one (fan-in), or many-to-many mode.

The pattern is illustrated in the following figure.
Figure 1. Message publish/subscribe pattern
messagingmodel
  • Producer cluster: A producer cluster indicates a message sending application. A producer cluster contains multiple producer instances, which can be multiple machines, or multiple processes of one machine, or multiple producer objects of one process.

    A producer cluster can send messages of multiple topics. When a distributed transactional message is sent, if the producer experiences an unexpected downtime, the broker actively calls back any machine in the producer cluster to confirm the transaction status.

  • Consumer cluster: A consumer cluster indicates a message consumption application. A consumer cluster contains multiple consumer instances, which can be multiple machines, multiple processes, or multiple consumer objects of one process.

    Multiple consumers in a consumer cluster consume messages in even mode. In broadcasting consumption pattern, all instances in the consumer cluster consume all the messages.

    One consumer cluster corresponds to one group ID, and a group ID can subscribe to multiple topics, such as Group 2 in Figure 1. You can configure the subscription relationships between groups and topics in the program. For more information, see the section Optimization of Resource Application Process in Updates.

Features

Message Queue for Apache RocketMQ provides highly available cloud messaging services in multiple regions of Alibaba Cloud. Multiple data centers are deployed in a single region, which makes the services highly available. Even if the entire data center is unavailable, message publishing services can still be provided for applications.

Message Queue for Apache RocketMQ allows access through TCP or HTTP and multiple programming languages. You can deploy applications on Alibaba Cloud Elastic Compute Service (ECS) instances and enterprise-built clouds, or embed them into mobile devices or Internet of Things (IoT) devices and then connect the applications to Message Queue for Apache RocketMQ to send and subscribe to messages. Local developers can also access Message Queue for Apache RocketMQ over the Internet to send and subscribe to messages.functionswithoutstomp
  • Multi-protocol access
    • HTTP: Message Queue for Apache RocketMQ supports HTTP and RESTful, making it easy to use, fast to access, and powerful in cross-network access. Message Queue for Apache RocektMQ supports clients in seven programming languages: Java, C++, .NET, Go, Python, Node.js, and PHP.
    • TCP: Different from the HTTP-based access mode, the TCP-based SDK access mode is more specialized, reliable, and stable. Supported languages include Java, C/C++, and .NET.
  • Management tools
    • Web console: It supports topic management, group management, message query, message trace display and query, resource report, and monitoring and alert management.
    • API: It allows you to integrate Message Queue for Apache RocketMQ management tools into your own console.
  • Message types
    • Normal messages: Normal messages do not have any special features in Message Queue for Apache RocketMQ. They are different from scheduled and delayed messages, ordered messages, and transactional messages.
    • Transactional messages: Message Queue for Apache RocketMQ provides a distributed transaction processing function similar to X/Open XA to ensure transaction consistency.
    • Scheduled messages and delayed messages: Message Queue for Apache RocketM allows producers to specify the length of time to wait before a scheduled or delayed message is delivered. The maximum length is 40 days.
    • Ordered messages: Message consumers can consume messages in the order in which messages are delivered.
  • Features
    • Query messages: Message Queue for Apache RocketMQ provides you with three ways to query messages: by message ID, by message key, and by topic.
    • Query a message trace: Message traces clearly display the complete route from message delivery by message producers, to the Message Queue for Apache RocketMQ broker, to message consumption by consumers. It facilitates troubleshooting.
    • Clustering consumption and broadcasting consumption: In clustering consumption pattern, a message only needs to be processed by any consumer in the cluster in Message Queue for Apache RocketMQ. In broadcasting consumption pattern, Message Queue for Apache RocketMQ broadcasts each message to all consumers registered in the cluster to ensure that the message is consumed by each consumer at least once.
    • Reset consumer offsets: You can reset the consumer progress by time to trace back messages or discard accumulated messages.
    • Dead-letter queue: Messages that cannot be consumed are stored into a special dead-letter queue for subsequent processing.
    • Message router: Message routers are often used to synchronize messages between regions to ensure data consistency between regions.
    • Statistics: This module allows you to collect statistics about message production and consumption. It allows you to view not only the total number of messages that a topic received or the message receiving transactions per second (TPS) of the topic during a certain period of time, but also the total number of messages that a topic sent to a group ID or its message sending TPS to that group ID during a certain period of time.
    • Alerts: You can use the monitoring and alert function of Message Queue for Apache RocketMQ to monitor the message consumption status of a topic subscribed to by a group ID and receive alert messages. This helps you learn the message consumption status in real time and handle consumption exceptions in a timely manner.