This topic describes the architecture of Message Queue for Apache Kafka and the publish/subscribe pattern.

Architecture of Message Queue for Apache Kafka

A typical Message Queue for Apache Kafka cluster consists of four parts:

Producer
sends messages to the Message Queue for Apache Kafka broker in push mode. The messages sent can be page visits (PVs), server logs, and information related to system resources such as CPU utilization and memory usage.
Broker
is a server that stores messages. You can add Message Queue for Apache Kafka brokers as needed. The more Message Queue for Apache Kafka brokers there are, the higher the throughput of the Message Queue for Apache Kafka cluster is.
Consumer group
subscribes to and consumes messages from the Message Queue for Apache Kafka broker in pull mode.
ZooKeeper
manages the cluster configuration, elects the leader partition, and balances the load when the consumer group changes.
系统架构

The publish/subscribe pattern of Message Queue for Apache Kafka

Message Queue for Apache Kafka uses a publish/subscribe pattern:

  • The relationship between consumer groups and topics is N:N. One consumer group can subscribe to multiple topics, and one topic can be subscribed to by multiple consumer groups.
  • One message under a topic can be subscribed to by multiple consumer groups but can be consumed only by one consumer in the same consumer group.
发布/订阅