This topic describes the terms used in ApsaraMQ for Kafka. This helps you better understand and use ApsaraMQ for Kafka.

Apache Kafka
A distributed and open source platform for data stream processing. This platform allows you to publish, subscribe to, store, and process data streams in real time. For more information, see Apache Kafka documentation.
ApsaraMQ for Kafka
A fully managed Apache Kafka service provided by Alibaba Cloud. This service frees you from deployment and O&M, and offers you a cost-effective service that features high scalability, reliability, and throughput. For more information, see What is ApsaraMQ for Kafka? .
ZooKeeper
A distributed and open source coordination service for applications. In ApsaraMQ for Kafka, ZooKeeper is used to manage clusters and configurations, and elect leaders. ZooKeeper is part of the ApsaraMQ for Kafka architecture. You do not need to care about ZooKeeper.
broker
A server in a ApsaraMQ for Kafka cluster. ApsaraMQ for Kafka provides a fully managed service that automatically changes the broker quantity and configuration based on the traffic specifications of your instances. You do not need to care about the specific broker information.
cluster
A collection of multiple brokers.
instance
An independent ApsaraMQ for Kafka resource entity that corresponds to a cluster.
VPC-connected instance
An instance that provides only a virtual private cloud (VPC) endpoint, and can be accessed only from the VPC in which the instance is deployed.
Internet- and VPC-connected instance
An instance that provides public and VPC endpoints, and can be accessed from the VPC in which the instance is deployed and the Internet.
major version upgrade
An upgrade across versions. For example, you can upgrade a ApsaraMQ for Kafka instance from version 0.10.x to version 2.x. For more information, see Upgrade the versions of instances.
minor version update
An update that does not cross versions. For example, you can update a ApsaraMQ for Kafka instance from version 0.10 to version 0.10.2, or from version 0.10.2 to the 0.10.2 kernel-optimized version. For more information, see Upgrade the versions of instances.
endpoint
An address used by producers or consumers to connect to ApsaraMQ for Kafka. This address can consist of the IP address and port number of a broker in the format of Broker IP address:Port number. This address can also consist of the domain name and port number of a ApsaraMQ for Kafka instance in the format of Instance domain name:Port number. For more information, see Comparison among endpoints.
message
A carrier for transferring information in ApsaraMQ for Kafka. Messages can be page views, server logs, and the information about system resources such as CPU and memory resources. In ApsaraMQ for Kafka, a message is a byte array.
message retention period
The maximum message retention period when the disk capacity is sufficient.
  • After the storage usage exceeds 85%, the system deletes messages in the order in which the messages are stored. The earliest message that is stored is the first message that is deleted. This helps ensure that the performance of the service is not degraded.
  • Valid values: 24 to 480. Default value: 72. Unit: hours.
maximum message size
The maximum size of a message that you can send and receive in ApsaraMQ for Kafka.
  • You can specify a message size up to 10 MB for instances of the Standard Edition and the Professional Edition.
  • Before you change the maximum message size to a new value, make sure that the new value is the same as the maximum message size that you specified for message producers and message consumers in the instance.
publish/subscribe pattern
An asynchronous communications model between services. A publisher sends a message directly to a specified topic, and does not need to know the consumer that subscribes to the message. A consumer receives a message directly from a specified topic, and does not need to know the publisher that sends the message. ApsaraMQ for Kafka uses the publish/subscribe pattern. For more information, see Publish/subscribe model of ApsaraMQ for Kafka.
subscription relationship
The subscription relationship between consumer groups and topics. In ApsaraMQ for Kafka, you can query the status of online consumer groups that have subscribed to a specified topic. The status of offline consumer groups cannot be queried.
producer
An application that sends messages to ApsaraMQ for Kafka.
consumer
An application that receives messages from ApsaraMQ for Kafka.
group
A group of consumers that have the same group ID. When a topic is consumed by multiple consumers in the same consumer group, each message in the topic is delivered only to one consumer. This balances the load over consumers and allows messages in a topic to be concurrently consumed.
topic
A message topic that is used to classify messages.
topic traffic rebalance
An operation that evenly redistributes the topic traffic after a ApsaraMQ for Kafka cluster is scaled out. For more information, see Rebalance the topic traffic.
partition
A message partition that is used to store messages. A topic consists of one or more partitions. Messages in each partition are stored on one or more brokers.
offset
A sequence number that is assigned to a message when the message arrives at a partition.
earliest offset
The earliest offset in a partition. This is the offset for the first message in the current partition. For more information about how to query the earliest offset in the current partition, see View partition status.
latest offset
The latest offset in a partition. This is the offset for the latest message in the current partition. For more information about how to query the latest offset in the current partition, see View partition status.
consumer offset
The latest offset of messages that are consumed in a partition. For more information about how to query the consumer offset, see View consumption details.
latest consumption time
The time when the latest message consumed by a consumer group was published to and stored on the ApsaraMQ for Kafka broker. If no message accumulation occurs, the time is close to the message sending time.
message accumulation
The total number of accumulated messages in the current partition. The value is equal to the latest offset minus the consumer offset. Message accumulation is a key metric. If a large number of messages are accumulated, consumers may be blocked, or the consumption speed cannot keep up with the production speed. In this case, you must analyze the status of consumers and improve the consumption speed. You can clear all accumulated messages, and start to consume messages from the latest offset or reset consumer offsets based on points in time. For more information, see Reset consumer offsets.
local storage
A storage engine that uses the In-Sync Replicas (ISR) algorithm of native Apache Kafka. If you have special requirements, such as compaction, idempotence, transactions, and partitionally ordered messages, we recommend that you use local storage. For more information, see Storage engine comparison.
cloud storage
A storage engine that uses the algorithm for Alibaba Cloud disks. Cloud storage has the benefits of the underlying storage of Alibaba Cloud. Compared with local storage, cloud storage provides better performance in auto scaling, reliability, availability, and cost-effectiveness. Therefore, in most cases, we recommend that you use cloud storage. For more information, see Storage engine comparison.
cleanup.policy
A log cleanup policy. Configure a log cleanup policy only if you use local storage. ApsaraMQ for Kafka supports the following log cleanup policies:
  • Delete: The default log cleanup policy is used. If sufficient storage space is available in the system, messages are retained based on the maximum retention period. After the storage usage exceeds 85%, the system deletes messages in the order in which the messages are stored. The earliest message that is stored is the first message that is deleted. This helps ensure that the performance of the service is not degraded.
  • Compact: The Apache Kafka log compaction policy is used. Log compaction ensures that Apache Kafka retains at least the last known value for each message key. This policy applies to scenarios such as restoring the system state after the application crashes or the system fails, or reloading caches after the application restarts during operational maintenance. For example, when you use Kafka Connect or Confluent Schema Registry, you must store the information about the system status and configurations in a log-compacted topic.
    Important You can use log-compacted topics only in specific cloud-native components such as Kafka Connect and Confluent Schema Registry. You cannot use the log compaction policy for a topic that is used to send and receive messages in other components. For more information, see aliware-kafka-demos.
normal message
By default, messages of the same key are stored in the same partition in the order in which they are sent. A small number of messages are out of order during cluster restart or downtime. For more information, see Storage engine comparison.
partitionally ordered message
By default, messages of the same key are stored in the same partition in the order in which they are sent. When a cluster failure occurs, the messages are still in order. However, messages in some partitions cannot be sent until the failed instance is restored. For more information, see Storage engine comparison.
connector
A component in ApsaraMQ for Kafka. It is used to synchronize data between ApsaraMQ for Kafka and other Alibaba Cloud services. For more information, see Overview.
tag
A tag that is used to identify ApsaraMQ for Kafka resources. You can use tags to classify ApsaraMQ for Kafka resources based on resource features for easy resource search and aggregation. For more information, see Manage tags.
RAM
A service provided by Alibaba Cloud to manage user identities and resource access permissions. Resource Access Management (RAM) is applicable only to scenarios in which you want to implement access control for users who send and receive messages in the ApsaraMQ for Kafka console or by using API operations. RAM is not applicable to scenarios in which you want to implement access control for users who use SDKs to send and receive messages. For more information, see Overview.
ACL
A service provided by ApsaraMQ for Kafka to manage the permissions of Simple Authentication and Security Layer (SASL) users and clients to send and receive messages by using SDKs. This feature is consistent with that in open source Apache Kafka. The access control list (ACL) feature is applicable only to scenarios in which you want to implement access control for users who use SDKs to send and receive messages. This feature is not applicable to scenarios in which you want to implement access control for users who send and receive messages in the ApsaraMQ for Kafka console or by using API operations. For more information, see Overview.