ApsaraMQ for Kafka limits the total number of topics and partitions per cluster to protect performance and stability. Each partition consumes broker resources -- file handles, memory, and replication threads -- so an excessive partition count degrades throughput and increases latency across the entire cluster.
How partitions affect cluster performance
In ApsaraMQ for Kafka, messages are stored and scheduled by partition. Every partition maps to a directory on each broker that holds a replica. The directory contains an index file and a data file per log segment. As the partition count grows, the following problems intensify:
| Impact area | What happens |
|---|---|
| File handle consumption | Each partition requires open file handles for its index and data segments. Thousands of partitions can exhaust the operating system's file descriptor limit. |
| Replication latency | Brokers replicate partition data to keep replicas in sync. More partitions increase the time required to commit messages, which raises end-to-end message latency. |
| Recovery time | When a broker restarts or fails over, it must reload metadata and catch up on every partition it hosts. More partitions extend the unavailability window during recovery. |
What to do when you reach the limit
If your workload requires more topics or partitions than your current cluster allows:
Consolidate topics. Check whether multiple low-traffic topics can share fewer partitions. Over-partitioning is a common cause of hitting limits prematurely.
Submit a ticket. For more information or to request a quota increase, submit a ticket.