ApsaraMQ for Kafka enforces limits on resources and operations. Stay within these limits to maintain stable cluster performance.
Instability caused by exceeding these limits is not covered by the Service-Level Agreement (SLA) and is not eligible for compensation.
Service limits
The following limits apply to all ApsaraMQ for Kafka instances regardless of edition or billing method.
| Resource | Limit | Notes |
|---|---|---|
| Maximum topics (total partitions) | Supported | ApsaraMQ for Kafka manages data at the partition level. Too many topics or partitions causes storage fragmentation and degrades cluster performance. |
| Decrease partitions for a topic | Not supported | Restricted by Apache Kafka's design. |
| ZooKeeper access | Not exposed | Since Apache Kafka 0.9.0, clients no longer need ZooKeeper access. ApsaraMQ for Kafka partially shares ZooKeeper internally and does not expose it for security reasons. |
| Log on to deployed machines | Not supported | -- |
| Change instance region | Not supported | Region is bound to physical resources at deployment time. To use a different region, release the current instance and purchase a new one. |
| Change instance network properties | Supported | See Upgrade instance configurations. |
| Maximum message size | 10 MB | Messages larger than 10 MB fail to send. |
| Monitoring and alerting | Supported | Monitoring data has a 1-minute delay. |
Supported versions
ApsaraMQ for Kafka supports versions 2.2.x to 3.3.x.
| Instance type | Supported versions |
|---|---|
| Non-serverless | 2.2.x to 2.6.x |
| Serverless | 3.3.x |
To upgrade, see Upgrade instance versions.
Minimum partitions per topic
| Instance type | Storage type | Minimum partitions |
|---|---|---|
| Subscription and pay-as-you-go | Cloud storage | 2 |
| Subscription and pay-as-you-go | Local storage | 1 |
| Serverless | Cloud-native storage | 1 |
A single partition under high traffic can cause data skew and hot spots. Set an appropriate number of partitions for your workload.
Partition-to-topic ratio (non-serverless only)
For non-serverless instances, the partition-to-topic ratio is 1:1. The number of available topics equals the total number of partitions.
Example: You purchase an instance with 50 partitions and select the alikafka.hw.2xlarge traffic specification, which includes 1,000 free partitions. Total partitions = 50 + 1,000 = 1,050, so you can create up to 1,050 topics.
Non-serverless instances purchased by topic count are no longer available for new purchases. Existing instances purchased by topic have a partition-to-topic ratio of 1:16. For Professional Edition instances purchased by topic, the number of available topics equals twice the number of purchased topics.
Access points
| Instance type | Supported endpoints |
|---|---|
| Non-serverless Standard Edition | Default, SSL |
| Non-serverless Professional Edition | Default, SSL, SASL |
| Serverless | Default, SSL, SASL |
Single-partition cloud storage availability
Single-partition cloud storage topics may become unavailable during breakdowns or upgrades. Create topics with more than one partition when possible. If your workload requires a single partition, use local storage.
This restriction applies only to non-serverless instances. Serverless instances provide high availability for single-partition cloud storage topics.
Only Professional Edition instances support local storage as the storage engine. Standard Edition does not.
Maximum messages per batch (non-serverless only)
The maximum number of messages per batch is 32,767. If individual messages are small, set batch.size to no more than 16,384.
Quota limits
Exceeding these limits may cause stability issues. All quotas apply per cluster unless otherwise stated.
To request higher quotas, submit a ticket.
The // symbol in formulas denotes integer division (floor division).
TCP connections per broker
| Quota | Subscription and pay-as-you-go | Serverless |
|---|---|---|
| Base | 1,000 | 2,000 |
| Increment | +1,000 per 100 MB/s actual traffic | +1,000 per 300 MB/s reserved production capacity |
| Maximum | 10,000 | 10,000 |
| Formula | C = min(10000, 1000 + (F // 100) * 1000) | C = min(10000, 2000 + (F // 300) * 1000) |
To request a higher connection limit, please submit a ticket.
Internet (SSL) connections per broker
| Quota | Subscription and pay-as-you-go | Serverless |
|---|---|---|
| Base | 200 | 200 |
| Increment | +100 per 100 MB/s actual traffic | +100 per 300 MB/s reserved production capacity |
| Maximum | 1,000 | 1,000 |
| Formula | C = min(1000, 200 + (F // 100) * 100) | C = min(1000, 200 + (F // 300) * 100) |
Connection attempts per second per broker
| Connection type | Subscription and pay-as-you-go | Serverless (Basic Edition) | Serverless (Standard Edition and Professional Edition) |
|---|---|---|---|
| TCP | 50 | 150 | 150 |
| Internet (SSL) | 10 | 10 | 10 |
These limits include failed attempts due to authentication errors.
Batch size
Fragmented sending occurs when the batch size TP50 falls below 4 KB. This refers to the message batch size in PRODUCE requests after client-side batching. Use client version 2.4 or later to improve batching efficiency.
For details, see Improve sending performance (reduce fragmented requests).
PRODUCE request rate (cluster)
| Quota | Subscription and pay-as-you-go | Serverless (Basic Edition) | Serverless (Standard Edition and Professional Edition) |
|---|---|---|---|
| Base | 10,000/s | 10,000/s | 10,000/s |
| Increment | +2,000 per 20 MB/s actual traffic | +5,000 per 300 MB/s reserved production capacity | +2,000 per 60 MB/s reserved production capacity |
| Formula | R = 10000 + (F // 20) * 2000 | R = 10000 + (F // 300) * 5000 | R = 10000 + (F // 60) * 2000 |
To request a higher limit, please submit a ticket.
FETCH request rate (cluster)
| Quota | Subscription and pay-as-you-go | Serverless (Basic Edition) | Serverless (Standard Edition and Professional Edition) |
|---|---|---|---|
| Base | 5,000/s | 5,000/s | 5,000/s |
| Increment | +1,000 per 20 MB/s actual consumption traffic | +2,500 per 100 MB/s reserved consumption capacity | +1,000 per 20 MB/s reserved consumption capacity |
| Formula | R = 5000 + (F // 20) * 1000 | R = 5000 + (F // 100) * 2500 | R = 5000 + (F // 20) * 1000 |
To request a higher limit, please submit a ticket.
OFFSET_COMMIT request rate per broker
| Quota | Subscription and pay-as-you-go | Serverless |
|---|---|---|
| Base | 100/s | 100/s |
| Increment | +100 per 100 MB/s actual traffic | +100 per 100 MB/s reserved production capacity |
| Maximum | 1,000/s | 1,000/s |
| Formula | R = min(1000, 100 + (F // 100) * 100) | R = min(1000, 100 + (F // 100) * 100) |
To request a higher limit, please submit a ticket.
Metadata request rate (cluster)
| Quota | Subscription and pay-as-you-go | Serverless |
|---|---|---|
| Base | 100/s | 100/s |
| Increment | +100 per 100 MB/s actual traffic | +100 per 100 MB/s reserved production capacity |
| Maximum | 1,000/s | 1,000/s |
| Formula | R = min(1000, 100 + (F // 100) * 100) | R = min(1000, 100 + (F // 100) * 100) |
Metadata requests include METADATA, INIT_PRODUCER_ID, CREATE_ACL, and JOIN_GROUP.
Excessive metadata requests can affect cluster stability.
Partitions
For maximum partitions per instance type, see Instance partitions. This count includes partitions from all topic types created by users.
To request a higher limit, please submit a ticket.
Partition creation and deletion rate: 900 partitions every 10 seconds (cluster-wide). This covers all operations through the console, OpenAPI, Kafka Admin, and other methods.
Consumer groups
Each cluster supports up to 2,000 consumer groups. Maintain a topic-to-consumer-group subscription ratio of 1:1. Do not exceed 3:1.
Too many consumer groups increase coordination load and metadata overhead, which degrades performance and extends fault recovery time.
To request a higher limit, please submit a ticket.
Message format version
The message format version must be greater than V1 for both produce and consume operations. Use client version 2.4 or later.
Older Kafka message formats increase server CPU usage, reduce throughput, and may cause compatibility and security issues.
Other considerations
The following scenarios can overload the server and affect stability. Plan your workload accordingly.
Compression algorithms: Enabling compression (such as GZIP) consumes more server resources, increasing latency and reducing throughput.
Transactional Producer ID initialization: Frequent initialization of transactional Producer IDs can cause memory overflow and server overload. The kernel parameter
transactional.id.expiration.msis set to 15 minutes. For special requirements, submit a ticket.Invalid message timestamps: When
message.timestamp.type=CreateTime, the broker rejects messages if the difference between the broker's receive timestamp and the message timestamp exceedsmessage.timestamp.difference.max.ms. This prevents incorrect timestamps from causing immediate log segment deletion (too small) or preventing deletion entirely (too large).Compacted topic storage: Each compacted topic partition has a default storage limit of 5 GB to prevent storage exhaustion from abnormal writes. For special requirements, submit a ticket.
CPU usage: If instance CPU usage exceeds 85%, the instance may become unstable. Symptoms include breakdowns and long-tail latency jitter in produce or consume operations.
Skewed message distribution: Kafka performance depends on cluster resources. Uneven message distribution or partition allocation prevents full utilization of cluster capacity.
Open-source transactional messaging: Known unresolved issues exist in Kafka's transactional messaging. Use with caution. For example, see KAFKA-12671 and related Kafka issues.
Duplicate messages during rebalancing: Kafka may deliver duplicate messages during consumer group rebalancing. Implement idempotency checks in your consumption logic to avoid business impact.