When partitions are unevenly distributed across topics in an ApsaraMQ for Kafka cluster, disk performance and capacity utilization degrade. Partition rebalancing redistributes partitions evenly. If a specific topic has high traffic that cannot wait for rebalancing, traffic redirection moves its load to a topic with low disk usage.
To fix partition skew, you can also increase partition counts to a multiple of the system-recommended value. Use partition rebalancing when partition counts for production workloads cannot be changed in a timely manner.
| Feature | When to use | What it does |
|---|---|---|
| Partition rebalancing | Partitions are unevenly distributed across cloud storage topics in a cluster | Redistributes partitions so that partition counts are as even as possible |
| Traffic redirection | A specific cloud storage topic has high traffic with uneven partition distribution | Redirects traffic from a high-traffic cloud storage topic to a topic with low disk usage |
Impacts and limitations
| Impact | Details |
|---|---|
| Concurrency | Only one topic can undergo traffic redirection at a time |
| Client reconnection | During traffic redirection, client metadata is updated and clients are reconnected. Make sure your clients support metadata updates and automatic reconnection before you proceed |
| Cloud storage topics | Traffic redirection applies only to cloud storage topics and completes within seconds |
Rebalance partitions
Partition rebalancing redistributes partitions of cloud storage topics across a cluster so that partition counts are as even as possible.
-
Log on to the ApsaraMQ for Kafka console. In the left-side navigation pane, click Instances.
-
In the top navigation bar, select the region where your instance resides.
-
On the Instances page, find the instance and choose in the Actions column.
-
Read the confirmation message and click OK.
After the operation completes, check the partition distribution across topics in your instance to confirm that partitions are distributed more evenly.
Redirect traffic
Traffic redirection moves traffic from a high-traffic cloud storage topic to a topic with low disk usage.
-
Log on to the ApsaraMQ for Kafka console. In the left-side navigation pane, click Instances.
-
In the top navigation bar, select the region where your instance resides.
-
On the Instances page, click the name of the instance.
-
In the left-side navigation pane, click Topics. On the Topics page, find the topic and choose in the Actions column.
-
Read the confirmation message and click OK.