When you upgrade the traffic specification of a Message Queue for Apache Kafka instance, the corresponding cluster may be scaled out. After the cluster is scaled out, you must rebalance the topic traffic to distribute the traffic evenly across brokers in the scaled-out cluster. Otherwise, the original topic traffic is still distributed to the original brokers before the scale-out. The peak traffic of the original topics is limited by the traffic specification before the scale-out. The traffic of the added topics is not subject to the traffic specification before the scale-out.
Prerequisites
Precautions
When your Message Queue for Apache Kafka instance is in the Running (Pending Rebalancing) state, you can use this instance to send and subscribe to messages but cannot create resources such as topics and consumer groups in this instance. You must complete topic traffic rebalancing or choose not to rebalance topic traffic before you can create a resource.
Traffic rebalancing method
The following table describes the traffic rebalancing methods supported by Message Queue for Apache Kafka.
Traffic rebalancing method | Principle | Impact | Scenario | Duration |
---|---|---|---|---|
Add Partitions to All Topics | Add partitions to the new brokers after the cluster scale-out for all topics on the original brokers. |
|
|
Seconds. |
Migrate Partitions of All Topics (recommended) |
|
|
All cluster scale-out scenarios are supported. |
|
Do Not Rebalance (not recommended) | You do not need to perform any operations. The original topics are still distributed on the brokers of the cluster before the scale-out, and the new topics are evenly distributed across all brokers after the scale-out. |
|
|
Takes effect immediately. |
Procedure
- Log on to the Message Queue for Apache Kafka console.
- In the top navigation bar, select a region.
- In the left-side navigation pane, click Instances.
- On the Instance Details page, click an instance. In the Status section, click Rebalance Now.
- In the Rebalancing Method dialog box, select a rebalancing method. The following rebalancing methods are supported:
- Add Partitions to All Topics
Select Add Partitions to All Topics and click OK.
- Migrate Partitions of All Topics
- Submita ticket and ask Message Queue for Apache Kafka Customer Services to upgrade your broker to the latest version.
- Select Migrate Partitions of All Topics and click OK.
- Do Not Rebalance
Select Do Not Rebalance and click OK.
- Add Partitions to All Topics