Adjust the bandwidth of buckets and users - Object Storage Service

After you add multiple buckets within your Alibaba Cloud account to the same resource pool, you can configure bandwidth throttling for buckets and requesters who access the buckets in a fine-grained manner based on your business requirements.

Prerequisites

Resource pool QoS is in invitational preview. If the throughput of your OSS buckets in a region has reached or exceeded 500 Gbps, you can contact technical support to apply for this feature.
You have installed ossutil 2.0.

Configuration examples

Example	Description
Specify the maximum bandwidth that multiple RAM users can use to access a resource pool	When multiple requesters access different buckets in the same resource pool at the same time as RAM users, you can specify the maximum bandwidth of different RAM users to prevent a RAM user from overusing the bandwidth of the resource pool.
Specify the maximum bandwidth that a RAM user can use to access buckets	When different applications access the same bucket in a resource pool, you can specify the maximum bandwidth that a specific RAM user can use to access the bucket. This prevents the RAM user from overusing the bandwidth of the bucket.
Specify the total maximum bandwidth of a bucket	When different applications access different buckets in a resource pool, you can specify the maximum bandwidth of a bucket to prevent traffic spikes on a single bucket from preempting the bandwidth of other buckets.
Specify the maximum bandwidth over the Internet of a bucket	When you access, preview, or download objects in a bucket in a resource pool over the Internet, outbound traffic is generated. You can specify the maximum download bandwidth over the Internet of the bucket to prevent high traffic fees caused by high concurrent access.
Specify the maximum bandwidth that multiple Alibaba Cloud services can use to access a bucket	In most cases, applications or services use RAM roles to access OSS across services. This way, you can specify the maximum bandwidth that a RAM role can use to access a specific bucket in a resource pool. This prevents the RAM role from consuming excessive bandwidth and affecting other services.
Specify the total maximum bandwidth of a bucket group (containing multiple low-priority buckets)	In the same resource pool, you can add many low-priority buckets to a bucket group and specify the maximum bandwidth of the bucket group. This ensures that the remaining high-priority buckets can obtain sufficient bandwidth for internal or external data communication, which guarantees the service quality and response speed of the high-priority buckets.
Specify the maximum bandwidth over the Internet of a bucket group (containing multiple low-priority buckets)	In the same resource pool, you can add many low-priority buckets to a bucket group and specify the maximum bandwidth of the bucket group. This ensures that the remaining high-priority buckets can obtain sufficient bandwidth over the Internet even during peak network loads.

Specify the maximum bandwidth that multiple RAM users can use to access a resource pool

The following examples describe how to specify the maximum bandwidth that two RAM users can use to access a resource pool:

Specify the maximum bandwidth that RAM user A can use to access the resource pool.
1. Use the local qos.xml configuration file to specify the maximum bandwidth that RAM user A can use to access the resource pool.
  - Set the total maximum upload bandwidth to 100 Gbit/s. The maximum upload bandwidth over the internal network is 50 Gbit/s, and the maximum upload bandwidth over the Internet is 50 Gbit/s.
  - Set the total maximum download bandwidth to 200 Gbit/s. The maximum download bandwidth over the internal network is 150 Gbit/s, and the maximum download bandwidth over the Internet is 50 Gbit/s.
  - Retain the default value -1 for the total queries per second (QPS), QPS for the internal network, and QPS for the Internet, which specifies that no limits are imposed on these items.
```
<QoSConfiguration>
  <TotalUploadBandwidth>100</TotalUploadBandwidth>
  <IntranetUploadBandwidth>50</IntranetUploadBandwidth>
  <ExtranetUploadBandwidth>50</ExtranetUploadBandwidth>
  <TotalDownloadBandwidth>200</TotalDownloadBandwidth>
  <IntranetDownloadBandwidth>150</IntranetDownloadBandwidth>
  <ExtranetDownloadBandwidth>50</ExtranetDownloadBandwidth>
  <TotalQps>-1</TotalQps>
  <IntranetQps>-1</IntranetQps>
  <ExtranetQps>-1</ExtranetQps>
</QoSConfiguration>
```
2. Add the preceding bandwidth configurations to RAM user A whose UID is 266xxxx.
```
ossutil api invoke-operation --op-name put-resource-pool-requester-qos-info --method PUT --parameters resourcePool=examplepool --parameters qosRequester=266xxxx --parameters requesterQosInfo --body=file://qos.xml
```
Specify the maximum bandwidth that RAM user B can use to access the resource pool.
1. Use the local qos.xml configuration file to specify the maximum bandwidth that RAM user B can use to access the resource pool.
  - Set the total maximum upload bandwidth to 50 Gbit/s. The maximum upload bandwidth over the internal network is 30 Gbit/s, and the maximum upload bandwidth over the Internet is 20 Gbit/s.
  - Set the total maximum download bandwidth to 50 Gbit/s. The maximum download bandwidth over the internal network is 30 Gbit/s, and the maximum download bandwidth over the Internet is 20 Gbit/s.
  - Retain the default value -1 for the total QPS, QPS for the internal network, and QPS for the Internet, which specifies that no limits are imposed on these items.
```
<QoSConfiguration>
  <TotalUploadBandwidth>50</TotalUploadBandwidth>
  <IntranetUploadBandwidth>30</IntranetUploadBandwidth>
  <ExtranetUploadBandwidth>20</ExtranetUploadBandwidth>
  <TotalDownloadBandwidth>50</TotalDownloadBandwidth>
  <IntranetDownloadBandwidth>30</IntranetDownloadBandwidth>
  <ExtranetDownloadBandwidth>20</ExtranetDownloadBandwidth>
  <TotalQps>-1</TotalQps>
  <IntranetQps>-1</IntranetQps>
  <ExtranetQps>-1</ExtranetQps>
</QoSConfiguration>
```
2. Add the preceding bandwidth configurations to RAM user B whose UID is 242xxxx.
```
ossutil api invoke-operation --op-name put-resource-pool-requester-qos-info --method PUT --parameters resourcePool=examplepool --parameters qosRequester=242xxxx --parameters requesterQosInfo --body=file://qos.xml
```

(Optional) List the bandwidth configurations of all RAM users in the resource pool.

ossutil api invoke-operation --op-name list-resource-pool-requester-qos-infos --method GET --parameters resourcePool=examplePool  --parameters requesterQosInfo

For more information, see View the information of a RAM user.

Specify the maximum bandwidth that a RAM user can use to access buckets

The following example describes how to specify the maximum bandwidth that a RAM user can use to access a bucket in a resource pool:

Use the local qos.xml configuration file to specify the maximum bandwidth that a specific RAM user can use to access a bucket in a resource pool.
- Set the total maximum upload bandwidth to 100 Gbit/s. No limits are imposed on the upload bandwidth over the internal network and the upload bandwidth over the Internet.
- Set the total maximum download bandwidth to 100 Gbit/s. No limits are imposed on the download bandwidth over the internal network and the download bandwidth over the Internet.
- Retain the default value -1 for the total QPS, QPS for the internal network, and QPS for the Internet, which specifies that no limits are imposed on these items.
```
<QoSConfiguration>
  <TotalUploadBandwidth>100</TotalUploadBandwidth>
  <IntranetUploadBandwidth>-1</IntranetUploadBandwidth>
  <ExtranetUploadBandwidth>-1</ExtranetUploadBandwidth>
  <TotalDownloadBandwidth>100</TotalDownloadBandwidth>
  <IntranetDownloadBandwidth>-1</IntranetDownloadBandwidth>
  <ExtranetDownloadBandwidth>-1</ExtranetDownloadBandwidth>
  <TotalQps>-1</TotalQps>
  <IntranetQps>-1</IntranetQps>
  <ExtranetQps>-1</ExtranetQps>
</QoSConfiguration>
```

Add the preceding bandwidth configurations to a RAM user whose UID is 266xxxx for access to a bucket named examplebucket.

ossutil api invoke-operation --op-name put-bucket-requester-qos-info --method PUT --bucket=examplebucket --parameters requesterQosInfo --parameters qosRequester=266xxxx --body file://qos.xml

(Optional) Query the bandwidth configurations of the RAM user for access to a bucket in the resource pool.

ossutil api invoke-operation --op-name get-bucket-requester-qos-info --method GET --bucket=examplebucket --parameters requesterQosInfo --parameters qosRequester=266xxxx

Specify the total maximum bandwidth of a bucket

The following example describes how to specify the total maximum bandwidth of a bucket in a resource pool:

Use the local qos.xml configuration file to specify the total maximum bandwidth of a bucket in a resource pool.
- Set the total maximum upload bandwidth to 100 Gbit/s. No limits are imposed on the upload bandwidth over the internal network and the upload bandwidth over the Internet.
- Set the total maximum download bandwidth to 200 Gbit/s. No limits are imposed on the download bandwidth over the internal network and the download bandwidth over the Internet.
```
<QoSConfiguration>
  <TotalUploadBandwidth>100</TotalUploadBandwidth>
  <IntranetUploadBandwidth>-1</IntranetUploadBandwidth>
  <ExtranetUploadBandwidth>-1</ExtranetUploadBandwidth>
  <TotalDownloadBandwidth>200</TotalDownloadBandwidth>
  <IntranetDownloadBandwidth>-1</IntranetDownloadBandwidth>
  <ExtranetDownloadBandwidth>-1</ExtranetDownloadBandwidth>
</QoSConfiguration>
```

Add the preceding bandwidth configurations to a bucket named examplebucket.

ossutil api invoke-operation --op-name put-bucket-qos-info --method PUT --bucket examplebucket --parameters qosInfo --body=file://qos.xml

(Optional) Query the total bandwidth configurations of the bucket.

ossutil api invoke-operation --op-name get-bucket-qos-info --method GET --bucket examplebucket --parameters qosInfo

Specify the maximum bandwidth over the Internet of a bucket

The following example describes how to specify the maximum bandwidth over the Internet of a bucket in a resource pool:

Use the local qos.xml configuration file to specify the maximum bandwidth over the Internet of a bucket.
- Set the total maximum upload bandwidth to 100 Gbit/s. No limits are imposed on the upload bandwidth over the internal network, and the maximum upload bandwidth over the Internet is 20 Gbit/s
- Set the total maximum download bandwidth to 100 Gbit/s. No limits are imposed on the download bandwidth over the internal network, and the maximum download bandwidth over the Internet is 20 Gbit/s.
```
<QoSConfiguration>
  <TotalUploadBandwidth>100</TotalUploadBandwidth>
  <IntranetUploadBandwidth>-1</IntranetUploadBandwidth>
  <ExtranetUploadBandwidth>20</ExtranetUploadBandwidth>
  <TotalDownloadBandwidth>100</TotalDownloadBandwidth>
  <IntranetDownloadBandwidth>-1</IntranetDownloadBandwidth>
  <ExtranetDownloadBandwidth>20</ExtranetDownloadBandwidth>
</QoSConfiguration>
```

Add the preceding bandwidth configurations to a bucket named examplebucket.

ossutil api invoke-operation --op-name put-bucket-qos-info --method PUT --bucket examplebucket --parameters qosInfo --body=file://qos.xml

(Optional) Query the bandwidth configurations over the Internet of the bucket.

ossutil api invoke-operation --op-name get-bucket-qos-info --method GET --bucket examplebucket --parameters qosInfo

Specify the maximum bandwidth that multiple Alibaba Cloud services can use to access a bucket

The following examples describe how to specify the maximum bandwidth that Alibaba Cloud CDN and Cloud Parallel File Storage (CPFS) can use to access a bucket in a resource pool:

Configure the maximum bandwidth of a RAM role when you use the RAM role to retrieve objects from a private bucket by using Alibaba Cloud CDN
You use a bucket in a resource pool to store static resources and use CDN to accelerate the distribution of static resources. To ensure that CDN can correctly obtain the required static files from your private bucket, you need to enable the back-to-origin feature for private buckets. After this feature is enabled, CDN accesses resources in the private bucket through the AliyunCDNAccessingPrivateOSSRole role.
1. Use the local qos.xml configuration file to specify the maximum bandwidth that the RAM role can use to access objects in the bucket.
  - Set the total maximum upload bandwidth to 20 Gbit/s. No limits are imposed on the upload bandwidth over the internal network and the upload bandwidth over the Internet.
  - Set the total maximum download bandwidth to 40 Gbit/s. No limits are imposed on the download bandwidth over the internal network and the download bandwidth over the Internet.
  - Retain the default value -1 for the total QPS, QPS for the internal network, and QPS for the Internet, which specifies that no limits are imposed on these items.
```
<QoSConfiguration>
  <TotalUploadBandwidth>20</TotalUploadBandwidth>
  <IntranetUploadBandwidth>-1</IntranetUploadBandwidth>
  <ExtranetUploadBandwidth>-1</ExtranetUploadBandwidth>
  <TotalDownloadBandwidth>40</TotalDownloadBandwidth>
  <IntranetDownloadBandwidth>-1</IntranetDownloadBandwidth>
  <ExtranetDownloadBandwidth>-1</ExtranetDownloadBandwidth>
  <TotalQps>-1</TotalQps>
  <IntranetQps>-1</IntranetQps>
  <ExtranetQps>-1</ExtranetQps>
</QoSConfiguration>
```
2. Add the preceding bandwidth configurations to a RAM role whose ID 362xxxx for access to objects in a bucket named examplebucket.
```
ossutil api invoke-operation --op-name put-bucket-requester-qos-info --method PUT --bucket=examplebucket --parameters requesterQosInfo --parameters qosRequester=362xxxx --body file://qos.xml
```
Configure the maximum bandwidth of a RAM role when you use the RAM role to access objects in a bucket by using CPFS
In high-performance computing and AI training scenarios, you need to frequently read and write large amounts of training data from OSS Buckets. To improve data read and write efficiency, you can create data flow tasks through CPFS to quickly load data from OSS to CPFS. During the data loading process, CPFS queries and reads/writes data in the specified OSS bucket through the AliyunServiceRoleForNasOssDataFlow role.
1. Use the local qos.xml configuration file to specify the maximum bandwidth that the RAM role can use to access objects in the bucket.
  - Set the total maximum upload bandwidth to 30 Gbit/s. No limits are imposed on the upload bandwidth over the internal network and the upload bandwidth over the Internet.
  - Set the total maximum download bandwidth to 50 Gbit/s. No limits are imposed on the download bandwidth over the internal network and the download bandwidth over the Internet.
  - Retain the default value -1 for the total QPS, QPS for the internal network, and QPS for the Internet, which specifies that no limits are imposed on these items.
```
<QoSConfiguration>
  <TotalUploadBandwidth>30</TotalUploadBandwidth>
  <IntranetUploadBandwidth>-1</IntranetUploadBandwidth>
  <ExtranetUploadBandwidth>-1</ExtranetUploadBandwidth>
  <TotalDownloadBandwidth>50</TotalDownloadBandwidth>
  <IntranetDownloadBandwidth>-1</IntranetDownloadBandwidth>
  <ExtranetDownloadBandwidth>-1</ExtranetDownloadBandwidth>
  <TotalQps>-1</TotalQps>
  <IntranetQps>-1</IntranetQps>
  <ExtranetQps>-1</ExtranetQps>
</QoSConfiguration>
```
2. Add the preceding bandwidth configurations to a RAM role whose ID 300xxxx for access to objects in a bucket named examplebucket.
```
ossutil api invoke-operation --op-name put-bucket-requester-qos-info --method PUT --bucket=examplebucket --parameters requesterQosInfo --parameters qosRequester=300xxxx --body file://qos.xml
```

(Optional) List the bandwidth configurations of all RAM roles in the resource pool.

ossutil api invoke-operation --op-name list-resource-pool-requester-qos-infos --method GET --parameters resourcePool=examplePool  --parameters requesterQosInfo

For more information, see View RAM roles.

Specify the total maximum bandwidth of a bucket group (containing multiple low-priority buckets)

Assume that the same resource pool contains three buckets: online, offline-01, and offline-02. The online bucket is used for online business and requires bandwidth priority. The offline-01 and offline-02 buckets are used for two offline tasks. To ensure that the offline tasks do not affect the online business, you can add offline-01 and offline-02 to the same bucket group and specify the maximum bandwidth of the bucket group to control the bandwidth of the offline tasks. This prevents the offline tasks from affecting the normal operations of the online business.

Use the local qos.xml configuration file to specify the maximum bandwidth of a bucket group in a resource pool.
- Set the total maximum upload bandwidth to 50 Gbit/s. No limits are imposed on the upload bandwidth over the internal network and the upload bandwidth over the Internet.
- Set the total maximum download bandwidth to 100 Gbit/s. No limits are imposed on the download bandwidth over the internal network and the download bandwidth over the Internet.
```
<QoSConfiguration>
  <TotalUploadBandwidth>50</TotalUploadBandwidth>
  <IntranetUploadBandwidth>-1</IntranetUploadBandwidth>
  <ExtranetUploadBandwidth>-1</ExtranetUploadBandwidth>
  <TotalDownloadBandwidth>100</TotalDownloadBandwidth>
  <IntranetDownloadBandwidth>-1</IntranetDownloadBandwidth>
  <ExtranetDownloadBandwidth>-1</ExtranetDownloadBandwidth>
</QoSConfiguration>
```

Add offline-01 and offline-02 to a bucket group.

Add offline-01 to the bucket group.

ossutil api invoke-operation --op-name put-bucket-resource-pool-bucket-group --method PUT --bucket offline-01 --parameters resourcePool=pool-for-ai --parameters resourcePoolBucketGroup=offline-group

Add offline-02 to the bucket group.

ossutil api invoke-operation --op-name put-bucket-resource-pool-bucket-group --method PUT --bucket offline-02 --parameters resourcePool=pool-for-ai --parameters resourcePoolBucketGroup=offline-group

(Optional) Query the list of buckets in the bucket group.

ossutil api invoke-operation --op-name list-resource-pool-bucket-groups --method GET --parameters resourcePool=pool-for-ai --parameters resourcePoolBucketGroup

Configure the bandwidth threshold for the bucket group.

ossutil api invoke-operation --op-name put-resource-pool-bucket-group-qos-info --method PUT --parameters resourcePoolBucketGroupQosInfo --parameters resourcePool pool-for-ai --parameters resourcePoolBucketGroup offline-group --body=file://qos.xml

(Optional) Query the bandwidth configurations of the bucket group.

ossutil api invoke-operation --op-name get-resource-pool-bucket-group-qos-info --method GET --parameters resourcePool=pool-for-ai --parameters resourcePoolBucketGroup=offline-group --parameters resourcePoolBucketGroupQoSInfo

Specify the maximum bandwidth over the Internet of a bucket group (containing multiple low-priority buckets)

Assume that the same resource pool contains three buckets: realtime-chat, scheduled-posts, and archived-comments. The realtime-chat bucket is used for online business that requires high real-time performance, and scheduled-posts and archived-comments have lower business priorities. To prevent the low-priority buckets from affecting the bandwidth usage over the Internet of the high-priority business bucket, you can add scheduled-posts and archived-comments to the same bucket group and specify the maximum bandwidth over the Internet of the bucket group. This ensures that the bandwidth requirements over the Internet of realtime-chat are prioritized to guarantee the smooth operations of the online business.

Use the local qos.xml configuration file to specify the maximum bandwidth of a bucket group in a resource pool.
- Set the total maximum upload bandwidth to 50 Gbit/s. The maximum upload bandwidth over the Internet is 20 Gbit/s, and no limits are imposed on the upload bandwidth over the internal network.
- Set the total maximum download bandwidth to 100 Gbit/s. The maximum download bandwidth over the Internet is 20 Gbit/s, and no limits are imposed on the download bandwidth over the internal network.
```
<QoSConfiguration>
  <TotalUploadBandwidth>50</TotalUploadBandwidth>
  <IntranetUploadBandwidth>-1</IntranetUploadBandwidth>
  <ExtranetUploadBandwidth>20</ExtranetUploadBandwidth>
  <TotalDownloadBandwidth>100</TotalDownloadBandwidth>
  <IntranetDownloadBandwidth>-1</IntranetDownloadBandwidth>
  <ExtranetDownloadBandwidth>20</ExtranetDownloadBandwidth>
</QoSConfiguration>
```

Add scheduled-posts and archived-comments to a bucket group.

Add scheduled-posts to the bucket group.

ossutil api invoke-operation --op-name put-bucket-resource-pool-bucket-group --method PUT --bucket scheduled-posts --parameters resourcePool=pool-for-ai --parameters resourcePoolBucketGroup=test-group

Add archived-comments to the bucket group.

ossutil api invoke-operation --op-name put-bucket-resource-pool-bucket-group --method PUT --bucket archived-comments --parameters resourcePool=pool-for-ai --parameters resourcePoolBucketGroup=test-group

(Optional) Query the list of buckets in the bucket group.

ossutil api invoke-operation --op-name list-resource-pool-bucket-groups --method GET --parameters resourcePool=pool-for-ai --parameters resourcePoolBucketGroup

Configure the bandwidth threshold for the bucket group.

ossutil api invoke-operation --op-name put-resource-pool-bucket-group-qos-info --method PUT --parameters resourcePoolBucketGroupQosInfo --parameters resourcePool pool-for-ai --parameters resourcePoolBucketGroup test-group --body=file://qos.xml

(Optional) Query the bandwidth configurations of the bucket group.

ossutil api invoke-operation --op-name get-resource-pool-bucket-group-qos-info --method GET --parameters resourcePool=pool-for-ai --parameters resourcePoolBucketGroup=test-group --parameters resourcePoolBucketGroupQoSInfo

References

After you configure the bandwidth thresholds for resource pools, requesters, and buckets in resource pools by using the preceding examples, you can use Cloud Monitor to view the bandwidth thresholds and bandwidth usage of resource pools in different dimensions. For more information, see Use monitoring service.