Object Storage Service:Troubleshoot slow uploads and downloads in OSS

Non-SDK

1. Check how objects are uploaded or downloaded.

   • If you experience slow performance with tools such as the OSS console, ossfs, ossbrowser, or ossftp, use ossutil, which is optimized for high-throughput uploads of large or numerous files. For more information, see cp (upload files) and cp (Download objects).

   • If ossutil does not improve performance, proceed to the next step.

2. Check whether your bandwidth usage exceeds the limit.

   1. Log on to the OSS console.

   2. In the left-side navigation pane, click Buckets, then click the name of the target bucket.

   3. In the left-side navigation pane, choose Data Usage > Basic Data to check your bandwidth usage. For more information about the bandwidth limits for each region, see Bandwidth. Further requests are throttled when the bandwidth threshold is reached.

3. Check your local network environment. Use MTR to test the bucket domain name and check for packet loss on the link between your client and OSS. For more information, see Use the MTR tool.

   Note

   If you upload files over an internal network, use an internal endpoint.

4. Use the ossutil probe command to test your network environment and upload speed, and to diagnose the network connection between your client and OSS. For more information, see probe.

5. Check whether the data transfer involves a long distance between the client and the OSS region.

   For example, if your bucket is in the China (Hangzhou) region and your client is in the China (Hong Kong) region, high network latency can limit upload performance. In this case, enable Transfer Acceleration.

   Transfer Acceleration uses a global network of data centers. It intelligently routes user traffic to the nearest access point, using optimized networks and protocols to accelerate internet-based uploads and downloads. For more information, see Access OSS using transfer acceleration.

   Note

   After enabling Transfer Acceleration, the change takes effect globally within 30 minutes. Use the valid Transfer Acceleration endpoint.

   • Global acceleration endpoint: oss-accelerate.aliyuncs.com. Access points for Transfer Acceleration are globally distributed. You can use this endpoint to accelerate data transfer for buckets in any region.

   • Endpoint for acceleration outside the Chinese mainland: oss-accelerate-overseas.aliyuncs.com. Acceleration endpoints are distributed in regions outside the Chinese mainland. This endpoint is used only when you map a domain name without an Internet Content Provider (ICP) filing to a bucket in the China (Hong Kong) region or in another region outside the Chinese mainland by using a CNAME record.

SDK

The following section provides an example on how to troubleshoot the slow speed when you use OSS SDK for Python to upload an object:

1. Check the method used to upload the object. If you upload an object larger than 100 MB, use resumable upload. For more information, see Multipart upload. When you use resumable upload, check the configurations of the following parameters:

   • multipart_threshold: If the object size is greater than the value of this parameter, multipart upload is used to upload the object. Unit: bytes.

   • part_size: the size of the part. Unit: bytes. You must specify the part size based on the object size. For example, if an object is greater than 1 GB, we recommend that you set the part_size parameter to 100 MB.

     Note

     Increase the size of each part when the network connection is stable and decrease the size of each part when the network connection is unstable.

   • num_threads: the number of threads for concurrent upload. In most cases, the num_threads parameter is set to 4.

   Recommended configurations:

   multipart_threshold=100*1024*1024,
   part_size=100*1024*1024,
   num_threads=4

2. Check whether CRC-64 is enabled.

   If CRC-64 is enabled, disable CRC-64 when you initialize the OSSClient instance. You can replace CRC-64 with MD5 by adding the Content-MD5 request header to the request. For more information about how to initialize OSS SDK for Python, see Initialization.

   # -*- coding: utf-8 -*-
   import oss2
   from oss2.credentials import EnvironmentVariableCredentialsProvider
   
   # Obtain access credentials from environment variables. Before you run the sample code, make sure that the environment variables are configured. 
   auth = oss2.ProviderAuth(EnvironmentVariableCredentialsProvider())
   
   # Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
   endpoint = 'yourEndpoint'
   # Specify the name of the bucket. Example: examplebucket. 
   # enable_crc=False specifies that CRC-64 is disabled. 
   bucket = oss2.Bucket(auth, endpoint, 'examplebucket',enable_crc=False)  

   Note

   CRC-64 is used only to verify data and does not ensure data security.