All Products
Search
Document Center

Object Storage Service:FAQ

Last Updated:Dec 21, 2023

This topic provides answers to some commonly asked questions about Object Storage Service (OSS) SDK for Python.

The multipart upload task initiated by using OSS SDK for Python failed.

Solutions:

  • Determine whether the object is directly uploaded to OSS or uploaded to OSS by using a proxy, such as CDN. If the object is uploaded to OSS by using CDN, expose the ETag of the object and configure cross-origin headers, such as Access-Control-Allow-Origin, Access-Control-Allow-Methods, and Access-Control-Allow-Headers. For more information, see PutBucketcors.

  • If the upload task failed due to the network timeout, we recommend that you use resumable upload to upload the object. This way, you can split the object into parts of a custom size and then concurrently upload the parts. If exceptions occur, view and analyze the error message returned by OSS SDK for Python.

  • Delete the parts generated by the failed upload task and reupload the object.

If the issue persists, submit a ticket that includes the following information:

  • The request ID in the error message returned by OSS SDK for Python.

  • The packets captured by running the following tcpdump command when you reupload the object.

    tcpdump -i <Port name> -s0 host <OSS endpoint> -w faild.pcap

The upload and download speeds of ossutil are faster than those of OSS SDK for Python.

  • Cause

    ossutil is developed based on OSS SDK for Go and provides better performance in concurrent upload scenarios. If the download speed of OSS SDK for Python is slower than that of ossutil, it may be caused by incorrect installation of crcmod.

  • Solution

    Correctly install crcmod. For more information, see Installation.

You can call the multipart upload function of OSS SDK for Python on the CentOS system. If you call the multipart upload function on the Ubuntu system, HTTP status code 403 is returned.

Use tcpdump to capture packets on the client. Analyze the packets to determine whether the calculated signature is different from that on the OSS server due to incorrect header information in the request.

POST /ttsservice%2Fpasswd?uploadId=D468E486D1D94D90A1AB8885A4E32AE0 HTTP/1.1
Host: rokid.oss-cn-hangzhou.aliyuncs.com
Accept-Encoding: identity
Accept: text/html
Content-Length: 137
date: Sat, 29 Dec 2018 07:32:34 GMT
authorization: OSS LTAIknFr:r2KPR0y4E0G5tnU/MYdcvXHP****
Content-Type: application/x-www-form-urlencoded
User-Agent: aliyun-sdk-python/2.6.0(Linux/4.4.0-31-generic/x86_64;3.4.3)

<CompleteMultipartUpload><Part><PartNumber>1</PartNumber><ETag>"3195544E19D99658706D5****"</ETag></Part></CompleteMultipartUpload>HTTP/1.1 403 Forbidden

Server: AliyunOSS
Date: Sat, 29 Dec 2018 07:33:43 GMT
Content-Type: application/xml
Content-Length: 1122
Connection: keep-alive
x-oss-request-id: 5C2723573183****
x-oss-server-time: 0

<?xml version="1.0" encoding="UTF-8"?>
<Error>
 <Code>SignatureDoesNotMatch</Code>
 <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
 <RequestId> 5C2723573183A12D </RequestId>
 <HostId>rokid.oss-cn-hangzhou.aliyuncs.com</HostId>
 <OSSAccessKeyId> LTAXXX </OSSAccessKeyId>
 <SignatureProvided>r2KPR0y4E0G5tnU/MYdc****</SignatureProvided>
 <StringToSign>POST
application/x-www-form-urlencoded
Sat, 29 Dec 2018 07:32:34 GMT
/rokid/ttsservice/passwd?uploadId=D468E486D1D94D90A1AB8885A4E3****</StringToSign>
 <StringToSignBytes>50 4F 53 54 0A 0A 61 70 70 6C 69 63 61 74 69 6F 6E 2F 78 2D 77 77 77 2D 66 6F 72 6D 2D 75 72 6C 65 6E 63 6F 64 65 64 0A 53 61 74 2C 20 32 39 20 44 65 63 20 32 30 31 38 20 30 37 3A 33 32 3A 33 34 20 47 4D 54 0A 2F 72 6F 6B 69 64 2D 6F 70 73 2D 6D 6F 64 65 6C 2F 74 74 73 73 65 72 76 69 63 65 2F 70 61 73 73 77 64 3F 75 70 6C 6F 61 64 49 64 3D 44 34 36 38 45 34 38 36 44 31 44 39 34 44 39 30 41 31 41 42 38 38 38 35 41 34 45 33 32 41 45 30 </StringToSignBytes>
</Error>

If the signature received by the server is different from that calculated on the client, the content of the request is modified. We recommend that you upload the object by using HTTPS.

The preceding code shows the packets captured by tcpdump on the client. Use the captured header information in the following script to calculate the signature and check whether the calculated signature is consistent with the signature in the captured packets.

import base64
import hmac
import sha
mac = hmac.new("<Secretkey>","POST\n\napplication/x-www-form-urlencoded\nSat, 29 Dec 2018 07:32:34 GMT\n/rokid/ttsservice/passwd?uploadId=D468E486D1D94D90A1AB8885A4E3****", sha)
Signature = base64.b64encode(mac.digest())
print(Signature)

If the signatures are consistent, the signature in the request generated by OSS SDK is correct. If the calculated signature is not consistent with the signature in the captured packets, the reason may be that the MD5 hash calculated based on the signature is different on the Ubuntu system due to compatibility issues.

When OSS SDK for Python is used on the macOS system to start multiple threads and perform operations on OSS resources in a subthread, an error occurs if TensorFlow is imported. If TensorFlow is not imported, no errors occur. In addition, TensorFlow can be imported without errors if you do not start multiple threads.

When you use OSS SDK for Python to start multiple threads, the following error occurs:

objc[2483]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called.
objc[2483]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

Solution:

Add the environment variable OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES. The following figure shows how to add the environment variable.fig_python_107749_02

For more information, visit Workarounds for compatibility.

How do I add retry policies?

OSS SDK for Python does not provide retry mechanisms. Requests may fail in poor network conditions. In this case, we recommend that you add retry policies to your code. The following sample code provides an example on how to add a retry policy:

def test_get_object():
    MAX_RETRIES = 3
    retry_count = 0
    while True:
        try:
            retry_count += 1
            # Specify the full path of the object. Do not include the bucket name in the full path. Example: abc/efg/example.jpg. 
            # Specify the full path in which you want to store the downloaded object. Example: /users/local/example.jpg. 
            bucket.get_object_to_file("yourObjectName", "yourFileName")
            break
        except Exception:
            if retry_count >= MAX_RETRIES:
                raise

What do I do if the crcmod extensions in C cannot be installed on some operating systems?

For systems on which the crcmod extensions in C cannot be installed, you can disable CRC-64. For more information about how to disable CRC-64, see Initialization.

How do I ensure that the generated signed URL starts with https?

If you want the generated signed URL to start with https, configure the endpoint to start with https when you initialize OSS SDK for Python, such as https://oss-cn-hangzhou.aliyuncs.com.

How do I use a bucket when multiple processes are running at the same time?

Create a bucket after the processes are initialized.