FAQ

Multipart upload tasks initiated by OSS SDK for Python failed.

Try the following solutions if parts failed to upload by using OSS SDK for Python:

• First, determine whether the object is directly uploaded to OSS or uploaded to OSS through a proxy such as CDN. If the object is uploaded to OSS through CDN, the ETag of the object must be exposed and the following headers must be configured in the request: Access-Control-Allow-Origin, Access-Control-Allow-Methods, and Access-Control-Allow-Headers. For more information, see PutBucketcors.
• If the upload failed because of timeout, we recommend that you use resumable upload to upload the object. By using resumable upload, you can split the objects into parts of a custom size and then upload the parts concurrently. When exceptions occur, view and analyze the error message returned by OSS SDK for Python.
• Delete the parts generated by the failed upload task and upload the object again.

The upload and download speeds of ossutil are far better than those of OSS SDK for Python.

• Cause

  ossutil is developed based on OSS SDK for Go and provides better performance in concurrent upload scenarios. If the download speed of OSS SDK for Python is far slower than that of ossutil, it may be because that crcmod is not properly installed.

• Solution

  For more information about how to install crcmode, see Installation. If the problem persists, submit a ticket.

The multipart upload function provided by OSS SDK for Python can be called in CentOS systems. However, a 403 error is returned when the multipart upload function is called on Ubuntu systems.

Use tcpdump to capture packets on the client. Analyze the packets to determine whether the calculated signature is different from that on the OSS server because of incorrect header information in the request.

POST /ttsservice%2Fpasswd? uploadId=D468E486D1D94D90A1AB8885A4E32AE0 HTTP/1.1
Host: rokid.oss-cn-hangzhou.aliyuncs.com
Accept-Encoding: identity
Accept: text/html
Content-Length: 137
date: Sat, 29 Dec 2018 07:32:34 GMT
authorization: OSS LTAIknFr:r2KPR0y4E0G5tnU/MYdcvXHP****
Content-Type: application/x-www-form-urlencoded
User-Agent: aliyun-sdk-python/2.6.0(Linux/4.4.0-31-generic/x86_64;3.4.3)

<CompleteMultipartUpload><Part><PartNumber>1</PartNumber><ETag>"3195544E19D99658706D5****"</ETag></Part></CompleteMultipartUpload>HTTP/1.1 403 Forbidden

Server: AliyunOSS
Date: Sat, 29 Dec 2018 07:33:43 GMT
Content-Type: application/xml
Content-Length: 1122
Connection: keep-alive
x-oss-request-id: 5C2723573183****
x-oss-server-time: 0

<? xml version="1.0" encoding="UTF-8"? >
<Error>
 <Code>SignatureDoesNotMatch</Code>
 <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method. </Message>
 <RequestId> 5C2723573183A12D </RequestId>
 <HostId>rokid.oss-cn-hangzhou.aliyuncs.com</HostId>
 <OSSAccessKeyId> LTAXXX </OSSAccessKeyId>
 <SignatureProvided>r2KPR0y4E0G5tnU/MYdc****</SignatureProvided>
 <StringToSign>POST
application/x-www-form-urlencoded
Sat, 29 Dec 2018 07:32:34 GMT
/rokid/ttsservice/passwd? uploadId=D468E486D1D94D90A1AB8885A4E3****</StringToSign>
 <StringToSignBytes>50 4F 53 54 0A 0A 61 70 70 6C 69 63 61 74 69 6F 6E 2F 78 2D 77 77 77 2D 66 6F 72 6D 2D 75 72 6C 65 6E 63 6F 64 65 64 0A 53 61 74 2C 20 32 39 20 44 65 63 20 32 30 31 38 20 30 37 3A 33 32 3A 33 34 20 47 4D 54 0A 2F 72 6F 6B 69 64 2D 6F 70 73 2D 6D 6F 64 65 6C 2F 74 74 73 73 65 72 76 69 63 65 2F 70 61 73 73 77 64 3F 75 70 6C 6F 61 64 49 64 3D 44 34 36 38 45 34 38 36 44 31 44 39 34 44 39 30 41 31 41 42 38 38 38 35 41 34 45 33 32 41 45 30 </StringToSignBytes>
</Error>

If the signature received by the server and the signature calculated on the client are inconsistent, the content of the request is modified. We recommend that you upload the object by using HTTPS.

import base64
import hmac
import sha
mac = hmac.new("<Secretkey>","POST\n\napplication/x-www-form-urlencoded\nSat, 29 Dec 2018 07:32:34 GMT\n/rokid/ttsservice/passwd? uploadId=D468E486D1D94D90A1AB8885A4E3****", sha)
Signature = base64.b64encode(mac.digest())
print(Signature)

If the signatures are consistent, it indicates that the signature in the request generated by OSS SDK is correct. If the calculated signature is not consistent with the signature in the captured packets, it may be because that the MD5 hash calculated based on the signature is different in Ubuntu systems due to compatibility issues.

When OSS SDK for Python is used in macOS systems to start multiple threads and perform operations on OSS resources in a subthread, an error occurs if tensorflow is imported. If tensorflow is not imported, no errors occur. In addition, tensorflow can be imported without errors if you do not start multiple threads.

objc[2483]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called.
objc[2483]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

Solution

Add an environment variable OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES. The following figure shows how to add the environment variable.

For more information, visit Workarounds for compatibility.

Add retry policies.

OSS SDK for Python does not provide retry mechanisms. Requests may fail in poor network conditions. In this case, we recommend that you add retry policies to your code. The following code provides an example on how to add a retry policy:

def test_get_object():
    MAX_RETRIES = 3
    retry_count = 0
    while True:
        try:
            retry_count += 1
            # yourObjectName indicates the complete path of the object excluding the bucket name. Example: abc/efg/example.jpg.
            # yourFileName indicates the complete path to which you want to download the object. Example: /users/local/example.jpg.
            bucket.get_object_to_file("yourObjectName", "yourFileName")
            break
        except Exception:
            if retry_count >= MAX_RETRIES:
                raise