Object Storage Service (OSS) SDK for Java uses MD5 verification and CRC-64 to ensure data integrity when you upload, download, and copy objects.

MD5 verification

If you configure Content-MD5 in an object upload request, OSS calculates the MD5 hash of the uploaded object. If the calculated MD5 hash is different from the MD5 hash configured in the upload request, InvalidDigest is returned. This allows OSS to ensure data integrity for object upload. If InvalidDigest is returned, you must upload the object again.

The following code provides an example on how to perform MD5 verification for object upload operations:

# -*- coding: utf-8 -*-
import oss2
# The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in Object Storage Service (OSS) is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
auth = oss2.Auth('yourAccessKeyId', 'yourAccessKeySecret')
# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
# Specify the bucket name. Example: examplebucket. 
bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'examplebucket')
# Specify the full path of the object. The full path of the object cannot contain the bucket name. 
object_name = 'yourObjectName'
content = 'yourContent'

# Calculate the MD5 hash of the uploaded content. 
content_md5 = oss2.utils.content_md5(content);
print('content_md5', content_md5)

# If the upload request contains the Content-MD5 header, the OSS server verifies the MD5 hash of the uploaded content. 
headers = dict()
headers['Content-MD5'] = content_md5
bucket.put_object(object_name, content, headers=headers)
Note MD5 verification can be used for the put_object, append_object, post_object, and upload_part operations.

CRC-64

Take note of the following items when you perform CRC-64 to verify data:

Note
  • CRC-64 can be used for the put_object, get_object, append_object, and upload_part operations. By default, CRC-64 is enabled when you upload an object. If the CRC-64 value calculated on the client is different from the CRC-64 value returned by the OSS server, InconsistentError is returned.
  • CRC-64 is not supported in range download.
  • CRC-64 consumes CPU resources and slows down uploads and downloads.
  • CRC-64 in object download

    The following code provides an example on how to perform CRC-64 for object download operations:

    # -*- coding: utf-8 -*-
    import oss2
    # The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
    auth = oss2.Auth('yourAccessKeyId', 'yourAccessKeySecret')
    # Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
    # Specify the bucket name. Example: examplebucket. 
    bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'examplebucket')
    # Specify the full path of the object. The full path of the object cannot contain the bucket name. 
    object_name = 'yourObjectName'
    
    # Check whether CRC-64 is enabled by default. 
    print('bucket.enable-crc:',  bucket.enable_crc)
    
    # The value returned by bucket.get_object is a file-like object that can be iterated. 
    object_stream = bucket.get_object(object_name)
    print(object_stream.read())
    
    # You must call read() to read the object from the stream that is returned by get_object before you can calculate the CRC-64 checksum of the object. 
    if object_stream.client_crc != object_stream.server_crc:
        print "The CRC checksum between client and server is inconsistent!"
  • CRC-64 in append upload

    If you specify init_crc in append upload, CRC-64 is enabled by default.

    # -*- coding: utf-8 -*-
    import oss2
    # The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
    auth = oss2.Auth('yourAccessKeyId', 'yourAccessKeySecret')
    # Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
    # Specify the bucket name. Example: examplebucket. 
    bucket = oss2.Bucket(auth, 'https://oss-cn-hangzhou.aliyuncs.com', 'examplebucket')
    
    object_name = "yourAppendObjectName"
    first_content = "yourFirstContent"
    second_content = "yourSecondContent"
    
    # Specify init_crc in the first append upload. 
    # If init_crc is specified, OSS SDKs perform CRC-64 for the returned results. 
    result = bucket.append_object(object_name, 0, first_content, init_crc=0)
    
    # Specify init_crc in the second append upload. 
    # Set init_crc to the CRC-64 value of the uploaded object. 
    result = bucket.append_object(object_name, result.next_position, second_content, init_crc=result.crc)

References

For the complete sample code that is used for MD5 verification and CRC-64, visit GitHub.