通過crc64校驗資料轉送的完整性

背景

數據在客戶端和伺服器之間傳輸時有可能會出錯。OSS現在支援對各種方式上傳的Object返回其crc64值，客戶端可以和本地計算的crc64值做對比，從而完成資料完整性的驗證。

• OSS對新上傳的Object進行crc64的計算，並將結果儲存為Object的元資訊儲存，隨後在返回的response header中增加x-oss-hash-crc64ecma頭部，表示其crc64值，該64位CRC根據ECMA-182標準計算得出。
• 對於crc64上線之前就已經存在於OSS上的Object，OSS不會對其計算crc64值，所以獲取此類Object時不會返回其crc64值。

操作說明

應用樣本

以下為完整的Python範例程式碼，示範如何基於crc64值驗證資料轉送的完整性。

1. 計算crc64。

   import oss2
   from oss2.models import PartInfo
   import os
   import crcmod
   import random
   import string
   do_crc64 = crcmod.mkCrcFun(0x142F0E1EBA9EA3693L, initCrc=0L, xorOut=0xffffffffffffffffL, rev=True)
   def check_crc64(local_crc64, oss_crc64, msg="check crc64"):
   if local_crc64 != oss_crc64:
   print "{0} check crc64 failed. local:{1}, oss:{2}.".format(msg, local_crc64, oss_crc64)
   return False
   else:
   print "{0} check crc64 ok.".format(msg)
   return True
   def random_string(length):
   return ''.join(random.choice(string.lowercase) for i in range(length))
   bucket = oss2.Bucket(oss2.Auth(access_key_id, access_key_secret), endpoint, bucket_name)

2. 驗證Put Object。

   content = random_string(1024)
    key = 'normal-key'
    result = bucket.put_object(key, content)
    oss_crc64 = result.headers.get('x-oss-hash-crc64ecma', '')
    local_crc64 = str(do_crc64(content))
    check_crc64(local_crc64, oss_crc64, "put object")

3. 驗證Get Object。

   result = bucket.get_object(key)
    oss_crc64 = result.headers.get('x-oss-hash-crc64ecma', '')
    local_crc64 = str(do_crc64(result.resp.read()))
    check_crc64(local_crc64, oss_crc64, "get object")

4. 驗證Upload Part 和 Complete。

   part_info_list = []
    key = "multipart-key"
    result = bucket.init_multipart_upload(key)
    upload_id = result.upload_id
    part_1 = random_string(1024 * 1024)
    result = bucket.upload_part(key, upload_id, 1, part_1)
    oss_crc64 = result.headers.get('x-oss-hash-crc64ecma', '')
    local_crc64 = str(do_crc64(part_1))
    #check 上傳的 part 1數據是否完整
    check_crc64(local_crc64, oss_crc64, "upload_part object 1")
    part_info_list.append(PartInfo(1, result.etag, len(part_1)))
    part_2 = random_string(1024 * 1024)
    result = bucket.upload_part(key, upload_id, 2, part_2)
    oss_crc64 = result.headers.get('x-oss-hash-crc64ecma', '')
    local_crc64 = str(do_crc64(part_2))
    #check 上傳的 part 2數據是否完整
    check_crc64(local_crc64, oss_crc64, "upload_part object 2")
    part_info_list.append(PartInfo(2, result.etag, len(part_2)))
    result = bucket.complete_multipart_upload(key, upload_id, part_info_list)
    oss_crc64 = result.headers.get('x-oss-hash-crc64ecma', '')
    local_crc64 = str(do_crc64(part_2, do_crc64(part_1)))
    #check 最終oss上的object和本地檔案是否一致
    check_crc64(local_crc64, oss_crc64, "complete object")

OSS SDK支援

部分OSS SDK已經支援上傳、下載使用crc64進行數據校驗，用法見下表中的樣本：