Protect data through client encryption

Last Updated: May 09, 2017

Client encryption means that encryption is completed before the user data is sent to the remote server, while the plaintext of the key used for encryption is kept in the local computer only. Therefore, the security of user data can be ensured because others cannot decrypt data to obtain the original data even if data leaks.

This document describes how to protect data through client encryption based on the current Python SDK version of OSS.

Introduction to principles

  1. The user maintains a pair of RSA keys (rsa_private_key and rsa_public_key) in the local computer.
  2. Each time when an object is uploaded, a symmetric key data_key of AES256 type is generated at random, and then data_key is used to encrypt the original content to obtain encrypt_content.
  3. Use rsa_public_key to encrypt data_key to obtain encrypt_data_key, place it in the request header as the custom meta of the user, and send it together with encrypt_content to the OSS.
  4. When Get Object is performed, encrypt_content and encrypt_data_key in the custom meta of the user are obtained first.
  5. The user uses rsa_private_key to decrypt encrypt_data_key to obtain data_key, and then uses data_key to decrypt encrypt_content to obtain the original content.

Note: The user’s key in this document is an asymmetric RSA key, and the AES256-CTR algorithm is used when object content is encrypted. For details, refer to PyCrypto Document. This document describes how to implement client encryption through the custom meta of an object. The user can select the encryption key type and encryption algorithm as needed.

Structural diagram

structure

Preparation

  1. For installation and use of the Python SDK, refer to Quick Installation of Python SDK.

  2. Install the PyCrypto library.

    1. pip install pycrypto

Example of complete Python code

  1. # -*- coding: utf-8 -*-
  2. import os
  3. import shutil
  4. import base64
  5. import random
  6. import oss2
  7. from Crypto.Cipher import PKCS1_OAEP
  8. from Crypto.PublicKey import RSA
  9. from Crypto.Cipher import AES
  10. from Crypto import Random
  11. from Crypto.Util import Counter
  12. # aes 256, key always is 32 bytes
  13. _AES_256_KEY_SIZE = 32
  14. _AES_CTR_COUNTER_BITS_LEN = 8 * 16
  15. class AESCipher:
  16. def __init__(self, key=None, start=None):
  17. self.key = key
  18. self.start = start
  19. if not self.key:
  20. self.key = Random.new().read(_AES_256_KEY_SIZE)
  21. if not self.start:
  22. self.start = random.randint(1, 10)
  23. ctr = Counter.new(_AES_CTR_COUNTER_BITS_LEN, initial_value=self.start)
  24. self.cipher = AES.new(self.key, AES.MODE_CTR, counter=ctr)
  25. def encrypt(self, raw):
  26. return self.cipher.encrypt(raw)
  27. def decrypt(self, enc):
  28. return self.cipher.decrypt(enc)
  29. # First, initialize the information such as AccessKeyId, AccessKeySecret, and Endpoint.
  30. # Obtain the information through environment variables or replace the information such as "<Your AccessKeyId>" with the real AccessKeyId, and so on.
  31. #
  32. # Use Hangzhou region as an example. Endpoint can be:
  33. # http://oss-cn-hangzhou.aliyuncs.com
  34. # https://oss-cn-hangzhou.aliyuncs.com
  35. # Access using the HTTP and HTTPS protocols respectively.
  36. access_key_id = os.getenv('OSS_TEST_ACCESS_KEY_ID', '<Your AccessKeyId>')
  37. access_key_secret = os.getenv('OSS_TEST_ACCESS_KEY_SECRET', '<Your AccessKeySecret>')
  38. bucket_name = os.getenv('OSS_TEST_BUCKET', '<Your Bucket>')
  39. endpoint = os.getenv('OSS_TEST_ENDPOINT', '<Your Access Domain Name>')
  40. # Make sure that all the above parameters have been filled in correctly.
  41. for param in (access_key_id, access_key_secret, bucket_name, endpoint):
  42. assert '<' not in param, 'Please set the parameter' + param
  43. ##### 0 prepare ########
  44. # 0.1 Generate the RSA key file and save it to the disk
  45. rsa_private_key_obj = RSA.generate(2048)
  46. rsa_public_key_obj = rsa_private_key_obj.publickey()
  47. encrypt_obj = PKCS1_OAEP.new(rsa_public_key_obj)
  48. decrypt_obj = PKCS1_OAEP.new(rsa_private_key_obj)
  49. # save to local disk
  50. file_out = open("private_key.pem", "w")
  51. file_out.write(rsa_private_key_obj.exportKey())
  52. file_out.close()
  53. file_out = open("public_key.pem", "w")
  54. file_out.write(rsa_public_key_obj.exportKey())
  55. file_out.close()
  56. # 0.2 Create the Bucket object. All the object-related interfaces can be implemented by using the Bucket object
  57. bucket = oss2.Bucket(oss2.Auth(access_key_id, access_key_secret), endpoint, bucket_name)
  58. obj_name = 'test-sig-1'
  59. content = "test content"
  60. #### 1 Put Object ####
  61. # 1.1 Generate the one-time symmetric key encrypt_cipher used to encrypt this object, where key and start are values generated at random
  62. encrypt_cipher = AESCipher()
  63. # 1.2 Use the public key to encrypt the information for assisting encryption, and save it in the custom meta of the object. When Get Object is performed later, we can use the private key to perform decryption and obtain the original content according to the custom meta
  64. headers = {}
  65. headers['x-oss-meta-x-oss-key'] = base64.b64encode(encrypt_obj.encrypt(encrypt_cipher.key))
  66. headers['x-oss-meta-x-oss-start'] = base64.b64encode(encrypt_obj.encrypt(str(encrypt_cipher.start)))
  67. # 1.3. Use encrypt_cipher to encrypt the original content to obtain encrypt_content
  68. encryt_content = encrypt_cipher.encrypt(content)
  69. # 1.4 Upload the object
  70. result = bucket.put_object(obj_name, encryt_content, headers)
  71. if result.status / 100 != 2:
  72. exit(1)
  73. #### 2 Get Object ####
  74. # 2.1 Download the encrypted object
  75. result = bucket.get_object(obj_name)
  76. if result.status / 100 != 2:
  77. exit(1)
  78. resp = result.resp
  79. download_encrypt_content = resp.read()
  80. # 2.2 Resolve from the custom meta the key and start that are previously used to encrypt this object
  81. download_encrypt_key = base64.b64decode(resp.headers.get('x-oss-meta-x-oss-key', ''))
  82. key = decrypt_obj.decrypt(download_encrypt_key)
  83. download_encrypt_start = base64.b64decode(resp.headers.get('x-oss-meta-x-oss-start', ''))
  84. start = int(decrypt_obj.decrypt(download_encrypt_start))
  85. # 2.3 Generate the cipher used for decryption, and decrypt it to obtain the original content
  86. decrypt_cipher = AESCipher(key, start)
  87. download_content = decrypt_cipher.decrypt(download_encrypt_content)
  88. if download_content != content:
  89. print "Error!"
  90. else:
  91. print "Decrypt ok. Content is: %s" % download_content
Thank you! We've received your feedback.