Object Storage Service:Client-side encryption

Disclaimer

• When you use client-side encryption, you must ensure the integrity and validity of the customer master key (CMK). If the CMK is incorrectly used or lost due to improper maintenance, you will be held responsible for all losses and consequences caused by decryption failures.
• When you copy or migrate encrypted data, you must ensure the integrity and validity of the object metadata related to client-side encryption. If the encrypted metadata is incorrectly used or lost due to improper maintenance, you will be held responsible for all losses and consequences caused by decryption failures.

Usage notes

• In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS by using other Alibaba Cloud services in the same region as OSS, use an internal endpoint For more information about the regions and endpoints supported by OSS, see Regions and endpoints.
• In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or STS, see Initialization.

Background information

In client-side encryption, a random data key is generated for each object to perform symmetric encryption on the object. The client uses a CMK to encrypt the random data key. The encrypted data key is uploaded as a part of the object metadata and stored in the OSS server. When an encrypted object is downloaded, the client uses the CMK to decrypt the random data key and then uses the data key to decrypt the object. The CMK is used only on the client and is not transmitted over the network or stored in the server. This ensures data security.

Encryption methods

You can use two types of CMKs for client-side encryption:

• KMS-managed CMKs

  When you use a CMK managed in Key Management Service (KMS) for client-side encryption, you must send the CMK ID to OSS SDK for Python.

• RSA-based CMKs managed by yourself

  When you use a CMK managed by yourself for client-side encryption, you must send the public key and the private key of your CMK to OSS SDK for Python as parameters.

You can use the preceding methods to prevent data leaks and protect your data on the client. Even if your data is leaked, the data cannot be decrypted by other users.

Object metadata related to client-side encryption

Use RSA-based CMKs to encrypt objects in simple upload or decrypt objects in simple download

The following code provides an example on how to use a CMK to encrypt objects in simple upload or decrypt objects in simple download:

package main

import (
  "bytes"
  "fmt"
  "io/ioutil"
  "os"

  "github.com/aliyun/aliyun-oss-go-sdk/oss"
  "github.com/aliyun/aliyun-oss-go-sdk/oss/crypto"
)

func main() {
  // Create an OSSClient instance. 
  // Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
  // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
  client, err := oss.New("yourEndpoint", "yourAccessKeyId", "yourAccessKeySecret")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Specify a description for the CMK. The description cannot be modified after being specified. You can specify only one description for a CMK. 
  // If all objects share a CMK, you can leave the description of the CMK empty. In this case, the CMK cannot be replaced. 
  // If the description of the CMK is left empty, the client cannot determine which CMK to use for decryption. 
  // We recommend that you configure a description that is a JSON string for each CMK and store the mapping relationship between the CMK and the description on the client. The server does not store the mapping relationship. 

  Obtain the mapping relationship based on the CMK description that is a JSON string. 
  materialDesc := make(map[string]string)
  materialDesc["desc"] = "your master encrypt key material describe information"

  // Create a CMK object based on the description of the CMK. 
  // Set yourRsaPublicKey to the public key of the CMK that you manage, and set yourRsaPrivateKey to the private key of the CMK that you manage. 
  masterRsaCipher, err := osscrypto.CreateMasterRsa(materialDesc, "yourRsaPublicKey", "yourRsaPrivateKey")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Create an encryption operation based on the CMK object and perform encryption in AES CTR mode. 
  contentProvider := osscrypto.CreateAesCtrCipher(masterRsaCipher)

  // Obtain an existing bucket to be used for client-side encryption. 
  // The bucket for which client-side encryption is configured is used in the same manner in which a common bucket is used. 
  cryptoBucket, err := osscrypto.GetCryptoBucket(client, "yourBucketName", contentProvider)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Encryption is automatically performed when you call the PutObject operation. 
  err = cryptoBucket.PutObject("yourObjectName", bytes.NewReader([]byte("yourObjectValueByteArrary")))
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Decryption is automatically performed when you call the GetObject operation. 
  body, err := cryptoBucket.GetObject("yourObjectName")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }
  defer body.Close()

  data, err := ioutil.ReadAll(body)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }
  fmt.Println("data:", string(data))
}

Use RSA-based CMKs to encrypt objects in multipart upload

The following code provides an example on how to use an RSA-based CMK to encrypt an object in multipart upload:

package main

import (
  "fmt"
  "os"

  "github.com/aliyun/aliyun-oss-go-sdk/oss"
  "github.com/aliyun/aliyun-oss-go-sdk/oss/crypto"
)

func main() {
  // Create an OSSClient instance. 
  // Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
  // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
  client, err := oss.New("yourEndpoint", "yourAccessKeyId", "yourAccessKeySecret")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Specify a description for the CMK. The description cannot be modified after being specified. You can specify only one description for a CMK. 
  // If all objects share a CMK, you can leave the description of the CMK empty. In this case, the CMK cannot be replaced. 
  // If the description of the CMK is left empty, the client cannot determine which CMK to use for decryption. 
  // We recommend that you configure a description that is a JSON string for each CMK and store the mapping relationship between the CMK and the description on the client. The server does not store the mapping relationship. 

  Obtain the mapping relationship based on the CMK description that is a JSON string. 
  materialDesc := make(map[string]string)
  materialDesc["desc"] = "your master encrypt key material describe information"

  // Create a CMK object based on the description of the CMK. 
  // Set yourRsaPublicKey to the public key of the CMK that you manage, and set yourRsaPrivateKey to the private key of the CMK that you manage. 
  masterRsaCipher, err := osscrypto.CreateMasterRsa(materialDesc, "yourRsaPublicKey", "yourRsaPrivateKey")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Create an encryption operation based on the CMK object and perform encryption in AES CTR mode. 
  contentProvider := osscrypto.CreateAesCtrCipher(masterRsaCipher)

  // Obtain an existing bucket to be used for client-side encryption. 
  // The bucket for which client-side encryption is configured is used in the same manner in which a common bucket is used. 
  cryptoBucket, err := osscrypto.GetCryptoBucket(client, "yourBucketName", contentProvider)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  fileName := "yourLocalFilePath"
  fileInfo, err := os.Stat(fileName)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }
  fileSize := fileInfo.Size()

  // Encrypt the context. 
  var cryptoContext osscrypto.PartCryptoContext
  cryptoContext.DataSize = fileSize

  // Specify the number of parts that you want to encrypt. The actual number is based on the calculation result. 
  expectPartCount := int64(10)

  // Specify the size of a part encrypted in AES CTR mode, which is an integer multiple of 16 bytes. 
  cryptoContext.PartSize = (fileSize / expectPartCount / 16) * 16

  imur, err := cryptoBucket.InitiateMultipartUpload("yourObjectName", &cryptoContext)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  chunks, err := oss.SplitFileByPartSize(fileName, cryptoContext.PartSize)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  var partsUpload []oss.UploadPart
  for _, chunk := range chunks {
    part, err := cryptoBucket.UploadPartFromFile(imur, fileName, chunk.Offset, chunk.Size, (int)(chunk.Number), cryptoContext)
    if err != nil {
      fmt.Println("Error:", err)
      os.Exit(-1)
    }
    partsUpload = append(partsUpload, part)
  }

  // Complete
  _, err = cryptoBucket.CompleteMultipartUpload(imur, partsUpload)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }
}

Decrypt objects that are encrypted by using different RSA-based CMKs

The following code provides an example on how to decrypt an object that is encrypted by using different RSA-based CMKs:

package main

import (
  "bytes"
  "fmt"
  "io/ioutil"
  "os"

  "github.com/aliyun/aliyun-oss-go-sdk/oss"
  "github.com/aliyun/aliyun-oss-go-sdk/oss/crypto"
)

// Query the CMK based on the description of the CMK. To decrypt objects that are encrypted by using different CMKs, you must perform this operation. 
type MockRsaManager struct {
}

func (mg *MockRsaManager) GetMasterKey(matDesc map[string]string) ([]string, error) {  
  keyList := []string{"yourRsaPublicKey", "yourRsaPrivatKey"}
  return keyList, nil
}

// Decrypt objects encrypted by different CMKs. 
func main() {
  // Create an OSSClient instance. 
  // Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
  // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
  client, err := oss.New("yourEndpoint", "yourAccessKeyId", "yourAccessKeySecret")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Specify a description for the CMK. The description cannot be modified after being specified. You can specify only one description for a CMK. 
  // If all objects share a CMK, you can leave the description of the CMK empty. In this case, the CMK cannot be replaced. 
  // If the description of the CMK is left empty, the client cannot determine which CMK to use for decryption. 
  // We recommend that you configure a description that is a JSON string for each CMK and store the mapping relationship between the CMK and the description on the client. The server does not store the mapping relationship. 

  Obtain the mapping relationship based on the CMK description that is a JSON string. 
  materialDesc := make(map[string]string)
  materialDesc["desc"] = "your master encrypt key material describe information"

  // Create a CMK object based on the description of the CMK. 
  // Set yourRsaPublicKey to the public key of the CMK that you manage, and set yourRsaPrivateKey to the private key of the CMK that you manage. 
  masterRsaCipher, err := osscrypto.CreateMasterRsa(materialDesc, "yourRsaPublicKey", "yourRsaPrivateKey")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Create an encryption operation based on the CMK object and perform encryption in AES CTR mode. 
  contentProvider := osscrypto.CreateAesCtrCipher(masterRsaCipher)

  // To decrypt objects that are encrypted by using different CMKs, you must perform this operation. 
  var mockRsaManager MockRsaManager
  var options []osscrypto.CryptoBucketOption
  options = append(options, osscrypto.SetMasterCipherManager(&mockRsaManager))

  // Obtain an existing bucket to be used for client-side encryption. 
  // The bucket for which client-side encryption is configured is used in the same manner in which a common bucket is used. 
  cryptoBucket, err := osscrypto.GetCryptoBucket(client, "yourBucketName", contentProvider, options...)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Encryption is automatically performed when you call the PutObject operation. 
  err = cryptoBucket.PutObject("yourObjectName", bytes.NewReader([]byte("yourObjectValueByteArrary")))
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Decryption is automatically performed when you call the GetObject operation. 
  body, err := cryptoBucket.GetObject("otherObjectNameEncryptedWithOtherRsa")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }
  defer body.Close()

  data, err := ioutil.ReadAll(body)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }
  fmt.Println("data:", string(data))
}

Use a CMK in KMS to encrypt an object in simple upload and decrypt an object in simple download

The following code provides an example on how to use a CMK stored in KMS to encrypt an object in simple upload or decrypt an object in simple download:

package main

import (
  "bytes"
  "fmt"
  "io/ioutil"
  "os"

  kms "github.com/aliyun/alibaba-cloud-sdk-go/services/kms"
  "github.com/aliyun/aliyun-oss-go-sdk/oss"
  "github.com/aliyun/aliyun-oss-go-sdk/oss/crypto"
)

func main() {
  // Create an OSSClient instance. 
  // Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
  // The AccessKey pair of an Alibaba Cloud account has permissions on all API operations. Using these credentials to perform operations in OSS is a high-risk operation. We recommend that you use a RAM user to call API operations or perform routine O&M. To create a RAM user, log on to the RAM console. 
  client, err := oss.New("yourEndpoint", "yourAccessKeyId", "yourAccessKeySecret")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Create a KMSClient instance. 
  kmsClient, err := kms.NewClientWithAccessKey("yourKmsRegion", "yourKmsAccessKeyId", "yourKmsAccessKeySecret")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Specify a description for the CMK. The description cannot be modified after being specified. You can specify only one description for a CMK. 
  // If all objects share a CMK, you can leave the description of the CMK empty. In this case, the CMK cannot be replaced. 
  // If the description of the CMK is left empty, the client cannot determine which CMK to use for decryption. 
  // We recommend that you configure a description that is a JSON string for each CMK and store the mapping relationship between the CMK and the description on the client. The server does not store the mapping relationship. 

    Obtain the mapping relationship based on the CMK description that is a JSON string. 
  materialDesc := make(map[string]string)
  materialDesc["desc"] = "your kms encrypt key material describe information"

  // Create a CMK object based on the description of the CMK. 
  // Set yourKmsId to the ID of the CMK in KMS. 
  masterkmsCipher, err := osscrypto.CreateMasterAliKms(materialDesc, "yourKmsId", kmsClient)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Create an encryption operation based on the CMK object and perform encryption in AES CTR mode. 
  contentProvider := osscrypto.CreateAesCtrCipher(masterkmsCipher)

  // Obtain an existing bucket to be used for client-side encryption. 
  // The bucket for which client-side encryption is configured is used in the same manner in which a common bucket is used. 
  cryptoBucket, err := osscrypto.GetCryptoBucket(client, "yourBucketName", contentProvider)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Encryption is automatically performed when you call the PutObject operation. 
  err = cryptoBucket.PutObject("yourObjectName", bytes.NewReader([]byte("yourObjectValueByteArrary")))
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }

  // Decryption is automatically performed when you call the GetObject operation. 
  body, err := cryptoBucket.GetObject("yourObjectName")
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }
  defer body.Close()

  data, err := ioutil.ReadAll(body)
  if err != nil {
    fmt.Println("Error:", err)
    os.Exit(-1)
  }
  fmt.Println("data:", string(data))
}