All Products
Search
Document Center

MaxCompute:Storage encryption

Last Updated:Dec 13, 2023

If sensitive information such as personal identity information, financial records, and health records is stored in a MaxCompute project, you can enable storage encryption to protect the data against access by unauthorized users. MaxCompute allows you to use Key Management Service (KMS) to encrypt data for storage. MaxCompute provides static data protection to meet the requirements of enterprise governance and security compliance.

Storage encryption mechanism

MaxCompute uses customer master keys (CMKs) from KMS to encrypt or decrypt data based on the following data encryption mechanism:

  • The data encryption feature is enabled for a MaxCompute project.

  • You can create and manage a CMK in the KMS console to ensure the security of the CMK.

  • MaxCompute supports the AES-256, AESCTR, and RC4 encryption algorithms.

  • MaxCompute allows you to use CMKs that are created based on MaxCompute Default Key and Bring Your Own Key (BYOK) to encrypt or decrypt data.

    • When you create a MaxCompute project, you can set Key to MaxCompute Default Key.

      MaxCompute automatically creates a key for the MaxCompute project in KMS and uses the key as the CMK of the project. You can view the key information in the KMS console.

    • To meet business and security requirements in different scenarios, MaxCompute can use BYOKs to encrypt or decrypt data.

      When you use bring your own keys (BYOKs) to encrypt or decrypt data, you must manually activate KMS. After KMS is activated, you can create BYOKs in the KMS console and select a BYOK as the CMK when you create a MaxCompute project. For more information about how to create a CMK in the KMS console, see CreateKey.

      Note

      If a MaxCompute project needs to use a BYOK, you must complete Resource Access Management (RAM) authorization as prompted when you create the project.

  • You can create custom RAM policies to manage permissions on MaxCompute projects, such as the permissions to encrypt data in a project. For more information, see RAM permissions.

Billing rules

You are not charged for enabling the data encryption feature for MaxCompute projects. During data encryption and decryption, MaxCompute interacts with the API operations of KMS. You are charged for using KMS. For more information about billing, see Billing of KMS.

Limits

The data encryption feature of MaxCompute has the following limits:

  • If you want to access encrypted data in MaxCompute from a Hologres instance by using an external table, the version of the Hologres instance must be V1.1 or later, and you must grant the permission on KMS to your Hologres instance. For more information, see Query MaxCompute data encrypted based on BYOK.

  • If you use BYOKs to encrypt or decrypt data, you must manually activate KMS in the region where the current MaxCompute project resides. Before you activate KMS, make sure that KMS is supported in the region. The following table describes the mappings between the regions where MaxCompute projects reside and the regions where KMS can be activated.

    Region where a MaxCompute project resides

    Region where KMS can be activated

    China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), and China (Chengdu)

    China (Shanghai)

    Regions other than the preceding regions

    Same as the region where the MaxCompute project resides

  • Your operations such as the disable or delete operation on your CMKs in KMS may affect data encryption and decryption in MaxCompute. MaxCompute caches historical configurations. Your operations in KMS take effect in a delayed manner within 24 hours.

  • You cannot disable the data encryption feature or change the storage encryption algorithm for existing projects.

  • If you enable storage encryption for an existing project, data in the project is not automatically encrypted when the data is read from or written to the project. If you want to encrypt data in an existing project, you must manually read the data from the project and then write the data to the project.

Procedure

Enable storage encryption for a new MaxCompute project

  • Method 1: Create a project in the MaxCompute console and enable storage encryption for the project

    1. Log on to the KMS console. On the Key Management Service page, read the terms of service, select Key Management Service Terms of Service, and then click Activate Now to activate KMS.开通KMS

      Note

      You can skip this step if you have activated KMS in the region to which your project belongs.

    2. Log on to the MaxCompute console. In the top navigation bar, select a region.

    3. In the left-side navigation pane, choose Workspace > Projects.

    4. On the Projects page, click Create Project.

    5. In the Create Project dialog box, configure the parameters as prompted.

      The following table describes the parameters on which you need to focus.

      Parameter

      Description

      Billing Method

      The billing method of computing resources, which is also the billing method of the default quota group.

      Default Quota

      Quota groups are used to allocate computing resources.

      If you do not specify a quota group for your project, the jobs initiated by your project consume the computing resources in the default quota group. For more information about how to use computing resources, see Use of computing resources.

      Max Resources Consumed by An SQL Statement

      The upper limit for the resources that can be consumed by an SQL job.

      Formula: Amount of scanned data (GB) × Complexity. This parameter is optional. If you select Pay-as-you-go for Billing Method, we recommend that you configure this parameter to prevent a single SQL job from consuming excessive resources. We also recommend that you configure real-time consumption control to monitor resources consumed by computing jobs. This helps you prevent high resource consumption. For more information, see Consumption control.

      Data Type Edition

      The data type edition of MaxCompute. Valid values: MaxCompute V1.0 Data Type Edition (Suitable for Early MaxCompute Projects), MaxCompute V2.0 Data Type Edition (Recommended), and Hive-Compatible Data Type Edition (Suitable for MaxCompute Projects Migrated from Hadoop).

      Select a data type edition based on your business requirements. For more information about the differences among the three data type editions, see Data type editions.

      Encrypt

      Specifies whether to enable the data encryption feature for the MaxCompute project that you create. If you select Yes, you must configure the following parameters:

      • Key: the type of the key that is used in the MaxCompute project. You can select MaxCompute Default Key or BYOK. If you select MaxCompute Default Key, the key that MaxCompute automatically creates for the project is used.

      • Algorithm: The encryption algorithm that is supported by the key. Valid values: AES256, AESCTR, and RC4.

    6. Click OK.

  • Method 2: Enable the data encryption feature for a MaxCompute project in the DataWorks console

    1. Log on to the KMS console. On the Key Management Service page, read the terms of service, select Key Management Service Terms of Service, and then click Activate Now to activate KMS.开通KMS

      Note

      You can skip this step if you have activated KMS in the region to which your project belongs.

    2. Log on to the DataWorks console. In the left-side navigation pane, click Workspaces.

    3. On the Workspaces page, select a region in the upper-left corner and click Create Workspace. In the Create Workspace panel, configure the parameters and click Commit. For more information, see Create a MaxCompute project.

    4. In the Create Workspace panel, find the MaxCompute compute engine in the Recommended Big Data Compute Engines: section and click Associate Now.

    5. Configure parameters on the Associate MaxCompute Compute Engine page. Select Encryption for the Encrypted parameter.

      引擎详情

      Item

      Parameter

      Description

      MaxCompute

      Resource Display Name

      • The display name of the MaxCompute compute engine instance. The display name is used to identify the configuration of the compute engine and is similar to the alias of the MaxCompute compute engine in DataWorks.

      • You can specify a display name based on your business requirements. The display name must be unique.

      Project Source

      The default value is Create project.

      Payment Model

      The billing method of the MaxCompute project. Valid values: Pay-as-you-go and Subscription. For more information about the billing methods of MaxCompute, see Overview.

      Note

      You cannot associate a MaxCompute project of the developer version with a workspace in standard mode as a compute engine instance.

      Quota Group

      The computing resource pool that is used by the MaxCompute project. For more information about quotas, see Quota.

      Data Type

      The data type edition of the MaxCompute project. The data type edition of the MaxCompute compute engine instance. Valid values: 2.0 Data Type (Recommended), 1.0 Data Type (for users who already use 1.0 data type), and Hive Compatible Type (for Hive migration users). For more information, see Data type editions.

      Encrypted

      Specifies whether to enable data encryption for an existing project.

      Project Name

      The name must be 3 to 27 characters in length, and can contain only letters, digits, and underscores (_). It must start with a letter.

      Scheduled Access Identity

      The account that is used to run code in the MaxCompute project after a DataWorks task is submitted and scheduled.

      RAM User

      The current logon account.

    6. Click Complete Association.

      After the data encryption feature is enabled, MaxCompute automatically encrypts or decrypts data that is written to and read from the MaxCompute project.

Enable storage encryption for existing projects

  • Precautions

    • To enable storage encryption for an existing project, you must modify the parameters in the Basic Properties section of the Parameter Configuration tab of the project. Only RAM users to which the Super_Administrator is assigned can enable storage encryption for existing projects.

    • To configure the permissions and the IP address whitelist of a MaxCompute project, you must grant the Super_Adminstrator, the Admin, or a custom administrator role the required permissions on the project. For more information, see Permissions on project management.

    • You can enable storage encryption only for projects for which storage encryption is not enabled. For projects for which storage encryption is enabled, you cannot disable storage encryption or change the storage encryption algorithm.

  • Procedure

    1. Log on to the MaxCompute console. In the top navigation bar, select a region.

    2. In the left-side navigation pane, choose Workspace > Projects.

    3. On the Projects page, find the desired project and click Manage in the Actions column.

    4. On the Parameter Configuration tab of the Project Settings page, click Edit in the Basic Properties section.

    5. Select Yes for Storage Encryption Status.

    6. In the Encryption Settings dialog box, configure the Key and Algorithm parameter and click OK.

      • Key: the type of the key that is used in the MaxCompute project. You can select MaxCompute Default Key or Bring Your Own Key (BYOK). If you select MaxCompute Default Key, the key that MaxCompute automatically creates for the project is used.

      • Algorithm: The encryption algorithm that is supported by the key. Valid values: AES256, AESCTR, and RC4.

    7. Click Submit in the Basic Properties section. The storage encryption feature is enabled for the existing project.

References

  • You can also use ACL-based access control to grant permissions on a project or a table to a user or a role. For more information, see ACL-based access control.

  • If a user has the permissions to query specific sensitive data in a MaxCompute project and you do not want the user to view complete sensitive data, you can enable the dynamic data masking feature of MaxCompute to dynamically mask sensitive data in the query results. For more information, see Dynamic data masking.

  • If you want to encrypt specific data in a table, you can use encryption functions of MaxCompute. For more information, see Encryption and decryption functions.