Perform static de-identification

Create a de-identification task

1. Log on to the DSC console.
2. In the left-side navigation pane, choose Data desensitization > Static Desensitization.
3. On the Static Desensitization page, click Add Desensitization Task.
4. Perform the steps in the wizard to create a custom de-identification task.
   1. Enter basic task information and click Next.
      Note You can specify a custom task name.
   2. Configure the data to be de-identified and click Next. The following tables describe the parameters for configuring the data to be de-identified of different types.
      • Specify an ApsaraDB RDS, MaxCompute, or PolarDB table to be de-identified

        Parameter	Description
        Types of data storage	The type of the data asset that contains the sensitive data to be de-identified. Set this parameter to RDS table/DRDS table/MaxCompute table/PolarDB table/OceanBase table.
        Source Product	The Alibaba Cloud service that contains the sensitive data to be de-identified. Valid values: RDS, MaxCompute, and PolarDB.
        Source Database/Project	Required. The project or database for storing the table that contains the sensitive data to be de-identified.
        Source Table Name	Required. The table that contains the sensitive data to be de-identified.
        Source Partition	Optional. The partition that contains the sensitive data to be de-identified. For more information about how to specify a partition, see Specify a partition. You can configure partitions when you create a MaxCompute table. Partitions define different logical divisions of a table to help you efficiently query specific content. For more information, see Partition. Note If you set the Source Product parameter to RDS or PolarDB, you do not need to set the Source Partition parameter. If you leave the Source Partition parameter unspecified, SDDP de-identifies sensitive data in all partitions of the table.
        Sample SQL	Optional. The SQL statement that specifies the data to be de-identified. If you leave this parameter unspecified, SDDP de-identifies all sensitive data in the table. Note If you set the Source Product parameter to MaxCompute or PolarDB, you do not need to set the Sample SQL parameter.

      • Specify an OSS file to be de-identified

        Parameter	Description
        Types of data storage	The type of the data asset that contains the sensitive data to be de-identified. Set this parameter to OSS files.
        File source	The source of the file that contains the sensitive data to be de-identified. Valid values: Uploaded Local File and OSS Bucket. Note You can upload only a TXT, CSV, XLSX, or XLS file.
        OSS Bucket where the source file is located	The OSS bucket where the source file resides. You can enter keywords to search for the OSS bucket where the source file resides.
        Source file names	The name of the source file. The file name must contain a file name extension. Only TXT, CSV, XLSX, and XLS files are supported. To specify multiple source files of the same format at a time, turn on Open the pass. Note After you turn on the switch, you can use the wildcard (*) to specify multiple source files at a time. You can use the wildcard only in the file name prefix, for example, test*.xls. SDDP applies the same de-identification rules to all the source files that are specified at a time. Make sure that the source files share the same schema.
        Source file description	The description of the source file. If you set the File source parameter to OSS Bucket, you do not need to specify this parameter.
        Separator selection	Optional. The field delimiter. This parameter is required for CSV and TXT files. Select a field delimiter based on the file format. Valid values: Semicolon ";" (macOS/Linux default) Comma "," (Windows default)
        Table contains header rows	Optional. Specifies whether the data to be de-identified includes the table header.

   3. Configure the de-identification algorithm and click Next.

      In the field list, turn on Desensitization for each field that requires de-identification and select a de-identification algorithm as required. For more information about de-identification algorithms, see Configure de-identification algorithms.

      Note

      • You can click View and Modify Parameters next to the selected algorithm to view or modify the rule of the algorithm.
      • A field is de-identified only after Desensitization is turned on for this field.
   4. Specify the destination location for storing the de-identified data. Click Test to check whether SDDP can write data to the specified destination location. After the test is passed, click Next.
   5. Configure the processing logic.
      

      Parameter	Description
      How the task is triggered	The mode in which the de-identification task is run. Valid values: Manual Only: You must manually run the de-identification task on the Static Desensitization page. Scheduled Only: The de-identification task is automatically run at the specified time on an hourly, daily, or monthly basis. Manual + Scheduled: The de-identification task is automatically run at the specified time on an hourly, daily, weekly, or monthly basis. You can also manually run the de-identification task as required.
      Turn on incremental desensitization	Optional. Specifies whether to enable incremental de-identification. If you turn on this switch, only the data that is added after the last de-identification will be de-identified. You must specify a field whose value is increased over time as the incremental identifier. For example, you can specify the creation time field or the auto-increment ID field as the incremental identifier. Note SDDP supports incremental de-identification only for ApsaraDB RDS databases.
      Shard field	Optional. The field based on which DSC divides the source data to multiple shards and concurrently de-identifies them. This improves the de-identification efficiency. You can specify one or more shard fields as required. Note SDDP supports incremental de-identification only for ApsaraDB RDS databases. We recommend that you use the primary key or a unique index as the shard field. If you do not specify a shard field, DSC uses the primary key as the shard field by default to de-identify the source data. You must specify a shard field if the source data does not have a primary key. Otherwise, the de-identification task fails. The query performance and data accuracy may deteriorate if you specify excessive shard fields. Exercise caution when you set this parameter.
      Table name conflict resolution	The solution used if the specified destination table already exists. Valid values: Delete the target table and create a new table with the same name Attach data to the target table: We recommend that you select this option.
      Row Conflict Resolution	The solution used if conflicting rows exist in the destination table. Valid values: Keep conflicting rows in the target table and discard the new data: We recommend that you select this option. Delete conflicting rows in the target table and insert the new data

   6. Click Submit.
      The de-identification task appears on the Desensitization task configuration tab of the Static Desensitization page.
5. Run the de-identification task.
   After the de-identification task is created, find the de-identification task, turn on the switch, and then click Start in the Actions column to run the de-identification task.

   Note DSC allows you to modify or delete a de-identification task after it is created. However, a running de-identification task cannot be modified or deleted.
6. View the running progress and status of the de-identification task.
   After the de-identification task is started, click the Task Execution Status tab. On the Task Execution Status tab, click Search to update the running progress and status of tasks.

   Note If you do not click Search, the created de-identification task may not be found on the Task Execution Status tab.

   You can check whether the de-identification task is run in the Status column. For more information about the failure causes, see Troubleshoot failures to run de-identification tasks.

   

Specify a partition

Query de-identification tasks

On the Static Desensitization page, you can view the details of de-identification tasks that you have created. In the task list, click the ID of a de-identification task in the Task ID column to view the task details.

To modify a de-identification task that has been created, enter the ID of the task in the search bar and click Desensitization Task Search. The de-identification task appears. Then, click Modify in the Actions column to modify the de-identification task.

Troubleshoot failures to run de-identification tasks