You can encrypt sensitive fields from an upstream source by selecting an encryption algorithm and a key to protect the data. This topic describes how to configure the Encryption transform component.
Prerequisites
An offline single pipeline has been created. For more information, see Create an integration node using a single pipeline.
Procedure
From the top menu bar on the Dataphin homepage, choose Develop > Data Integration.
From the top menu bar of the integration page, select a Project. If you are in Dev-Prod mode, also select an environment.
In the navigation pane on the left, click Batch Pipeline. On the Batch Pipeline page, click the target offline pipeline to open its configuration page.
In the upper-right corner of the page, click Component Library to open the Component Library pane.
In the Component Library pane, select Transform from the navigation pane on the left, and then drag the Encryption component from the list on the right to the canvas.
Click and drag the
icon on the target input component to connect it to the Encryption component.Click the
icon on the card for the Encryption component to open the Field Encryption Configuration dialog box.
In the Field Encryption Configuration dialog box, go to the Select Fields step and select the fields to sync from the upstream component. If a field name contains a table name, the table name is also displayed.
Click Next.
ImportantThe selected fields are encrypted and passed to the downstream component. Other fields are passed to the downstream component with their original values.
Because encryption consumes extra resources, encrypt only sensitive data.
After a field is encrypted, its data type is converted to String. When you use a decryption component to decrypt the field, you must specify the Output Field Type.
In the Encryption Configuration step, you can configure the encryption parameters.
The required configuration varies depending on the encryption algorithm. Select an encryption algorithm and configure its parameters. For more information about encryption algorithms, see Encryption and decryption algorithms.
AES, DES, 3DES, SM4, SM2, and RSA encryption algorithms.
Parameter
Description
Key
The key used for encryption. You can select any key that has been created for the selected encryption algorithm. For more information, see Manage keys.
You can use keys for which you have permissions. To use a key for which you do not have permissions, you must request permissions. For more information, see Request, renew, and return key permissions.
Advanced Configuration
The AES, DES, 3DES, SM4, and RSA encryption algorithms support advanced configurations. You can use advanced configurations to specify parameters such as the output encoding. In most cases, the default configurations are sufficient.
To exchange Dataphin data with external data, ensure that the advanced configurations are consistent. Different encryption algorithms support different advanced configuration items. The details are as follows:
Encryption Mode: Select an encryption mode based on your business scenario. Different algorithms support different encryption modes. You can select ECB, CBC, CFB, CTR, or OFB. ECB does not have an initialization vector (IV) configuration. The encryption and decryption configurations must be consistent.
Padding: You can select NoPadding, PKCS5Padding, or PKCS7Padding. The encryption and decryption configurations must be consistent. Different encryption algorithms support different padding modes. The actual options on the page prevail.
IV: An IV is an offset. Different IVs produce different encrypted strings. The IV must be a 16-digit number. The same IV must be used for encryption and decryption.
Encoding Format: The output can be in Base64 or Hex format.
For more information about the advanced settings for encryption algorithms, see Advanced settings for encryption and decryption algorithms.
Note: If you select SM4 as the encryption algorithm and the output destination is AnalyticDB for PostgreSQL, select the Output destination is AnalyticDB PostgreSQL option. Otherwise, the encrypted data cannot be decrypted directly in AnalyticDB for PostgreSQL.
Format-Preserving Encryption (FPE) FF1 algorithm.
Parameter
Description
Encryption Range
When the encryption algorithm is Format-Preserving Encryption (FPE) FF1, you can configure the Encryption Range. The options are Specify Range and All.
Specify Range: Defines the start and end positions of the characters to be encrypted. The decryption configuration must be consistent with the encryption configuration. Otherwise, the decrypted data may not match the raw data. You can add a range by sliding or by direct input. You can add a maximum of 10 ranges.
ImportantEach digit, English letter, Chinese character, and symbol is counted as 1 character. For example, in the string 'test', the 3th character is 's'.
Add by sliding: You can add a range by clicking or sliding. After you slide the slider to define a range, click OK in the dialog box that appears to add the range. If the range to be encrypted exceeds 24 characters, add the range by direct input.
Add by direct input: In the input fields, enter the Start Position, End Position, Range Length, and Encryption Alphabet. You can also View Encryption Alphabet, Edit Custom Encryption Alphabet, and Delete an added range.
Start Position: The start position of the encryption range.
Range Length: Enter a positive integer
>=1or select a hyphen (-). A hyphen (-) indicates from the current start position to the end of the string.End Position: The end position of the encryption range. Enter a positive integer
>=1or select End.Encryption Alphabet: The encryption alphabet for the encryption range. The following encryption alphabets are supported:
Built-in: Includes Digits, Uppercase letters, Lowercase letters, Digits + Uppercase letters, Digits + Lowercase letters, Digits + Letters, and Special symbols.
Custom: In the Custom Encryption Alphabet dialog box, enter the encryption characters. Each encryption character must be a single character. Spaces are not supported. Duplicate characters are not supported. If you enter duplicate characters, the system automatically removes them. You can enter a maximum of 10,000 encryption characters. You can select the check boxes for space, line feed (\n), carriage return (\r), or tab character (\t) to use them as encryption characters. If you do not select any of these check boxes, characters such as
\nare identified as separate\andncharacters.
View Encryption Alphabet: For a Built-in encryption alphabet, click
to view its encryption characters.Edit Custom Encryption Alphabet: For a Custom encryption alphabet, click
to edit its encryption characters.Delete: Click
to delete the current range.
All: Encrypts all characters in the field.
Key
The key used for encryption. You can select any key that has been created for the selected encryption algorithm. For more information, see Manage keys.
You can use keys for which you have permissions. To use a key for which you do not have permissions, you must request permissions. For more information, see Request, renew, and return key permissions.
Compatibility Issues
If the plaintext does not meet the algorithm requirements, the key does not match, or the encoding format is inconsistent, the plaintext data is processed based on the selected policy. The options are Return Null and Return Plaintext.
Click OK to configure the Field Encryption Configuration component.