Mask sensitive data - Simple Log Service - Alibaba Cloud Documentation Center

When you transform, ship, or use data, you can configure data masking rules to reduce the exposure of sensitive data. This way, you can mitigate the risk of data leaks in an efficient manner. This topic describes how to use functions to mask sensitive data in various scenarios.

Background information

Data masking is commonly used to mask sensitive data such as mobile phone numbers, bank card numbers, email addresses, IP addresses, AccessKey pairs, ID numbers, URLs, order numbers, and strings. When you transform data in the Log Service console, you can use one of the following solutions to mask data: replacement by regular expressions (key function regex_replace), Base64 transcoding (key function base64_encoding), MD5 encoding (key function md5_encoding), str_translate mapping (key function str_translate), Grok capture (key function grok). For more information, see Regular expression functions, Grok function, and Encoding and decoding functions.

Scenario 1: Mask mobile phone numbers

Solution
To mask the mobile phone numbers in log entries, you can use the regex_replace function.

Example

Raw log entry
```
iphone: 13900001234
```

Domain-specific language (DSL) orchestration

e_set(
    "sec_iphone",
    regex_replace(v("iphone"), r"(\d{0,3})\d{4}(\d{4})", replace=r"\1****\2"),
)

Result:

iphone: 13900001234
sec_iphone: 139****1234

Scenario 2: Mask bank card information

Solution
To mask the bank card information in log entries, for example, bank card numbers, you can use the regex_replace function.

Example

Raw log entry

content: bank number is 491648411333978312 and credit card number is 4916484113339780

DSL orchestration

e_set(
    "bank_number",
    regex_replace(
        v("content"), r"([1-9]{1})(\d{14}|\d{13}|\d{11})(\d{4})", replace=r"****\3"
    ),
)

Result

content: bank number is 491648411333978312 and credit card number is 4916484113339780 
bank_number: bank number is ****8312 and credit card number is ****9780

Scenario 3: Mask email addresses

Solution
To mask the email addresses in log entries, you can use the regex_replace function.

Example

Raw log entry
```
content: email is username@example.com
```

DSL orchestration

e_set(
    "email_encrypt",
    regex_replace(
        v("content"),
        r"[A-Za-z\d]+([-_.][A-Za-z\d]+)*(@([A-Za-z\d]+[-.])+[A-Za-z\d]{2,4})",
        replace=r"****\2",
    ),
)

Result

content: email is username@example.com
email_encrypt: email is ****@example.com

Scenario 4: Mask AccessKey pairs

Solution
To mask the AccessKey pairs in log entries, you can use the regex_replace function.

Example

Raw log entry

content: ak id is <testAccessKey ID> and ak key is <testAccessKey Secret>

DSL orchestration

e_set(
    "akid_encrypt",
    regex_replace(
        v("content"),
        r"([a-zA-Z0-9]{4})(([a-zA-Z0-9]{26})|([a-zA-Z0-9]{12}))",
        replace=r"\1****",
    ),
)

Result

content: ak id is <testAccessKey ID> and ak key is <testAccessKey Secret>
akid_encrypt: ak id is rDhc**** and ak key is XQr1****

Scenario 5: Mask IP addresses

Solution
To extract and mask the IP addresses in log entries, you can use the regex_replace function and the grok function.

Example

Raw log entry
```
content: ip is 192.0.2.10
```

DSL orchestration

e_set("ip_encrypt",regex_replace(v('content'), grok('(%{IP})'), replace=r"****"))

Result

content: ip is 192.0.2.10
ip_encrypt: ip is ****

Scenario 6: Mask ID card numbers

Solution
To mask the ID card numbers in log entries, you can use the regex_replace function and the grok function.

Example

Raw log entry
```
content: Id card is 111222190002309999
```

DSL orchestration

e_set(
    "id_encrypt", regex_replace(v("content"), grok("(%{CHINAID})"), replace=r"\1****")
)

Result

content: Id card is 111222190002309999
id_encrypt: Id card is 111222****

Scenario 7: Mask URLs

Solution
To mask the URLs in log entries, you can convert the URLs to plaintext and then use the Base64 encoding and decoding functions to transcode the URLs.

Example

Raw log entry

url: https://www.aliyun.com/sls?logstore

DSL orchestration

e_set("base64_url",base64_encoding(v("url")))

Result
```
url: https://www.aliyun.com/sls?logstore
base64_url: aHR0cHM6Ly93d3cuYWxpeXVuLmNvbS9zbHM/bG9nc3RvcmU=
```
Note To decode the value of the base64_url field, you can use the base64_decoding(v("base64_url") function.

Scenario 8: Mask order numbers

Solution
To mask the order numbers in log entries and prevent other users from decoding the order numbers, you can use the MD5 encoding function to encode the order numbers.

Example

Raw log entry
```
orderId: 20210101123456
```

DSL orchestration

e_set("md5_orderId",md5_encoding(v("orderId")))

Result

orderId: 20210101123456
md5_orderId: 852751f9aa48303a5691b0d020e52a0a

Scenario 9: Mask strings

Solution
To mask the strings in log entries, you can use the str_translate function to configure mapping rules for the strings.

Example

Raw log entry
```
data: message level is info_
```

DSL orchestration

e_set("data_translate", str_translate(v("data"),"aeiou","12345"))

Result

data: message level is info
data_translate: m2ss1g2 l2v2l 3s 3nf4