Mask Sensitive Data with DSL Functions - Simple Log Service - Alibaba Cloud - Simple Log Service

Masking techniques

Log Service provides five masking techniques through its domain-specific language (DSL):

Technique	Key function	Best for
Regex replacement	regex_replace	Phone numbers, bank cards, emails, AccessKey pairs, IPs, ID card numbers
Grok capture	grok	Structured patterns (IP, ID) using built-in Grok patterns
Base64 transcoding	base64_encoding	Reversible encoding for URLs and arbitrary values
MD5 encoding	md5_encoding	Non-reversible hashing for order numbers and IDs
String mapping	str_translate	Character-level substitution for arbitrary strings

For reference documentation on these functions, see Regular expression functions, Grok function, and Encoding and decoding functions.

Scenario 1: Mask mobile phone numbers

Use regex_replace to replace the middle digits of a phone number with asterisks, preserving the first three and last four digits.

DSL orchestration

e_set(
    "sec_iphone",
    regex_replace(v("iphone"), r"(\d{0,3})\d{4}(\d{4})", replace=r"\1****\2"),
)

Input and output

Raw log entry
```
iphone: 13900001234
```

Result

iphone: 13900001234
sec_iphone: 139****1234

Scenario 2: Mask bank card information

Use regex_replace to retain only the last four digits of bank card and credit card numbers.

DSL orchestration

e_set(
    "bank_number",
    regex_replace(
        v("content"), r"([1-9]{1})(\d{14}|\d{13}|\d{11})(\d{4})", replace=r"****\3"
    ),
)

Input and output

Raw log entry

content: bank number is 491648411333978312 and credit card number is 4916484113339780

Result

content: bank number is 491648411333978312 and credit card number is 4916484113339780 
bank_number: bank number is ****8312 and credit card number is ****9780

Scenario 3: Mask email addresses

Use regex_replace to replace the local part of an email address (before the @ symbol) with asterisks.

DSL orchestration

e_set(
    "email_encrypt",
    regex_replace(
        v("content"),
        r"[A-Za-z\d]+([-_.][A-Za-z\d]+)*(@([A-Za-z\d]+[-.])+[A-Za-z\d]{2,4})",
        replace=r"****\2",
    ),
)

Input and output

Raw log entry
```
content: email is username@example.com
```

Result

content: email is username@example.com
email_encrypt: email is ****@example.com

Scenario 4: Mask AccessKey pairs

Use regex_replace to redact AccessKey IDs (30 characters) and AccessKey Secrets (16 characters), keeping only the first four characters visible.

DSL orchestration

e_set(
    "akid_encrypt",
    regex_replace(
        v("content"),
        r"([a-zA-Z0-9]{4})(([a-zA-Z0-9]{26})|([a-zA-Z0-9]{12}))",
        replace=r"\1****",
    ),
)

Input and output

Raw log entry

content: ak id is <testAccessKey ID> and ak key is <testAccessKey Secret>

Result

content: ak id is <testAccessKey ID> and ak key is <testAccessKey Secret>
akid_encrypt: ak id is rDhc**** and ak key is XQr1****

Scenario 5: Mask IP addresses

Combine grok (to match the IP pattern) with regex_replace (to replace the matched value) — no manual regex required.

DSL orchestration

e_set("ip_encrypt",regex_replace(v('content'), grok('(%{IP})'), replace=r"****"))

Input and output

Raw log entry
```
content: ip is 192.0.2.10
```

Result

content: ip is 192.0.2.10
ip_encrypt: ip is ****

Scenario 6: Mask ID card numbers

Combine grok with the built-in CHINAID pattern and regex_replace to mask all but the first six digits of an 18-digit ID card number.

DSL orchestration

e_set(
    "id_encrypt", regex_replace(v("content"), grok("(%{CHINAID})"), replace=r"\1****")
)

Input and output

Raw log entry
```
content: Id card is 111222190002309999
```

Result

content: Id card is 111222190002309999
id_encrypt: Id card is 111222****

Scenario 7: Mask URLs

Use base64_encoding to transcode a URL into a Base64 string. This produces a reversible encoding — decode the value with base64_decoding when you need to retrieve the original URL.

DSL orchestration

e_set("base64_url",base64_encoding(v("url")))

Input and output

Raw log entry

url: https://www.aliyun.com/sls?logstore

Result

url: https://www.aliyun.com/sls?logstore
base64_url: aHR0cHM6Ly93d3cuYWxpeXVuLmNvbS9zbHM/bG9nc3RvcmU=

Note

To decode the value of the base64_url field, use the base64_decoding(v("base64_url")) function.

Scenario 8: Mask order numbers

Use md5_encoding to replace an order number with its MD5 hash. Unlike Base64, MD5 is non-reversible, so the original order number cannot be recovered from the stored log.

DSL orchestration

e_set("md5_orderId",md5_encoding(v("orderId")))

Input and output

Raw log entry
```
orderId: 20210101123456
```

Result

orderId: 20210101123456
md5_orderId: 852751f9aa48303a5691b0d020e52a0a

Scenario 9: Mask strings

Use str_translate to define a character-to-character mapping that substitutes specific characters throughout a string value.

DSL orchestration

e_set("data_translate", str_translate(v("data"),"aeiou","12345"))

Input and output

Raw log entry
```
data: message level is info_
```

Result

data: message level is info
data_translate: m2ss1g2 l2v2l 3s 3nf4