This topic describes how to mask sensitive log data during data transformation.

Use a regular expression to mask sensitive log data

The following example demonstrates how to use regular expression syntax of domain-specific language (DSL) to mask cell phone numbers.
  • Raw log entry
    iphone: 15036171958
  • DSL orchestration
    e_set("sec_iphone",regex_replace(v('iphone'), r"(\d{0,3})\d{4}(\d{4})", replace=r"\1****\2"))
  • Result
    iphone: 15036171958
    sec_iphone: 150****1958

Use Base64 encoding to mask sensitive log data

The following example demonstrates how to use Base64 encoding to mask sensitive data by transforming the data into plaintext.
  • Raw log entry
    url: https://www.aliyun.com/sls?logstore
  • DSL orchestration
    e_set("base64_url",base64_encoding(v("url")))
  • Result
    url: https://www.aliyun.com/sls?logstore
    base64_url: aHR0cHM6Ly93d3cuYWxpeXVuLmNvbS9zbHM/bG9nc3RvcmU=
Note To decode base64_url, you can use the base64_decoding(v("base64_url") function.

Use MD5 encoding to mask sensitive log data

The following example demonstrates how to use MD5 encoding to mask sensitive log data and prevent unauthorized decoding.
  • Raw log entry
    orderId: 15121412314
  • DSL orchestration
    e_set("md5_orderId",md5_encoding(v("orderId")))
  • Result
    orderId: 15121412314
    md5_orderId: 852751f9aa48303a5691b0d020e52a0a

Use the str_translate function to mask sensitive log data

The str_translate function returns a string where each character is mapped to its corresponding character in the specified translation table.
  • Raw log entry
    data: message level is info
  • DSL orchestration
    e_set("data_translate", str_translate(v("data"),"aeiou","12345"))
  • Result
    data: message level is info
    data_translate: m2ss1g2 l2v2l 3s 3nf4

Mask debit card and credit card numbers

The following example demonstrates how to mask debit card and credit card numbers in a log entry.
  • Raw log entry
    content: bank number is 491648411333978312 and credit card number is 4916484113339780
  • DSL orchestration
    e_set("bank_number",regex_replace(v('content'), r'([1-9]{1})(\d{11}|\d{13}|\d{14})(\d{4})', replace=r"****\3"))
  • Result
    content: bank number is 491648411333978312 and credit card number is 4916484113339780 
    bank_number: bank number is ****978312 and credit card number is ****9780

Mask an email address

The following example demonstrates how to mask an email address in a log entry.
  • Raw log entry
    content: email is twiss2345@aliyun.com
  • DSL orchestration
    e_set("email_encrypt",regex_replace(v('content'), r'[A-Za-z\d]+([-_.][ A-Za-z\d]+)*(@([A-Za-z\d]+[-.]) +[A-Za-z\d]{2,4})', replace=r"****\2"))
  • Result
    content: email is twiss2345@aliyun.com
    email_encrypt: email is ****@aliyun.com

Mask an AccessKey pair

The following example demonstrates how to mask an AccessKey pair in a log entry.
  • Raw log entry
    content: ak id is rDhc9qxjhIhlBiyphP7buo5yg5h6Eq and ak key is XQr1EPtfnlZLYlQc
  • DSL orchestration
    e_set("akid_encrypt",regex_replace(v('content'), r'([a-zA-Z0-9]{4})(([a-zA-Z0-9]{26})|([a-zA-Z0-9]{12}))', replace=r"\1****"))
  • Result
    content: ak id is rDhc9qxjhIhlBiyphP7buo5yg5h6Eq and ak key is XQr1EPtfnlZLYlQc
    akid_encrypt: ak id is rDhc**** and ak key is XQr1****

Mask an IP address

The following example demonstrates how to mask an IP address in a log entry.
  • Raw log entry
    content: ip is 192.168.1.1
  • DSL orchestration
    e_set("ip_encrypt",regex_replace(v('content'), grok('(%{IP})'), replace=r"****"))
  • Result
    content: ip is 192.168.1.1
    ip_encrypt: ip is ****

Mask an ID card number

The following example demonstrates how to mask an ID card number in a log entry.
  • Raw log entry
    content: Id card is 11010519491231002X
  • DSL orchestration
    e_set("id_encrypt",regex_replace(v('id_card'content'), grok('(%{CHINAID})'), replace=r"\1****"))
  • Result
    content: Id card is 11010519491231002X
    id_encrypt: idcard is 110105****