This topic describes the syntax and parameters of encoding and decoding functions. This topic also provides examples on how to use the functions.

Functions

CategorySubcategoryFunctionDescription
Encoding and decodingStringstr_encodeEncodes data.
str_decodeDecodes data.
Base64base64_encodingEncodes data by using the Base64 algorithm.
base64_decodingDecodes data by using the Base64 algorithm.
HTMLhtml_encodingEncodes data in the HTML format.
html_decodingDecodes HTML-encoded data.
URLurl_encodingPerforms URL encoding on data.
url_decodingPerforms URL decoding on data.
Protobufprotobuf_decodingParses data into the JSON format by using a specified Protobuf template.
JSON Web Token (JWT)jwt_encodingEncodes JSON data based on the JWT standard.
jwt_decodingDecodes data to raw JSON data based on the JWT standard.
Hashidshashids_encodingEncodes data by using the Hashids library.
hashids_decodingDecodes data that is encoded by using the Hashids library.
Compression and decompressionGzipgzip_compressCompresses and encodes data.
gzip_decompress Decompresses compressed data.
Zlibzlib_compressCompresses and encodes data.
zlib_decompressDecompresses compressed data.
Encryption and decryptionAdvanced Encryption Standard (AES)aes_encryptEncrypts data by using the AES algorithm.
aes_decryptDecrypts data by using the AES algorithm.
Hash (digest)MD5md5_encodingEncodes data by using the MD5 algorithm.
SHA1sha1_encodingEncodes data by using the SHA1 algorithm.
Cyclic redundancy check (CRC)crc32_encodingCalculates a CRC code for data.

str_encode

The function is used to encode a string by using a specified encoding format.
  • Syntax

    str_encode(value, "utf8", errors="ignore")
  • Parameters

    ParameterTypeRequiredDescription
    valueArbitrary (automatically converted to the string type)YesThe value that you want to encode.
    encodingStringNoThe encoding format. Default value: utf8. ASCII is supported.
    errorsStringNoThe method that is used to process characters if some characters cannot be recognized based on the encoding format. Valid values:
    • ignore: No characters are encoded. This is the default value.
    • strict: An error is reported and the data of the log is discarded.
    • replace: The unrecognized characters are replaced with question marks (?).
    • xmlcharrefreplace: The unrecognized characters are replaced with XML characters.
  • Response

    An encoded string is returned.

  • Examples

    • Example 1
      • Raw log:
        test: asewds
      • Transformation rule:
        e_set("f1", str_decode(str_encode("hello", "utf8"), "utf8"))
      • Result:
        test: asewds
        f1: hello
    • Example 2
      • Raw log:
        f2: test test data
      • Transformation rule:
        e_set("f1", str_encode(v("f2"), "ascii", errors="ignore"))
      • Result:
        f1:test 
        f2:test test data
    • Example 3
      • Raw log:
        f2: test test data
      • Transformation rule:
        e_set("f1", str_encode(v("f2"), "ascii", errors="strict"))
      • Result:

        An error is reported during execution.

    • Example 4
      • Raw log:
        f2: test test data
      • Transformation rule:
        e_set("f1", str_encode(v("f2"), "ascii", errors="replace"))
      • Result:
        f1:test ????
        f2:test test data
    • Example 5
      • Raw log:
        f2: test test data
      • Transformation rule:
        e_set("f1", str_encode(v("f2"), "ascii", errors="xmlcharrefreplace"))
      • Result:
        f1:test 测试数据
        f2:test test data

str_decode

The function is used to decode an input value by using a specified encoding format.
  • Syntax

    str_decode(value, "utf8", errors="ignore")
    Note The str_decode function can process only the data of the byte data type.
  • Parameters

    ParameterTypeRequiredDescription
    valueArbitrary (automatically converted to the string type)YesThe value that you want to decode.
    encodingArbitrary (automatically converted to the string type)NoThe encoding format. Default value: utf8. ASCII is supported.
    errorsArbitrary (automatically converted to the string type)NoThe method that is used to process characters if some characters cannot be recognized based on the encoding format. Valid values:
    • ignore: No characters are decoded. This is the default value.
    • strict: An error is reported and the data of the log is discarded.
    • replace: The unrecognized characters are replaced with question marks (?).
    • xmlcharrefreplace: The unrecognized characters are replaced with XML characters.
  • Response

    A decoded value is returned.

  • Examples

    • Raw log:
      test: asewds
    • Transformation rule:
      e_set("encoding", str_decode(b'\xe4\xbd\xa0\xe5\xa5\xbd', "utf8", 'strict'))
    • Result:
      test: asewds
      encoding: hello

base64_encoding

The base64_encoding function encodes data by using the Base64 algorithm.
  • Syntax

    base64_encoding(value, format=None)
  • Parameters

    ParameterTypeRequiredDescription
    valueStringYesThe value that you want to encode.
    formatStringNoThe Base64 encoding scheme. Valid values: RFC3548 and RFC4648. Default value: RFC3548.
  • Response

    An encoded string is returned.

  • Examples

    • Raw log
      str_en : data to be encoded
    • Transformation rule
      e_set("str_base64",base64_encoding(v("str_en")))
    • Result
      str_en : data to be encoded
      str_base64 : ZGF0YSB0byBiZSBlbmNvZGVk

base64_decoding

The base64_decoding function decodes data by using the Base64 algorithm.
  • Syntax

    base64_decoding(value, format=None)
  • Parameters

    ParameterTypeRequiredDescription
    valueStringYesThe value that you want to decode.
    formatStringNoThe Base64 decoding scheme. Valid values: RFC3548 and RFC4648. Default value: RFC3548.
    Note The Base64 decoding scheme in RFC 4648 uses equal signs (=) to pad a decoded value to a multiple of 4 bytes.
  • Response

    A decoded string is returned.

  • Examples

    • Raw log
      str_de: ZGF0YSB0byBiZSBlbmNvZGVk
    • Transformation rule
      e_set("str_de_base64",base64_decoding(v("str_de")))
    • Result
      str_de: ZGF0YSB0byBiZSBlbmNvZGVk
      str_de_base64: data to be encoded

html_encoding

The html_encoding function encodes data in the HTML format.
  • Syntax

    html_encoding(value)
  • Parameters

    ParameterTypeRequiredDescription
    valueStringYesThe value that you want to encode.
  • Response

    An encoded string is returned.

  • Examples

    • Raw log
      str : <img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />
    • Transformation rule
      e_set("str_html_en",html_encoding(v("str")))
    • Result
      str : <img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />
      str_html_en : &lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;

html_decoding

The html_decoding function decodes HTML-encoded data.
  • Syntax

    html_decoding(value)
  • Parameters

    ParameterTypeRequiredDescription
    valueStringYesThe value that you want to decode.
  • Response

    A decoded string is returned.

  • Examples

    • Raw log
      str : &lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;
    • Transformation rule
      e_set("str_html_de",html_decoding(v("str")))
    • Result
      str : &lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;
      str_html_de : <img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />

url_encoding

The url_encoding function performs URL encoding on data.
  • Syntax

    url_encoding(value, plus=False)
  • Parameters

    ParameterTypeRequiredDescription
    valueStringYesThe value that you want to encode.
    plusBooleanNoSpecifies whether to convert spaces to plus signs (+). Default value: False. Valid values:
    • True
    • False
  • Response

    An encoded string is returned.

  • Examples

    • Example 1: Perform URL encoding on data.
      • Raw log
        content : https://www.example.org/hello/asdah
      • Transformation rule
        e_set("url",url_encoding(v("content")))
      • Result
        content : https://www.example.org/hello/asdah
        url: https%3A%2F%www.example.org%2FHello%2Fasdah
    • Example 2: Perform URL encoding on data. Leave the plus parameter unconfigured to ensure that spaces are not converted to plus signs (+).
      • Raw log
        content : 1 2+3:4
      • Transformation rule
        e_set("URL",url_encoding(v("content")))
      • Result
        content:1 2+3:4
        URL:1%202%2B3%3A4
    • Example 3: Perform URL encoding on data. Set the plus parameter to True to convert spaces to plus signs (+).
      • Raw log
        content : 1 2+3:4
      • Transformation rule
        e_set("URL", url_encoding(v("content"), plus=True))
      • Result
        content:1 2+3:4
        URL:1+2%2B3%3A4

url_decoding

The url_decoding function performs URL decoding on data.
  • Syntax

    url_decoding(value, plus=False)
  • Parameters

    ParameterTypeRequiredDescription
    valueStringYesThe value that you want to decode.
    plusBooleanNoSpecifies whether to convert plus signs (+) to spaces. Default value: False. Valid values:
    • True
    • False
  • Response

    A decoded string is returned.

  • Examples

    • Example 1: Perform URL decoding on data.
      • Raw log
        content : https%3A%2F%www.example.org%2FHello%2Fasdah
      • Transformation rule
        e_set("URL",url_decoding(v("content")))
      • Result
        content : https%3A%2F%www.example.org%2FHello%2Fasdah
        URL : https://www.example.org/hello/asdah
    • Example 2: Perform URL decoding on data. Leave the plus parameter unconfigured to ensure that plus signs (+) are not converted to spaces.
      • Raw log
        content : /answer?event_date=2022-06-30+09%3A06%3A53%20123
      • Transformation rule
        e_set("URL",url_decoding(v("content")))
      • Result
        content:/answer?event_date=2022-06-30+09%3A06%3A53%20123
        URL:/answer?event_date=2022-06-30+09:06:53 123
    • Example 3: Perform URL decoding on data. Set the plus parameter to True to convert plus signs (+) to spaces.
      • Raw log
        content : /answer?event_date=2022-06-30+09%3A06%3A53%20123
      • Transformation rule
        e_set("URL",url_decoding(v("content"),plus=True))
      • Result
        content:/answer?event_date=2022-06-30+09%3A06%3A53%20123
        URL:/answer?event_date=2022-06-30 09:06:53 123

protobuf_decoding

The protobuf_decoding function parses data into the JSON format by using a specified Protobuf template.

  • Syntax

    protobuf_decoding(data, protocol, input_format="bytes", message_name=None)
  • Parameters

    ParameterTypeRequiredDescription
    dataStringYes The Protobuf-serialized data.
    protocolStringYesThe Protobuf template.
    input_formatStringNoThe data type. Valid values:
    • hex: hexadecimal
    • bytes: bytes
    • raw: bytes
    • base64: Base64-encoded
    message_nameStringNo The name of the message that you want to obtain.
  • Response

    If you do not configure message_name, the function returns a JSON list. If you configure message_name, the function returns a JSON object.

  • Examples

    • Example 1: Parse the Protobuf data that is encoded in Base64 into the JSON format by using a template.
      • Raw log
        {
          "protocol": 'syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}  ',
          "data": "Cgl4aWFvIG1pbmcQARoOMTIzMTIzQDEyMy5jb20="
        }
      • Transformation rule
        e_set("data", protobuf_decoding(v("data"), v("protocol"), "base64", "Person"))
      • Result
        data:{
          "name": "xiao ming",
          "id": 1,
          "email": "123123@123.com"
        }
        protocol:syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}
    • Example 2: Parse the Protobuf data into the JSON format by obtaining a template from advanced parameters.
      • Raw log
        {
            "data": "Cgl4aWFvIG1pbmcQARoOMTIzMTIzQDEyMy5jb20="
        }
      • Transformation rule
        e_set("data",
              protobuf_decoding(
                  v("data"),
                  res_local("protocol"),
                  "base64",
                  "Person"
              )
        )
      • Advanced Parameter Settings

        In the Create Data Transformation Job panel, configure advanced parameters. Set key to protocol and value to syntax = "proto3"; message Person { string name = 1; int32 id = 2; string email = 3;}. For more information, see Create a data transformation job.

        protobuf_decoding
      • Result
        data:{
          "name": "xiao ming",
          "id": 1,
          "email": "123123@123.com"
        }
    • Example 3: Parse the Protobuf data into the JSON format by obtaining a template from an Object Storage Service (OSS) file.
      • Raw log
        {
            "data": "Cgl4aWFvIG1pbmcQARoOMTIzMTIzQDEyMy5jb20="
        }
      • OSS file content
        syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}
      • Transformation rule
        e_set(
            "protocol",
            res_oss_file(
                endpoint="http://oss-cn-chengdu.aliyuncs.com",
                ak_id="your_ak_id",
                ak_key="your_ak_key",
                bucket="test-protobuf",
                file="test.json",
            ),
        )
        e_set("data", protobuf_decoding(v("data"), v("protocol"), "base64", "Person"))
      • Result
        data:{
          "name": "xiao ming",
          "id": 1,
          "email": "123123@123.com"
        }
        protocol:syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}

jwt_encoding

The jwt_encoding function encodes JSON data based on the JWT standard.
Note JWT is an open standard that defines a compact and self-contained method to transmit information between parties as JSON objects in a secure manner. For more information, see RFC 7519. The information is digitally signed and can be verified and trusted. JWTs can be signed by using a secret and the HMAC algorithm. JWTs can also be signed by using a public/private key pair and the RSA or ECDSA algorithm. For more information, see Introduction to JSON Web Tokens.
  • Syntax

    jwt_encoding(payload, key, algorithms="HS256", headers=None)
  • Parameters

    ParameterTypeRequiredDescription
    payloadJSONYesThe value that you want to encode. The value must be in the JSON format. The JWT standard defines the following fields. You can also define custom fields.
    • iss: issuer
    • exp: expiration time
    • sub: subject
    • aud: audience
    • nbf: Not Before
    • iat: Issued At
    • jti: JWT ID
    The following sample code provides an example of raw JSON data:
    {
        "iss": "localhost",
        "sub": "name",
        "aud": "user",
        "address": {
            "street": "street number",
            "city": "hangzhou",
            "country": "china"
        }
    }
    keyStringYesThe key that you want to use to encrypt a JWT. The key varies based on the encryption algorithm that you select.
    • Asymmetric encryption algorithm: PEM-encoded private key
    • Symmetric encryption algorithm: raw key
    algorithmsStringNoThe algorithm that you want to use to sign a JWT. Default value: HS256. Valid values: HS256, HS384, HS512, ES256, ES256K, ES384, ES512, RS256, RS384, RS512, PS256, PS384, PS512, and EdDSA.
    headersJSONNoThe information about the JWT header. Default value:
    {
        "typ": "JWT",
        "alg": "HS256"
    }
  • Response

    An encoded string is returned.

  • Examples

    • Example 1: Encode the value of the data field.
      • Raw log
        data: {"some": "payload"}
      • Transformation rule
        e_set("jwt_token", jwt_encoding(v("data"),"secret", algorithms="HS256"))
      • Result
        data:{"some": "payload"}
        jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Joh1R2dYzkRvDkqv3sygm5YyK8Gi4ShZqbhK2gxcs2U
    • Example 2: Use an asymmetric encryption algorithm to encode the value of the data field.
      • Raw log
        data: {"some": "payload"}
      • Transformation rule
        e_set(
            "jwt_token",
            jwt_encoding(
                v("data"),
                "-----BEGIN RSA PRIVATE KEY-----\nMIICYQIBAAKBgQC1iaUr5cgShCn0127+w14XN297q/IviaewIeIJsKTZF1hBPsLn\nNIPsnqtQ9DFbjIyqyZvdmQFDJCSLpXaVc648yepnFDKbOfs3r+K4Crnpo2SuZmNV\nNDVEi4pECXlBz810zJY1wqVArM7qGAyCcRLBprwXB6wfEhk3CAP3c29+pwIDAQAB\nAoGAARo65I9arbIbxx7fz7BEDAQMK0YaDGvbltg91S07cw4PPSYybNEG1BMKm01A\nV3v9BrR+u9PIDC5WAnsYwiODqEoSyk8OwO1E2kWA6+MNclYYfVjaJeiRJ5PzCud/\niUObonptRzxuTng+u1oGuX7QwUhwGJdXVBUAtJFYwXR2qVECRQC5S+6vdFESRLSX\n7yBZVM6+49lZcdehMv0HwT17UseLvWcjeSbiogvv02HbYilrW9ZydKsixAWP5w/U\nS3L34CS811VcDQI9APrOiL7c1xg5fX8wAWv2d+e+MfZoB3ohb8671W3pmp3JVnjY\ntzhoYNNQmnmRQQWf7n3J63MQz4sYYNn0gwJEE/pl37Dw1MFnn0H/AOKt79LtKkGl\n+BFhSqbBFDzWmvBu4Fo9oQ3Lr63gzSCGSrb6JhkCIptz5hIJmOARozwdeebVozkC\nPQDwOqVmU3c/P8nB6oRiGditw1Jt1yTaSW6jkOyUc73iRngqFkIgqHGd1kWwDX4/\nWfoAyEhalY6Fh5s1COsCRQCy4b3hnQws5zz/gmGNnoyxn9N+A09ySaFtn2WkUlrR\nZ6DwoJz+n6EgjLY8z6ZQyv342iobO5zKkZHFvO9QYGqk7y8lJA==\n-----END RSA PRIVATE KEY-----\n",
                algorithms="RS256",
            ),
        )
      • Result
        data:{"some": "payload"}
        jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Ewwls5YPuJCmAR3XR2tcptOLrH83wVCzmUaUpGMzMLcPknRrIvbDmFGlNlQha-PMx0jsxt3t1oxpz7P3z3SR9o4qyWusAb99UG_Jn8oP8W0a5GKSy4UEJB0xgpVvJl5F2JaIPeUSHpV0VeS2WAsGSBBSAaOMkrc-8uie-H4J9M0
    • Example 3: Set the algorithm parameter to HS256 and configure the headers parameter to encode the value of the data field.
      • Raw log
        data: {"some": "payload"}
      • Transformation rule
        e_set(
            "jwt_token",
            jwt_encoding(
                v("data"),
                "secret",
                algorithms="HS256",
                headers={"kid": "230498151c214b788dd97f22b85410a5"},
            ),
        )                                        
      • Result
        data:{"some": "payload"}
        jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiIsImtpZCI6IjIzMDQ5ODE1MWMyMTRiNzg4ZGQ5N2YyMmI4NTQxMGE1In0.eyJzb21lIjoicGF5bG9hZCJ9.gdQ884yjlnLnIrYjfQaClE6rJC2x8v2OP2s_eXOLhZA
    • Example 4: Specify a custom JWT field for the data field and encode the value of the data field.
      • Raw log
        data: {"some": "payload", "iss": 9, "sub": "name", "nbf": 123, "iat": "22"}
      • Transformation rule
        e_set("jwt_token", jwt_encoding(v("data"),"secret"))
      • Result
        data:{"some": "payload", "iss": 9, "sub": "name", "nbf": 123, "iat": "22"}
        jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6OSwic3ViIjoibmFtZSIsIm5iZiI6MTIzLCJpYXQiOiIyMiJ9.c6MxZHdXsgASchQ1Mdqj208NO_3rNGjYnvo6c7HNAyk

jwt_decoding

The jwt_decoding function decodes data to raw JSON data based on the JWT standard.
Note JWT is an open standard that defines a compact and self-contained method to transmit information between parties as JSON objects in a secure manner. For more information, see RFC 7519. The information is digitally signed and can be verified and trusted. JWTs can be signed by using a secret and the HMAC algorithm. JWTs can also be signed by using a public/private key pair and the RSA or ECDSA algorithm. For more information, see Introduction to JSON Web Tokens.
  • Syntax

    jwt_decoding(jwt_payload, key="", algorithms=None, options=None, audience=None, issuer=None, leeway=0)
  • Parameters

    ParameterTypeRequiredDescription
    jwt_payloadStringYesThe value that you want to decode.
    keyStringYesThe key that you want to use to decrypt data. This key must be the same as the key that is used for encoding.
    algorithmsListNoThe algorithm that you want to use to sign a JWT. Example: HS256.
    optionsJSONNoThe extended options for decoding and verification. Valid values:
    • verify_signature: specifies whether to verify the encrypted JWT signature. Default value: True.
    • require: specifies the fields that must exist in the encoded data. By default, this option is empty. Example: ["exp", "iat", "nbf"].
    • verify_aud: specifies whether to check the aud field. Default value: True.
    • verify_iss: specifies whether to check the iss field. Default value: True.
    • verify_exp: specifies whether to check the exp field. Default value: True.
    • verify_iat: specifies whether to check the iat field. Default value: True.
    • verify_nbf: specifies whether to check the nbf field. Default value: True.
    Example: {"require": ["aud"], "verify_aud": True}.
    audienceString/ListNoChecks the value of the aud field.

    You must configure this parameter if the aud field is specified for encoding and "require": ["aud"] or "verify_aud": True is specified for the options parameter.

    issuerStringNoChecks the value of the iss field.

    You must configure this parameter if the iss field is specified for encoding and "require": ["iss"] or "verify_iss": True is specified for the options parameter.

    leewayfloatNoThe interval at which an expiration check is performed. Unit: seconds.
  • Response

    A decoded string is returned.

  • Examples

    • Example 1: Set the signature algorithm for decryption to HS256 to decode the value of the data field.
      • Raw log
        data:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Joh1R2dYzkRvDkqv3sygm5YyK8Gi4ShZqbhK2gxcs2U
      • Transformation rule
        e_set("data_decoded", jwt_decoding(v("data"), "secret", algorithms="HS256"))
      • Result
        data_decoded: {"some": "payload"}
    • Example 2: Use a public key to decode the value of the data field.
      • Raw log
        data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Ewwls5YPuJCmAR3XR2tcptOLrH83wVCzmUaUpGMzMLcPknRrIvbDmFGlNlQha-PMx0jsxt3t1oxpz7P3z3SR9o4qyWusAb99UG_Jn8oP8W0a5GKSy4UEJB0xgpVvJl5F2JaIPeUSHpV0VeS2WAsGSBBSAaOMkrc-8uie-H4J9M0"
      • Transformation rule
        e_set(
            "data_decoded",
            jwt_decoding(
                v("data"),
                "-----BEGIN RSA PUBLIC KEY-----\nMIGJAoGBALWJpSvlyBKEKfTXbv7DXhc3b3ur8i+Jp7Ah4gmwpNkXWEE+wuc0g+ye\nq1D0MVuMjKrJm92ZAUMkJIuldpVzrjzJ6mcUMps5+zev4rgKuemjZK5mY1U0NUSL\nikQJeUHPzXTMljXCpUCszuoYDIJxEsGmvBcHrB8SGTcIA/dzb36nAgMBAAE=\n-----END RSA PUBLIC KEY-----\n",
                algorithms="RS256",
            ),
        )                                        
      • Result
        data_decoded: {"some": "payload"}
        data:"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Ewwls5YPuJCmAR3XR2tcptOLrH83wVCzmUaUpGMzMLcPknRrIvbDmFGlNlQha-PMx0jsxt3t1oxpz7P3z3SR9o4qyWusAb99UG_Jn8oP8W0a5GKSy4UEJB0xgpVvJl5F2JaIPeUSHpV0VeS2WAsGSBBSAaOMkrc-8uie-H4J9M0"
    • Example 3: Use the default value of the options parameter to decode the value of the data field.
      • Raw log
        data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6OSwic3ViIjoibmFtZSIsIm5iZiI6MTIzLCJpYXQiOiIyMiJ9.DzvqhJd0PrTFk6eeASGZxOoDtrLBt_H3xC7CqOATRRw"
      • Transformation rule
        e_set(
            "data_decoded", jwt_decoding(v("data"), "secret", algorithms="HS256", options=None)
        )
      • Result
        data_decoded:{
                "some": "payload",
                "iss": 9,
                "sub": "name",
                "nbf": 123,
                "iat": "22"
            }
        data:"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6OSwic3ViIjoibmFtZSIsIm5iZiI6MTIzLCJpYXQiOiIyMiJ9.DzvqhJd0PrTFk6eeASGZxOoDtrLBt_H3xC7CqOATRRw"
    • Example 4: Specify a custom value for the options parameter to decode the value of the data field.
      • Raw log
        data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6Im5hbWUifQ.XwT9jqwofcdSP6olidbiYPC6CnZd36OEqCHZmGmooWM"
      • Transformation rule
        e_set(
            "data_decoded",
            jwt_decoding(
                v("data"),
                "secret",
                algorithms="HS256",
                options={"require": ["iss"], "verify_iss": True},
                issuer="name",
            ),
        )
      • Result
        data_decoded:{"some": "payload", "iss": "name"}
        data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6Im5hbWUifQ.XwT9jqwofcdSP6olidbiYPC6CnZd36OEqCHZmGmooWM"

hashids_encoding

The hashids_encoding function encodes data by using the Hashids library.

  • Syntax

    hashids_encoding(value, salt="", min_length=0)
  • Parameters

    ParameterTypeRequiredDescription
    valueNumber, list, or tuple YesThe value that you want to encode.

    If you want to specify a number for this parameter, you must specify a number that is no less than 0.

    saltStringNoThe salt that you want to add to the value. Default value: an empty string.
    min_lengthNumberNoThe minimum length of the hash value that is generated. Default value: 0.
  • Response

    An encoded string is returned.

  • Examples

    • Example 1: Use the default salt and minimum hash value length to encode a number.
      • Raw log
        content:test
      • Transformation rule
        e_set("hashid", hashids_encoding(123))
      • Result
        content:test
        hashid:Mj3
    • Example 2: Use a custom salt and minimum hash value length to encode list data.
      • Raw log
        content:test
      • Transformation rule
        e_set("hashid", hashids_encoding([123, 456], salt="test"))
      • Result
        content:test
        hashid:bpBHYO
    • Example 3: Use a custom salt and minimum hash value length to encode list data.
      • Raw log
        content:test
      • Transformation rule
        e_set("hashid", hashids_encoding([123, 456], salt="test", min_length=32))
      • Result
        content:test
        hashid:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
    • Example 4: Use a custom salt and minimum hash value length to encode tuple data.
      • Raw log
        content:test
      • Transformation rule
        e_set("hashid", hashids_encoding((123, 456), salt="test", min_length=32))
      • Result
        content:test
        hashid:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g

hashids_decoding

The hashids_decoding function decodes data that is encoded by using the Hashids library.

  • Syntax

    hashids_decoding(hashid, salt="", min_length=0)
  • Parameters

    ParameterTypeRequiredDescription
    hashidStringYesThe value that you want to decode.
    saltStringNoThe salt that you want to add to the value. Default value: an empty string.
    min_lengthNumberNoThe minimum length of the hash value that is generated. Default value: 0.
  • Response

    A decoded string is returned.

  • Examples

    • Example 1: Use the default salt and minimum hash value length to decode a number that is encoded.
      • Raw log
        content:Mj3
      • Transformation rule
        e_set("value", hashids_decoding(v("content")))
      • Result
        content:Mj3
        value:[123]
    • Example 2: Use a custom salt and the default minimum hash value length to decode list data that is encoded.
      • Raw log
        content:bpBHYO
      • Transformation rule
        e_set("value", hashids_decoding(v("content"), salt="test"))
      • Result
        content:bpBHYO
        value:[123, 456]
    • Example 3: Use a custom salt and minimum hash value length to decode list data that is encoded.
      • Raw log
        content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
      • Transformation rule
        e_set("value", hashids_decoding(v("content"), salt="test", min_length=32))
      • Result
        content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
        value:[123, 456]
    • Example 4: Use a custom salt and minimum hash value length to decode tuple data that is encoded.
      • Raw log
        content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
      • Transformation rule
        e_set("value", hashids_decoding(v("content"), salt="test", min_length=32))
      • Result
        content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
        value:[123, 456]

gzip_compress

The gzip_compress function compresses and encodes data.
  • Syntax

    gzip_compress(data, compresslevel=6, to_format="base64", encoding="utf-8")
  • Parameters

    ParameterTypeRequiredDescription
    dataStringYesThe data that you want to compress.
    compresslevelIntNoThe compression level. Valid values: 0 to 9. Default value: 6.
    • 1: The compression speed is the highest, and the compression ratio is the lowest.
    • 9: The compression speed is the lowest, and the compression ratio is the highest.
    • 0: Data is not compressed.
    to_formatStringNoThe encoding format for the compressed data. Valid values: base64 and hex.
    encodingStringNoThe encoding format for the raw data. Default value: utf-8. For more information about other encoding formats, see Standard encoding formats.
  • Response

    An encoded string is returned.

  • Examples

    • Example 1: Compress a log field and encode the field by using the Base64 algorithm.
      • Raw log
        content: I always look forward to my holidays whether I travel or stay at home.
      • Transformation rule
        e_set("base64_encode_gzip_compress",gzip_compress(v("content"),to_format="base64"))
      • Result
        content: I always look forward to my holidays whether I travel or stay at home.
        base64_encode_gzip_compress: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA
    • Example 2: Encode a log field by using the HEX encoding format.
      • Raw log
        content: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA
      • Transformation rule
        e_set("hex_encode_gzip_compress", gzip_compress(v("content"), to_format="hex"))
      • Result
        content:H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA
        hex_encode_gzip_compress:1f8b08004a478c6202ff0dc1dd0e43301800d047aa65156e3ff52f4a2ba17649c43255194d4a9f9e73527c64007e2e2426e81485c35628c1c42616535079bc6405e5d1e92ef009b59c906786a879efe1c50fb55d6c5de44cb717b2dae6d4f103f8feecbf4f88a2a441bae618c679575d9bc0e306907876806c000000

gzip_decompress

The gzip_decompress function decompresses compressed data.
  • Syntax

    gzip_decompress(data, from_format="base64", encoding="utf-8")
  • Parameters

    ParameterTypeRequiredDescription
    dataArbitraryYesThe data that you want to decompress.
    from_formatStringNoThe encoding format for the compressed data. Valid values: base64 and hex.
    encodingStringNoThe encoding format for the raw data. Default value: utf-8. For more information about other encoding formats, see Standard encoding formats.
  • Response

    Decompressed data is returned.

  • Examples

    • Example 1: Decode a log field by using the Base64 algorithm.
      • Raw log
        content: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA
      • Transformation rule
        e_set("gzip_decompress",gzip_decompress(v("content"),from_format="base64"))
      • Result
        content: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA
        gzip_decompress: I always look forward to my holidays whether I travel or stay at home.
    • Example 2: Decode a log field by using the HEX decoding format.
      • Raw log
        content:1f8b0800bff8856202ff0dc1dd0e43301800d047aa65156e3ff52f4a2ba17649c43255194d4a9f9e73527c64007e2e2426e81485c35628c1c42616535079bc6405e5d1e92ef009b59c906786a879efe1c50fb55d6c5de44cb717b2dae6d4f103f8feecbf4f88a2a441bae618c679575d9bc0e306907876806c000000
      • Transformation rule
        e_set("gzip_decompress", gzip_decompress(v("content"), from_format="hex"))
      • Result
        content:1f8b0800bff8856202ff0dc1dd0e43301800d047aa65156e3ff52f4a2ba17649c43255194d4a9f9e73527c64007e2e2426e81485c35628c1c42616535079bc6405e5d1e92ef009b59c906786a879efe1c50fb55d6c5de44cb717b2dae6d4f103f8feecbf4f88a2a441bae618c679575d9bc0e306907876806c000000
        gzip_decompress:H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA

zlib_compress

The zlib_compress function compresses and encodes data.
  • Syntax

    zlib_compress(data, compresslevel=6, to_format="base64", encoding="utf-8")
  • Parameters

    ParameterTypeRequiredDescription
    dataStringYesThe data that you want to compress.
    compresslevelIntNoThe compression level. Valid values: 0 to 9. Default value: 6.
    • 1: The compression speed is the highest, and the compression ratio is the lowest.
    • 9: The compression speed is the lowest, and the compression ratio is the highest.
    • 0: Data is not compressed.
    to_formatStringNoThe encoding format for the compressed data. Set the value to base64.
    encodingStringNoThe encoding format for the raw data. Default value: utf-8. For more information about other encoding formats, see Standard encoding formats.
  • Response

    Encoded data is returned.

  • Examples

    • Raw log
      content: I always look forward to my holidays whether I travel or stay at home.
    • Transformation rule
      e_set("zlib_compress", zlib_compress(v("content"), to_format="base64"))
    • Result
      zlib_compress: "eJwVytEJgDAMBcBV3gTu0TECjURMeZAGQ7ZXv+8GxEt6w8kbJ6MkJpJYDaNf87cyTdPAQIY86mBgpzQkv7T0eAGNshln"
      content: "I always look forward to my holidays whether I travel or stay at home."

zlib_decompress

The zlib_decompress function decompresses compressed data.
  • Syntax

    zlib_decompress(data, from_format="base64", encoding="utf-8")
  • Parameters

    ParameterTypeRequiredDescription
    dataStringYesThe data that you want to decompress.
    from_formatStringNoThe encoding format for the compressed data. Set the value to base64.
    encodingStringNoThe encoding format. Default value: utf-8. For more information about other encoding formats, see Standard encoding formats.
  • Response

    Decompressed data is returned.

  • Examples

    • Raw log
      content: "eJwVytEJgDAMBcBV3gTu0TECjURMeZAGQ7ZXv+8GxEt6w8kbJ6MkJpJYDaNf87cyTdPAQIY86mBgpzQkv7T0eAGNshln"
    • Transformation rule
      e_set("zlib_decompress", zlib_decompress(v("content"), from_format="base64"))
    • Result
      content: "eJwVytEJgDAMBcBV3gTu0TECjURMeZAGQ7ZXv+8GxEt6w8kbJ6MkJpJYDaNf87cyTdPAQIY86mBgpzQkv7T0eAGNshln"
      zlib_decompress: "I always look forward to my holidays whether I travel or stay at home."

aes_encrypt

The aes_encrypt function encrypts data by using the AES algorithm. The AES algorithm is a common symmetric encryption algorithm that is used. To improve data security, you can use the AES algorithm to encrypt data.
  • Syntax

    aes_encrypt(data, key, mode, pad_style, pad_block, input_format, input_encoding, output_format, iv)
  • Parameters

    ParameterTypeRequiredDescription
    dataStringYesThe data that you want to encrypt.
    keyStringYesThe key that you want to use to encrypt data.
    modeStringNoThe AES encryption mode.
    • CBC: Cipher Block Chaining (default)
    • ECB: Electronic Code Book
    • CFB: Cipher Feedback
    • OFB: Output Feedback
    • CTR: Counter
    • OPENPGP
    pad_styleStringNoThe padding mode. Default value: pkcs7. Valid values: iso7816, x923, and pkcs7.
    input_formatStringNoThe format of input characters. Default value: raw. Valid values:
    • raw: bytes
    • hex: hexadecimal
    • base64: Base64 encoded
    input_encodingStringNoThe encoding format for input characters. This parameter is required only if you set the input_format parameter to raw. Default value: utf-8.
    output_formatStringNoThe format of output characters. Default value: hex. Valid values:
    • raw: bytes
    • hex: hexadecimal
    • base64: Base64 encoded
    ivBytesNoThe offset that you want to use for encryption.
  • Response

    An encrypted string is returned.

  • Examples

    • Example 1
      • Raw log
        "test": "aliyuntest"
      • Transformation rule
        e_set('result',aes_encrypt(v("test"), "qwertyuiopasdfgd", iv=b"xxywosjdapdiawdk", output_format="base64"))
      • Result
        "result": "gXIqu0cBBtZHQxJBK8GLeA=="
    • Example 2
      • Raw log
        "test": "aliyuntest"
      • Transformation rule
        e_set('result',aes_encrypt(v("test"), "qwertyuiopasdfgh", iv=b"ywisnjaduaqibdqi", mode="OFB"))
      • Result
        "result":  "5cac3e9e1c42f713dc6d"

aes_decrypt

The aes_decrypt function decrypts data by using the AES algorithm.
  • Syntax

    aes_decrypt(data, key, mode, pad_style, input_format, input_encoding, output_format, iv, output_encoding)
  • Parameters

    ParameterTypeRequiredDescription
    dataStringYesThe data that you want to decrypt.
    keyStringYesThe key that you want to use to decrypt data.
    modeStringNoThe AES decryption mode.
    • CBC: Cipher Block Chaining (default)
    • ECB: Electronic Code Book
    • CFB: Cipher Feedback
    • OFB: Output Feedback
    • CTR: Counter
    • OPENPGP
    pad_styleStringNoThe padding mode. Default value: pkcs7. Valid values: iso7816, x923, and pkcs7.
    input_formatStringNoThe format of input characters. Default value: hex. Valid values:
    • raw: bytes
    • hex: hexadecimal
    • base64: Base64 encoded
    input_encodingStringNoThe encoding format for input characters. This parameter is required only if you set the input_format parameter to raw. Default value: utf-8.
    output_formatStringNoThe format of output characters. Default value: raw. Valid values:
    • raw: bytes
    • hex: hexadecimal
    • base64: Base64 encoded
    ivBytesNoThe offset that you want to use for decryption.
    output_encodingStringNoThe encoding format for output characters. Default value: None.
  • Response

    A decrypted string is returned.

  • Examples

    • Example 1
      • Raw log
        "test": "gXIqu0cBBtZHQxJBK8GLeA=="
      • Transformation rule
        e_set('result', aes_decrypt(v("test"), "qwertyuiopasdfgd", iv=b"xxywosjdapdiawdk", input_format="base64"))
      • Result
        "result": "aliyuntest"
    • Example 2
      • Raw log
        "test": "5cac3e9e1c42f713dc6d"
      • Transformation rule
           e_set('result', aes_decrypt(v("test"), "qwertyuiopasdfgh", iv=b"ywisnjaduaqibdqi", mode="OFB"))
      • Result
        "result": "aliyuntest"

md5_encoding

The md5_encoding function encodes data by using the MD5 algorithm.
  • Syntax

    md5_encoding(value, format="hex")
  • Parameters

    ParameterTypeRequiredDescription
    valueStringYesThe value that you want to encode.
    formatStringNoValid values: binary and hex. Default value: hex.
  • Response

    An encoded string is returned.

  • Examples

    • Example 1
      • Raw log
        str : GeeksforGeeks
      • Transformation rule
        e_set("str_md5_en",md5_encoding(v("str")))
      • Result
        str : GeeksforGeeks
        str_md5_en : f1e069787ece74531d112559945c6871
    • Example 2
      • Raw log
        str : GeeksforGeeks
      • Transformation rule
        e_set("str_md5_en",base64_encoding(md5_encoding(v("str"), format="binary")))
      • Result
        str : GeeksforGeeks
        str_md5_en : 8eBpeH7OdFMdESVZlFxocQ==

sha1_encoding

The sha1_encoding function encodes data by using the SHA1 algorithm.
  • Syntax

    sha1_encoding(value, format=None)
  • Parameters

    ParameterTypeRequiredDescription
    valueStringYesThe value that you want to encode.
    formatStringNoThe encoding type. Valid values: SHA1, SHA224, SHA256, SHA384, and SHA512. Default value: SHA1.
  • Response

    An encoded string is returned.

  • Examples

    • Raw log
      str : GeeksforGeeks
    • Transformation rule
      e_set("str_sha1",sha1_encoding(v("str")))
      e_set("str_sha512",sha1_encoding(v("str"),format='SHA512'))
      e_set("str_sha224",sha1_encoding(v("str"),format='SHA224'))
      e_set("str_sha384",sha1_encoding(v("str"),format='SHA384'))
      e_set("str_sha256",sha1_encoding(v("str"),format='SHA256'))
    • Result
      str : GeeksforGeeks
      str_sha1 : 4175a37afd561152fb60c305d4fa6026b7e79856
      str_sha512 : 0d8fb9370a5bf7b892be4865cdf8b658a82209624e33ed71cae353b0df254a75db63d1baa35ad99f26f1b399c31f3c666a7fc67ecef3bdcdb7d60e8ada90b722
      str_sha224 : 173994f309f727ca939bb185086cd7b36e66141c9e52ba0bdcfd145d
      str_sha384 : d1e67b8819b009ec7929933b6fc1928dd64b5df31bcde6381b9d3f90488d253240490460c0a5a1a873da8236c12ef9b3
      str_sha256 : f6071725e7ddeb434fb6b32b8ec4a2b14dd7db0d785347b2fb48f9975126178f

crc32_encoding

The crc32_encoding function calculates a CRC code for data.
  • Syntax

    crc32_encoding(data, input_format="raw", input_encoding="utf-8")
  • Parameters

    ParameterTypeRequiredDescription
    dataStringYesThe value for which you want to calculate a CRC code.
    input_formatStringNoThe format of input characters. Default value: raw. Valid values:
    • raw: bytes
    • hex: hexadecimal
    • base64: Base64 encoded
    input_encodingStringNoThe encoding format for input characters. This parameter is required only if you set the input_format parameter to raw. Default value: utf-8.
  • Response

    The CRC code for the input value is returned.

  • Examples

    • Example 1: Calculate a CRC code for the test field.
      • Raw log
        test: aliyuntest
      • Transformation rule
        e_set("str_crc32", crc32_encoding(v("test")))
      • Result
        str_crc32:1434103726
        test:aliyuntest
    • Example 2: Join the test1 field and test2 field, encode the fields by using the MD5 algorithm, and calculate a CRC code for the fields.
      • Raw log
        test1: test1
        test2: test2
      • Transformation rule
        e_set(
            "str_crc32",
            crc32_encoding(
                md5_encoding(str_join("+", v("test1"), v("test2")), format="binary")
            ),
        )
      • Result
        str_crc32:369733261
        test1:test1
        test2:test2
    • Example 3: Calculate a CRC code for the test field. The field value is Base64-encoded.
      • Raw log
        test: Taloz+e+PzP3NltrEXiCig==
      • Transformation rule
        e_set("str_crc32", crc32_encoding(v("test"), input_format="base64"))
      • Result
        str_crc32:1093789404
        test:Taloz+e+PzP3NltrEXiCig==