SLS data transformation functions reference - Simple Log Service

Lists all built-in functions for SLS data transformation, organized by category.

Global processing functions

Function classification	Function name	Description
Flow control functions	e_if	Evaluates multiple condition-operation pairs in sequence. If a condition is met, performs the corresponding operation. If not, evaluates the next condition. If an operation deletes a log, subsequent operations are skipped.
	e_if_else	Performs different operations based on whether a condition evaluates to true or false.
	e_switch	Evaluates multiple condition-operation pairs and performs the operation for the first matched condition. If a condition is met, performs the corresponding operation and returns the result. If not, evaluates the next condition. If no conditions are met and a default parameter is configured, performs the default operation and returns the result. If an operation deletes a log, subsequent operations are skipped.
	e_compose	Combines multiple operations into a single composite operation. Commonly used within e_if, e_switch, or e_if_else functions. Executes operations on a log in sequence and returns the result. If an operation deletes a log, subsequent operations are skipped.
Event operation functions	e_drop	Discards a log based on whether a specified condition is met.
	e_keep	Retains a log based on whether a specified condition is met.
	e_split	Splits a log into multiple logs based on a field value. Supports JMESPath extraction before splitting.
	e_output	Outputs a log to a specified Logstore with configurable topic, source, tag, and shard hash. Stops further transformation on the log.
	e_coutput	Outputs a log to a specified Logstore with configurable topic, source, tag, and shard hash. Continues further transformation on the log.
	e_to_metric	Converts a log to a metric that can be stored in a Metricstore.
Field operation functions	v	Extracts a field value from a log. When multiple field names are specified, returns the first existing field's value.
	e_set	Adds a field or sets a new value for an existing field.
	e_drop_fields	Deletes log fields that match a specified condition.
	e_keep_fields	Retains log fields that match a specified condition.
	e_pack_fields	Packs log fields and assigns them as the value of a new field.
	e_rename	Renames log fields that match a specified condition.
Value extraction functions	e_regex	Extracts a field value by using a regular expression and assigns the result to other fields.
	e_json	Processes JSON objects in a field. Supports expanding, extracting values with JMESPath, or extracting and expanding JSON data.
	e_kv	Extracts key-value pairs from multiple source fields by using a specified quote character.
	e_kv_delimit	Extracts key-value pairs from source fields by using a specified delimiter.
	e_csv	Extracts multiple fields from a specified field by using a delimiter and predefined field names. The default delimiter is a comma (,).
	e_tsv	Extracts multiple fields from a specified field by using a delimiter and predefined field names. The default delimiter is `\t`.
	e_psv	Extracts multiple fields from a specified field by using a delimiter and predefined field names. The default delimiter is a vertical bar (\|).
	e_syslogrfc	Calculates the facility and severity from a priority value based on the Syslog protocol, and determines the corresponding log level.
	e_anchor	Extracts strings by using specified anchor rules.
Mapping and enrichment functions	e_dict_map	Maps an input field to a new field by using a mapping dictionary.
	e_table_map	Looks up the matching row in a mapping table and returns the corresponding field value based on the input field.
	e_tablestore_map	Enriches log data by using a Tablestore data table as the dimension table.
	e_redis_map	Enriches log data by using an ApsaraDB for Redis data table as the dimension table.

Expression functions

Function classification	Function name	Description
Event check functions	e_has	Checks whether a field exists.
	e_not_has	Checks whether a field does not exist.
	e_search	Searches for a log by using a simplified query syntax similar to Lucene.
	e_match, e_match_all, and e_match_any	Checks whether a log field value matches a specified regular expression.
Operator functions	op_if	Returns the value of an expression based on a specified condition.
	op_ifnull	Returns the value of the first expression whose value is not None.
	op_coalesce	Returns the value of the first expression whose value is not None.
	op_nullif	Returns None if the values of two expressions are equal. Otherwise, returns the value of the first expression.
	op_and	Returns True if all expressions evaluate to true. Accepts any data type.
	op_not	Returns the inverse Boolean value of an expression. Accepts any data type.
	op_or	Returns True if any expression evaluates to true, or False if all evaluate to false. Accepts any data type.
	op_eq	Evaluates the `a==b` condition and returns True or False.
	op_ge	Evaluates the `a>=b` condition and returns True or False.
	op_gt	Evaluates the `a>b` condition and returns True or False.
	op_le	Evaluates the `a<=b` condition and returns True or False.
	op_lt	Evaluates the `a<b` condition and returns True or False.
	op_ne	Evaluates the `a!=b` condition and returns True or False.
	op_len	Returns the number of elements in a string, tuple, list, or dictionary.
	op_in	Checks whether a string, tuple, list, or dictionary contains a specified element and returns True or False.
	op_not_in	Checks whether a string, tuple, list, or dictionary does not contain a specified element and returns True or False.
	op_slice	Slices a specified string, array, or tuple.
	op_index	Returns the element at a specified index in a string, array, or tuple.
	op_add	Returns the sum of multiple values. The values can be strings or numbers.
	op_max	Returns the largest value among the values of multiple fields or expressions.
	op_min	Returns the smallest value among the values of multiple fields or expressions.
Conversion functions	ct_int	Converts the value of a field or an expression to an integer.
	ct_float	Converts the value of a field or an expression to a floating-point number.
	ct_str	Converts the value of a field or an expression to a string.
	ct_bool	Converts the value of a field or an expression to a Boolean value.
	ct_chr	Converts the ANSI or Unicode value of a field or an expression to a character.
	ct_ord	Converts the characters in a field or expression to their corresponding ANSI or Unicode values.
	ct_hex	Converts the value of a field or an expression to a hexadecimal number.
	ct_oct	Converts the value of a field or an expression to an octal number.
	ct_bin	Converts the value of a field or an expression to a binary number.
	bin2oct	Converts a binary byte string to an octal string.
	bin2hex	Converts a binary byte string to a hexadecimal string.
Arithmetic functions	op_abs	Returns the absolute value of an input value.
	op_div_floor	Calculates the integer division of the input value.
	op_div_true	Divides the input value.
	op_pow	Returns an input value raised to a specified power.
	op_mul	Multiplies an input value.
	op_neg	Returns the opposite number of an input value.
	op_mod	Performs a modulo calculation on the input value.
	op_sub	Applies a subtraction to the input value.
	op_round	Rounds the incoming value.
	op_sum	Returns the sum of input values.
	mat_ceil	Rounds an input value up to the nearest integer.
	mat_exp	Returns e raised to the power of the input value.
	mat_fabs	Returns the absolute value of an input value.
	mat_floor	Rounds an input value down to the nearest integer.
	mat_log	Calculates the logarithm of the input value.
	mat_log10	Returns the base-10 logarithm of an input value.
	mat_sqrt	Returns the square root of an input value.
	mat_degrees	Converts radians to degrees.
	mat_radians	Converts degrees to radians.
	mat_sin	Returns the sine of an input value in radians.
	mat_cos	Returns the cosine of an input value in radians.
	mat_tan	Returns the tangent of an input value in radians.
	mat_acos	Returns the arc cosine of an input value in radians.
	mat_asin	Returns the arc sine of an input value in radians.
	mat_atan	Returns the arc tangent of an input value in radians.
	mat_atan2	Returns the arc tangent of X and Y coordinates.
	mat_atanh	Returns the inverse hyperbolic tangent of an input value.
	mat_hypot	Computes the Euclidean norm of the input values.
String functions	str_format	Formats a string by using a specified format.
	str_join	Concatenates input strings by using a specified separator.
	str_zip	Splits two values or expression results in parallel and combines the results into a single string.
	str_encode	Encodes a string using a specified encoding format.
	str_decode	Decodes an input value using a specified encoding format.
	str_hex_escape_encode	Escapes special characters in a string and converts hexadecimal escape sequences to their corresponding characters.
	str_sort	Sorts a specified object.
	str_reverse	Reverses a string.
	str_replace	Replaces a specified substring within a string with a new string.
	str_logtash_config_normalize	Converts data in the Logstash configuration language to the JSON format.
	str_translate	Replaces specified characters in a string with mapping characters.
	str_strip	Deletes specified characters from a string.
	str_lstrip	Deletes specified characters from the start of a string.
	str_rstrip	Deletes specified characters from the end of a string.
	str_lower	Converts a string to lowercase.
	str_upper	Converts a string to uppercase.
	str_title	Capitalizes the first letter of each word and lowercases the rest.
	str_capitalize	Capitalizes the first letter of a string and lowercases the rest.
	str_swapcase	Swaps uppercase and lowercase letters in a string.
	str_count	Counts the number of occurrences of a character in a string.
	str_find	Checks whether a string contains a specified substring.
	str_rfind	Returns the position of the last occurrence of a specified character in a string.
	str_endswith	Checks whether a string ends with a specified suffix.
	str_startswith	Checks whether a string starts with a specified string.
	str_split	Splits a string using a specified delimiter.
	str_splitlines	Splits a string using a line feed.
	str_partition	Splits a string into three parts from left to right using a specified delimiter.
	str_rpartition	Splits a string into three parts from right to left using a specified delimiter.
	str_center	Pads a string to a specified length using a specified character.
	str_ljust	Pads a string to a specified length using a specified character from the end of the string.
	str_rjust	Pads a string to a specified length using a specified character from the start of the string.
	str_zfill	Pads a string to a specified length using 0 from the start of the string.
	str_expandtabs	Converts `\t` characters in a string to spaces.
	str_isalnum	Checks whether a string contains only letters and digits.
	str_isalpha	Checks whether a string contains only letters.
	str_isascii	Checks whether a string is in the ASCII table.
	str_isdecimal	Checks whether a string contains only decimal characters.
	str_isdigit	Checks whether a string contains only digits.
	str_isidentifier	Checks whether a string is a valid Python identifier.
	str_islower	Checks whether a string contains only lowercase letters.
	str_isnumeric	Checks whether a string contains only numeric characters.
	str_isprintable	Checks whether all characters in a string are printable.
	str_isspace	Checks whether a string contains only whitespace.
	str_istitle	Checks whether a string is title-cased.
	str_isupper	Checks whether a string is all uppercase.
	str_uuid	Generates a random universally unique identifier (UUID).
Date and time functions	dt_parse	Converts a value or the value of a time expression to a datetime object.
	dt_str	Converts a value or the value of a time expression to a string.
	dt_parsetimestamp	Converts a value or the value of a time expression to a UNIX timestamp.
	dt_prop	Returns a specific attribute (such as day or year) of a datetime value or time expression.
	dt_now	Returns the current date and time.
	dt_today	Returns the current date.
	dt_utcnow	Returns the current datetime object in the current time zone.
	dt_fromtimestamp	Converts a UNIX timestamp to a datetime object.
	dt_utcfromtimestamp	Converts a UNIX timestamp to a datetime object in the current time zone.
	dt_strptime	Parses a time string into a datetime object.
	dt_currentstamp	Returns the current UNIX timestamp.
	dt_totimestamp	Converts a datetime object to a UNIX timestamp.
	dt_strftime	Converts a datetime object to a string in a specified format.
	dt_strftimestamp	Converts a UNIX timestamp to a string in a specified format.
	dt_truncate	Extracts a time value from a value or the value of a time expression based on a specified time granularity.
	dt_add	Changes a value or the value of a time expression based on a specified time granularity.
	dt_MO	The value passed to the `weekday` parameter in the `dt_add` function, representing the offset for a specific Monday.
	dt_TU	The value passed to the `weekday` parameter in the `dt_add` function, representing the offset for a specific Tuesday.
	dt_WE	The value passed to the `weekday` parameter in the `dt_add` function, representing the offset for a specific Wednesday.
	dt_TH	The value passed to the `weekday` parameter in the `dt_add` function, representing the offset for a specific Thursday.
	dt_FR	The value passed to the `weekday` parameter in the `dt_add` function, representing the offset for a specific Friday.
	dt_SA	The value passed to the `weekday` parameter in the `dt_add` function, representing the offset for a specific Saturday.
	dt_SU	The value passed to the `weekday` parameter in the `dt_add` function, representing the offset for a specific Sunday.
	dt_astimezone	Converts a value or the value of a time expression to a datetime object in a specified time zone.
	dt_diff	Returns the difference between two values or between the values of two time expressions based on a specified time granularity.
Regular expression functions	regex_select	Extracts a value that matches a regular expression.
	regex_findall	Extracts all values that match a regular expression.
	regex_match	Checks whether a value matches a regular expression.
	regex_replace	Replaces the characters that match a regular expression in a string.
	regex_split	Splits a string into an array of strings.
Grok function	grok	Extracts a value that matches a regular expression.
Specific structured data functions	json_select	Extracts or calculates specific values from a JSON expression using JMESPath.
	json_parse	Parses a value into a JSON object.
	xml_to_json	Converts XML data to JSON data, and then expands the converted data.
IP address parsing functions	geo_parse	Identifies the city, province, and country based on an IP address.
	ip_cidrmatch	Checks whether an IP address belongs to a Classless Inter-Domain Routing (CIDR) block.
	ip_version	Checks whether the version of an IP address is IPv4 or IPv6.
	ip_type	Checks whether an IP address is private or public.
	ip_makenet	Converts an IP address to a CIDR block.
	ip_to_format	Converts a CIDR block to a netmask or prefix length format.
	ip_overlaps	Checks whether two CIDR blocks overlap.
	ip2long	Converts an IP address to a value of the long type.
	long2ip	Converts a value of the long type to an IP address.
Encoding and decoding functions	url_encoding	Performs URL encoding on data.
	url_decoding	Performs URL decoding on data.
	protobuf_decoding	Parses data into the JSON format using a specified Protobuf template.
	str_encode	Encodes a string using a specified encoding format.
	str_decode	Decodes an input value using a specified encoding format.
	base64_encoding	Encodes data using the Base64 algorithm.
	base64_decoding	Decodes data using the Base64 algorithm.
	html_encoding	Encodes data in the HTML format.
	html_decoding	Decodes HTML-encoded data.
	md5_encoding	Encodes data using the MD5 algorithm.
	sha1_encoding	Encodes data using the SHA1 algorithm.
	crc32_encoding	Calculates a cyclic redundancy check (CRC) code for data.
	gzip_compress	Compresses and encodes data.
	gzip_decompress	Decompresses compressed data.
	zlib_compress	Compresses and encodes data.
	zlib_decompress	Decompresses compressed data.
	aes_encrypt	Encrypts data using the AES algorithm.
	aes_decrypt	Decrypts data using the AES algorithm.
	jwt_encoding	Encodes JSON data based on the JSON Web Token (JWT) standard.
	jwt_decoding	Decodes data to raw JSON data based on the JWT standard.
	hashids_encoding	Encodes data using the Hashids library.
	hashids_decoding	Decodes data that is encoded using the Hashids library.
Parsing functions	ua_parse_device	Parses User-Agent and returns the device information.
	ua_parse_os	Parses User-Agent and returns the operating system information.
	ua_parse_agent	Parses User-Agent and returns the browser information.
	ua_parse_all	Parses User-Agent and returns all information.
	url_parse	Parses a URL and returns the components of the URL.
	url_parse_qs	Parses the query string of a URL and returns the components of the query string.
List functions	lst_make	Constructs a list.
	lst_insert	Inserts elements to a specified position in a list.
	lst_append	Appends elements to a list.
	lst_delete_at	Deletes the element at a specified position from a list.
	lst_reverse	Reverses the order of elements in a list.
	lst_get	Returns the element at a specified position in a list or a tuple.
Dictionary functions	dct_make	Constructs a dictionary.
	dct_update	Updates a dictionary.
	dct_delete	Deletes key-value pairs from a dictionary.
	dct_keys	Returns the keys of a dictionary.
	dct_values	Returns the values of a dictionary.
	dct_get	Returns the value that corresponds to a specified key in a dictionary.
Table functions	tab_parse_csv	Constructs a table from comma-separated values (CSV) text.
Table functions	tab_to_dict	Constructs a dictionary from a table.
Resource functions	res_local	Pulls advanced parameter values from the current data transformation job.
	res_rds_mysql	Pulls data from an ApsaraDB RDS for MySQL table or SQL query result. Supports periodic updates.
	res_log_LogStore_pull	Pulls data from another Logstore. Supports continuous data pulling.
	res_oss_file	Pulls data from an OSS object. Supports periodic updates.