Use dynamic parameters in MaxCompute UDAFs and UDTFs with the Resolve annotation - MaxCompute

This topic describes how to use dynamic parameters with the Resolve annotation in MaxCompute UDAFs and UDTFs.

Extended syntax for the Resolve annotation

In MaxCompute, UDAFs and UDTFs use the Resolve annotation to determine the function's signature.

@com.aliyun.odps.udf.annotation.Resolve("BIGINT->DOUBLE")
public class UDTFClass extends UDTF {
    ...
}

Note

This example defines a UDTF that accepts a BIGINT input parameter and returns a DOUBLE value.

MaxCompute supports the following extensions for the Resolve annotation syntax:

You can use * in the input parameter list to accept a variable number of input parameters of any type. For example, @Resolve('double,*->String') indicates that the function accepts a DOUBLE as its first input parameter, followed by any number of subsequent parameters of any type. You must then write code to check the number and types of the input parameters and process them accordingly. This mechanism is similar to the printf function in C.

Note
The asterisk has a different meaning in the return value list.
You can use the any keyword in the input parameter list to represent a parameter of any type. For example, @Resolve('double,any->string') indicates that the function accepts a first input parameter of the DOUBLE type and a second input parameter of any type.

Note
The any keyword cannot be used in the return value list or for subtypes of complex types, such as ARRAY.
The return value of a UDTF can be an asterisk (*), which represents an arbitrary number of STRING type values. The number of return values depends on the number of aliases that are specified when the function is called. For example, with the annotation @Resolve("ANY,ANY->DOUBLE,*"), the function is called as UDTF(x, y) as (a, b, c). In this case, three aliases are specified after as: a, b, and c. The editor determines that a is of the DOUBLE type, which is specified as the first return value in the annotation, and that b and c are of the STRING type. Because three return values are specified, the UDTF must forward an array of length 3 when the forward method is called. Otherwise, a runtime error occurs.

Note
This type of error cannot be detected at compile time. Therefore, when you call a UDTF, the number of aliases in your SQL query must match the number of columns returned by the UDTF. This feature does not apply to UDAFs, which always have a single return value.

UDTF example

import com.aliyun.odps.udf.UDFException;
import com.aliyun.odps.udf.UDTF;
import com.aliyun.odps.udf.annotation.Resolve;
import org.json.JSONException;
import org.json.JSONObject;
@Resolve("STRING,*->STRING,*")
public class JsonTuple extends UDTF {
  private Object[] result = null;
  @Override
  public void process(Object[] input) throws UDFException {
    if (result == null) {
      result = new Object[input.length];
    }
    try {
      JSONObject obj = new JSONObject((String)input[0]);
      for (int i = 1; i < input.length; i++) {
        // The variable-length part of the return value must consist of STRING types.
        result[i] = String.valueOf(obj.get((String)(input[i])));
      }
      result[0] = null;
    } catch (JSONException ex) {
      for (int i = 1; i < result.length; i++) {
        result[i] = null;
      }
      result[0] = ex.getMessage();
    }
    forward(result);
  }
}

In this UDTF example, the number of return values is determined by the number of input parameters. The first input parameter is a JSON string, and the subsequent parameters are the keys to extract from the JSON object. The first return value reports any parsing errors. If parsing is successful, this value is null, and the subsequent values are the content extracted for each key. The following examples demonstrate this UDTF.

-- Customize the number of output aliases based on the number of input parameters.
SELECT my_json_tuple(json, 'a', 'b') as exceptions, a, b FROM jsons;

-- When no JSON fields are to be parsed, expect only the error message column.
SELECT my_json_tuple(json) as exceptions FROM jsons;

-- The following SQL statement causes a runtime error because the number of aliases
-- does not match the actual number of outputs.
-- Note that this error cannot be detected at compile time.
SELECT my_json_tuple(json, 'a', 'b') as exceptions, a, b, c FROM jsons;

If the extensions described in this topic do not meet your business requirements, you can implement similar functionality by using UDTs. For more information, see UDT Overview.

Note

For Python examples of UDAFs and UDTFs, see Python 3 UDAF and Python 3 UDTF to read MaxCompute resource example.