Create a MaxCompute user-defined function (UDF) - DataWorks

Background

A UDF extends the built-in function library with custom logic and calculations for queries. For more information, see UDF overview. You can also create UDFs by using MaxCompute Studio or the command line. For more information, see Create a UDF in MaxCompute Studio and Create a UDF in MaxCompute using commands.

Prerequisites

Before you create a function, create and upload a MaxCompute resource to DataWorks. For more information, see Create a MaxCompute resource.

Note

To prepare your resource file, see Develop a UDF (Java) or Develop a UDF (Python 3).

Limitations

DataWorks can only view and manage UDFs uploaded through its visual interface. If a UDF was added to MaxCompute using another tool, such as MaxCompute Studio, you must manually load the UDF into DataWorks using the MaxCompute Functions feature. For more information, see Manage MaxCompute functions.

Register a function

Log on to the DataWorks console. In the target region, click Data Development and O&M > Data Development in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.
Create a workflow. For more information, see Create a recurring workflow.

Create a function.

Open the workflow, right-click MaxCompute, and select Create Function.
In the Create Function dialog box, enter a Name and select a Path.
Click Create.

In the Register Function dialog box, configure the parameters.

Parameter	Description
Function Type	Select the function type. Valid values: Mathematical Operation Functions, Aggregate Functions, String Processing Functions, Date Functions, Window Functions, and Other Functions. For more information, see Use built-in functions.
MaxCompute Engine Instance	Read-only.
Function Name	The name of the UDF, used to reference the function in SQL statements. The name must be globally unique and cannot be modified after registration.
Owner	By default, the currently logged-in account is displayed. You can select a different account if needed.
Class Name	The class name of the UDF, in the format `resource_name.class_name`. The resource name can be a Java package name or a Python resource name. When you create a UDF in DataWorks, you can use MaxCompute resources of the JAR or Python type. The class name configuration varies by resource type: If the resource type is JAR, the Class Name is in the format `package_name.actual_class_name`. You can obtain this value in IntelliJ IDEA using the `copy reference` command. For example, if the Java package name is `com.aliyun.odps.examples.udf` and the actual class name is `UDAFExample`, set the Class Name parameter to `com.aliyun.odps.examples.udf.UDAFExample`. If the resource type is Python, the Class Name is in the format `python_resource_name.actual_class_name`. For example, if the Python resource name is `LcLognormDist_sh` and the actual class name is `LcLognormDist_sh`, set the Class Name parameter to `LcLognormDist_sh.LcLognormDist_sh`. Note Do not add the .jar or .py suffix to the resource name. The resource must be committed and published before it can be used. For more information, see Create and use MaxCompute resources.
Resources	Select the resources required to register the function. Visual mode: You can select only resources that have been uploaded or added to DataWorks. Code editor: You can select all resources in the corresponding MaxCompute compute engine. Note You do not need to enter the paths of the added resources. If the UDF calls multiple resources, separate the resource names with a comma (,).
Description	A brief description of the UDF's purpose.
Expression Syntax	An example of how to use the UDF. For example, `test`.
Parameter Description	A description of the supported input parameter types and the return value type.
Return Value	An example of the return value, such as 1. This parameter is optional.
Example	A usage example for the function. This parameter is optional.

Click the Save icon in the toolbar to save the function.
Commit the function.
1. Click the Submit icon in the toolbar.
2. In the Submission dialog box, enter a Change Description.
3. Click OK.

For more information about how to view functions in MaxCompute and their change history, see Manage MaxCompute functions.

View and roll back function versions

Right-click a function name and click View Earlier Versions to view its version history or roll it back.

The version history panel lists the commit time, change description, and submitter for each version. To restore a previous version, select it and click Roll Back.

Use a function in a node

Reference a UDF by name directly in SQL statements. To quickly insert the function name into the editor for the current node, right-click Insert Function. In the project file tree on the left, under the Function node, right-click the target function, such as getregion, and select Reference Function to insert it into the SQL editor. For example, use getregion(ip) AS region in a SELECT statement to call the UDF.

Appendix 1: View user-defined functions

Run the SHOW FUNCTIONS command to list all registered UDFs in the MaxCompute project bound to the DataStudio component of your DataWorks workspace.
MaxCompute provides many built-in functions. For more information, see Overview of built-in functions.

// View the functions in the current project.
SHOW FUNCTIONS;

Appendix 2: View function details

Run the DESCRIBE or DESC command with a function name to view the details of a UDF.

// Use the short form to view the details of a UDF.
DESC FUNCTION <function_name>;

If the built-in functions in a workflow do not meet your requirements, create a MaxCompute UDF and upload and associate its resources, such as JAR or Python files. For more information, see Manage MaxCompute resources.

Best practices

After you create a UDF, see Grant access to a specific UDF to a specified user to configure access control for the UDF.

FAQ

Q: After I upload a resource and define it as a UDF in DataWorks, can I use it in SQL queries in DataAnalysis in addition to ODPS SQL nodes in DataStudio?

A: Yes. UDFs registered in DataWorks are stored in the underlying MaxCompute project, making them available to both ODPS SQL nodes and SQL queries in DataAnalysis.

DataWorks:Create and use a user-defined function