Dataphin:Create a User-Defined Function

Prerequisites

Resources are created. For more information, see upload resources and references.

Background information

• Dataphin organizes functions into directories by type for easier management.

• Different compute engines support different types of functions.

  Compute Engine Type		Supported Functions
  Offline Engine	MaxCompute	MAXC functions
  	Hologres	Custom functions are not supported
  	Hadoop	Hadoop functions (Hive functions), Impala functions
  	TDH Inceptor	Custom functions are not supported
  	ADB for PostgreSQL	ADB functions
  	SelectDB	Custom functions are not supported
  	Doris	Custom functions are not supported
  Real-time Engine	Alibaba Blink	FLINK functions
  	Ververica Flink	FLINK functions
  	Open-source Flink	FLINK functions

• External MaxCompute projects do not support creating custom functions.

Procedure

1. In the top menu bar of the Dataphin homepage, choose Develop > Data Development.

2. In the top menu bar, select Project (In Dev-Prod mode, you need to select Environment).

3. In the navigation pane on the left, choose Data Processing > Function.

4. In the function list on the right, click the icon and select the target function type.

5. In the Create Function dialog box, configure the parameters.

   Parameter	Description
   Name	Enter a function name. The name can contain letters, numbers, and underscores (_), and must start with a letter. Note Within the same project, Impala functions and Hive functions cannot have the same name as any custom function in any directory of the project.
   Select Resource	Select the resource file. The drop-down list shows resources that belong to the current project. Note Only JAR files are supported for function definition. When selecting multiple resources, they must be of the same type. If no resources exist, create one first. For more information, see upload resources and references.
   Programming Language	Impala supports functions defined in C++ and Java. Select the programming language that matches your resource type.
   Class Name	Enter the fully qualified class name from the resource, for example, test_udf.UDFGETSrcId.
   Type	Select the function type. Options include Window, Statistics, Numeric, String, Time, IP Address Related Functions, URL, Encoding, Business, and Others.
   Register Function	When defining a C++ Impala function, enter the CREATE FUNCTION statement. The statement must follow the syntax below. The Location clause supports resource file substitution. Create C++ scalar function CREATE FUNCTION [IF NOT EXISTS] [db_name.]function_name([arg_type[, arg_type...]) RETURNS return_type SYMBOL='symbol_name' Create C++ aggregate function CREATE [AGGREGATE] FUNCTION [IF NOT EXISTS] [db_name.]function_name([arg_type[, arg_type...]) RETURNS return_type [INTERMEDIATE type_spec] [INIT_FN='function'] UPDATE_FN='function' MERGE_FN='function' [PREPARE_FN='function'] [CLOSEFN='function'] [SERIALIZE_FN='function'] [FINALIZE_FN='function'] For more information, see User-Defined Functions (UDF).
   Syntax	Enter the function call signature, for example: bigintweekday (datetime date).
   Usage Documentation	Enter a usage example for the function, for example: select get_week_date("20170810",0,2),--Query the date of Tuesday in the week of August 10. from cndata.dual
   Select Directory	Defaults to the directory of the current function type. You can only change it to a subdirectory under that function type directory. For example, when you create a MAXC function, the MAXC function directory is selected by default. You can only select subdirectories under the MAXC function directory.

6. After completing the configuration, click Submit. In the dialog box that appears, enter Submission Notes, and then click Confirm And Submit.

   Note

   • If a resource referenced by the custom function is updated, resubmit the function to update its registration with the compute engine.

   • After submission, dependent tasks automatically reference the new version of the function, which may cause those tasks to fail. Verify affected tasks promptly.

   You can use Ad Hoc Query (see query and download data) to run SQL that references the function and verify it works as expected. Example:

   select   
   get_week_date("20170810",0,2),--Query the date of Tuesday in the week of August 10.
   from  cndata.dual

What to do next

• In Dev-Prod mode, publish the function to the production environment. For more information, see manage publishing tasks.

• In Basic mode, the custom function is available for computing task development immediately after submission. For more information, see data development overview.