Register, update, and delete UDFs - Realtime Compute for Apache Flink

Before you begin

If a RAM user or RAM role needs to manage UDFs, ensure it has permission to access the target namespace. For more information, see development console authorization.
To avoid dependency conflicts in JAR files when developing UDFs, follow these guidelines:
- Ensure that the Flink version you select on the development page matches the Flink version in your POM dependency.
- For Flink-related dependencies, set their scope to provided by adding <scope>provided</scope>.
- Use the Apache Maven Shade Plugin to package other third-party dependencies. For more information, see Apache Maven Shade Plugin.
Note
For more information about Flink dependency conflicts, see How do I troubleshoot Flink dependency conflicts?.

Register a UDF

Catalog UDFs

Navigate to the UDF registration page.
1. Log on to the Realtime Compute for Apache Flink console.
2. In the Actions column of the target workspace, click Console.
3. In the navigation pane on the left, click Development > ETL.
4. Click the Functions tab.
Click Register UDF.
Upload the UDF JAR file.
You can upload a UDF JAR file in one of the following ways:
- Upload a file: Click Select a file next to the Select a file field to choose your UDF artifact. If the UDF has dependencies, click Select a file next to the Dependency File field to upload the required files.
  Note
  
  Your UDF file is uploaded to the sql-artifacts directory in the OSS bucket that you selected.
  
  For a Java UDF, dependencies can be packaged into the UDF JAR file or uploaded separately. For a Python UDF, we recommend that you upload dependencies separately.
- External URL: To use a UDF file from another service, provide its external URL.
  Note
  Two types of external URLs are supported:
  
  The OSS bucket associated with your Flink workspace. View the bucket details on the Workspace Details page in the Management Portal.
  
  A URL for an external storage system that Realtime Compute for Apache Flink has access to (public-read or with granted permission).
Click OK.
On the Available Functions page, select the UDFs that you want to register and click Create Function.

The development console parses the artifact file, identifies classes that use Flink UDF, UDAF, and UDTF interfaces, and automatically populates the Function Name field with the class names. After a UDF is registered, you can view it in the Functions list. A yellow "fx" icon appears to the left of each registered UDF.

Note
When you register a catalog UDF, the latest Flink version is used for parsing by default. If your SQL deployment uses an earlier engine version, this can cause incompatibility issues. To resolve this, you can implement the UDF code based on the required engine version and register it as a deployment-level UDF. For more information, see Deployment-level UDFs.

Deployment-level UDFs

Important

Deployment-level Python UDFs are supported only in VVR 8.0.3 and later.
When you use a deployment-level Python UDF, you can configure dependency files by using parameters such as python.files or python.archives in the deployment parameters.
When you use a deployment-level Python UDF, a syntax check is not supported. You must skip the syntax check when you start the deployment.

Follow these steps to register a deployment-level UDF:

Upload the UDF JAR file or Python file.

In the navigation pane on the left, go to Artifacts and upload the UDF JAR or Python file.
Specify the deployment-level UDF in the deployment.

In the Additional Dependencies section for the deployment, specify the UDF JAR or Python file.

Register the deployment-level UDF.

Java UDF

CREATE TEMPORARY FUNCTION yourfunctionname;

Python UDF

CREATE TEMPORARY FUNCTION yourfunctionname LANGUAGE Python;

Update a UDF

Follow these steps to update a UDF JAR file after adding a new UDF or modifying an existing one.

Navigate to the UDF registration page.
1. Log on to the Realtime Compute for Apache Flink console.
2. In the Actions column of the target workspace, click Console.
3. In the navigation pane on the left, click Development > ETL.
4. Click the Functions tab.
In the Functions list, hover over the target UDF name and click the icon.
Upload the UDF JAR file. In the Update UDF JAR dialog box, select either the Upload a file or External URL method, configure the Select a file and Dependency File fields, and click Modify.
Important
- The new UDF JAR file must contain the classes for all currently registered UDFs.
- Changes in the new UDF JAR file take effect only for new or restarted deployments. Running deployments continue to use the old file until they are restarted.
Click Modify.

Delete a UDF

Note

Before you delete a UDF JAR file, ensure that no deployments or SQL files reference the UDFs registered from it.

To delete a UDF JAR file, follow these steps:

Navigate to the UDF registration page.
1. Log on to the Realtime Compute for Apache Flink console.
2. In the Actions column of the target workspace, click Console.
3. In the navigation pane on the left, click Development > ETL.
4. Click the Functions tab.
In the Functions list, hover over the target UDF name and click the icon.
Select the Delete all functions and the associated file checkbox.

To delete the UDF JAR file, you must also delete all UDFs that were registered from it to avoid data inconsistencies.
Click OK.

Realtime Compute for Apache Flink:Manage UDFs