This topic describes how to use IntelliJ IDEA to develop a UDX in Realtime Compute. You can build a development environment and reference the UDX in Realtime Compute jobs.

Note
  • The operations in this topic are performed on IntelliJ IDEA. You need to download and install IntelliJ IDEA.
  • Only Realtime Compute in exclusive mode supports UDXs.

Download and configure Maven

  1. Download Maven.
    1. Visit the download page of Apache Maven and download apache-maven-3.5.3-bin.tar.gz.
    2. Decompress the downloaded package to the specified directory, such as /Users/<userName>/Documents/maven.
  2. Configure environment variables.
    1. Run the vim ~/.bash_profile command.
    2. In the .bash_profile file, add the following commands to configure environment variables:
      export M2_HOME=/Users/<userName>/Documents/maven/apache-maven-3.5.3
      export PATH=$PATH:$M2_HOME/bin
    3. Save the .bash_profile file and exit.
    4. Run the source ~/.bash_profile command for the configuration to take effect.
  3. Verify that the configuration takes effect.

    Run the mvn -v command. If information similar to the following is displayed, the configuration takes effect:

    Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-04T03:39:06+08:00)
    Maven home: /Users/<userName>/Documents/maven/apache-maven-3.5.0
    Java version: 1.8.0_121, vendor: Oracle Corporation
    Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home/jre
    Default locale: zh_CN, platform encoding: UTF-8
    OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "mac"

Build a development environment

  1. Visit Build a development environment and download the demo package RealtimeCompute-udxDemo.gz.
  2. Run the following command to decompress the RealtimeCompute-udxDemo.gz package in a Linux environment:
    tar xzvf RealtimeCompute-udxDemo.gz                
  3. Open IntelliJ IDEA and click Open to open the demo.

Develop a UDX

  1. Create a package.
    1. Right-click java and navigate to New > Package.
    2. In the New Package dialog box, enter a package name. In this example, a package named com.hjc.test.blink.sql.udx is created.
    3. Click OK.
  2. Create a class.
    1. Right-click com.hjc.test.blink.sql.udx and navigate to New > Java Class.
    2. In the Create New Class dialog box that appears, enter a class name.
      Note Retain the default value Class of Kind.
    3. Click OK.
  3. Enter the following code in the class:
    package com.hjc.test.blink.sql.udx;
    
    import org.apache.flink.table.functions.FunctionContext;
    import org.apache.flink.table.functions.ScalarFunction;
    
    public class StringLengthUdf extends ScalarFunction {
        // The open method is optional.
        // To write the open method, you must add "import org.apache.flink.table.functions.FunctionContext;".
        @Override
        public void open(FunctionContext context) {
            }
        public long eval(String a) {
            return a == null ? 0 : a.length();
        }
        public long eval(String b, String c) {
            return eval(b) + eval(c);
        }
        // The close method is optional.
        @Override
        public void close() {
            }
    }
  4. In the terminal, run the mvn package or mvn assembly:assembly command to add the project to the JAR package.
    Note
    1. If you need to add a third-party dependency to the JAR package, run the mvn assembly:assembly command.
    2. The compiled JAR package is RealtimeCompute-udxDemo/target/RTCompute-udx-1.0-SNAPSHOT.jar or RealtimeCompute-udxDemo/target/RTCompute-udx-1.0-SNAPSHOT-jar-with-dependencies.jar. It contains the third-party dependency.
  5. Reference the JAR package in Realtime Compute jobs.
    Note The maximum size of the JAR package that can be uploaded is 300 MB. If the JAR package exceeds 300 MB, you must upload it to the Object Storage Service (OSS) bucket bound to your cluster or use OpenAPI to upload it.
    1. Log on to the Realtime Compute development platform.
    2. In the top navigation bar, click Development.
    3. Click Resources on the left.
    4. In the upper-right corner of the Resources pane, click Create Resource.
    5. In the Upload Resource dialog box, configure resource parameters.
      Parameter Description
      Location You can only upload local JAR packages.
      Note The maximum size of a JAR package that you can upload is 300 MB. If the JAR package exceeds 300 MB, you must upload it to the Object Storage Service (OSS) bucket that is bound to your cluster or use OpenAPI to upload it.
      Resource Click Upload Resource to select the resource that you want to reference.
      Resource Name Enter a name for the resource.
      Resource Description Enter a resource description.
      Resource Type Select the type of the resource: JAR, DICTIONARY, or PYTHON.
    6. On the Resources pane, locate the new resource, and move the pointer over More in the Actions column.
    7. Select Reference from the drop-down list to present the resource code in the code editor.
    8. In the code editor, declare the UDX at the beginning. An example is as follows:
      CREATE FUNCTION stringLengthUdf AS 'com.hjc.test.blink.sql.udx.StringLengthUdf';