Realtime Compute for Apache Flink:Run and debug jobs that contain connectors locally

Step 1: Add dependencies to the job configuration

First, download the uber JAR package that contains the required runtime classes from the Maven Central Repository. For example, for the ververica-connector-odps dependency used by MaxCompute, if the version is 1.17-vvr-8.0.11-1, you can find the ververica-connector-odps-1.17-vvr-8.0.11-1-uber.jar file—identified by the -uber.jar suffix—in the corresponding directory of the Maven repository. Download it to a local directory.

When you create the environment in your code, add the pipeline.classpaths configuration and set it to the path of the uber JAR. If you have multiple connector dependencies, separate their paths with a semicolon (;). For example, file:///path/to/a-uber.jar;file:///path/to/b-uber.jar. On Windows, you must include the disk partition. For example, file:///D:/path/to/a-uber.jar;file:///E:/path/to/b-uber.jar. For a DataStream API job, use the following code to add the configuration:

Configuration conf = new Configuration();
conf.setString("pipeline.classpaths", "file://" + "absolute path of the uber jar");
StreamExecutionEnvironment env =
  	StreamExecutionEnvironment.getExecutionEnvironment(conf);

For a Table API job, use the following code to add the configuration:

Configuration conf = new Configuration();
conf.setString("pipeline.classpaths", "file://" + "absolute path of the uber jar");
EnvironmentSettings envSettings =
  	EnvironmentSettings.newInstance().withConfiguration(conf).build();
TableEnvironment tEnv = TableEnvironment.create(envSettings);

Step 2: Configure the ClassLoader JAR package required for the runtime

To allow Flink to load the connector’s runtime classes, you must also add the ClassLoader JAR package. First, download the appropriate ClassLoader JAR based on your Ververica Runtime (VVR) database engine version. The download links are as follows:

• Download ClassLoader JAR for VVR 6.x

• Download ClassLoader JAR for VVR 8.x

• Download ClassLoader JAR for VVR 11.x

As an example, modify the local run configuration for the job in IntelliJ IDEA. Click the green icon to the left of the entry class to expand the menu bar, and select Modify Run Configuration...:

In the run configuration window that opens, click Modify options and select Modify classpath. A Modify classpath section appears at the bottom of the window. Click the + icon, select the ClassLoader JAR package that you downloaded, and save the run configuration. If an error message indicates that common Flink classes such as org.apache.flink.configuration.Configuration are missing and the job fails to execute, select Add dependencies with provided scope to classpath under Modify options.

Starting with VVR 11.1, the VVR engine uses JDK 11. You must add the --add-opens java.base/jdk.internal.loader=ALL-UNNAMED option in the JVM Options, as shown in the following figure: