Realtime Compute for Apache Flink:Run and debug connectors locally

Step 1: Add dependencies

1. Download the connector uber JAR package that contains runtime classes from the Maven central repository.

   For example, if the version of the ververica-connector-odps dependency of the MaxCompute connector is 1.17-vvr-8.0.11-1, you need to download the ververica-connector-odps-1.17-vvr-8.0.11-1-uber.jar package from the directory of the Maven central repository to your on-premises directory.

2. When you obtain the execution environment, set the pipeline.classpaths parameter to the path of the uber JAR package.

   If multiple connector dependencies exist, separate the paths of the packages with semicolons (;). For example, you can set this parameter to file:///path/to/a-uber.jar;file:///path/to/b-uber.jar. In Windows, you must add the name of the related disk to the paths, such as file:///D:/path/to/a-uber.jar;file:///E:/path/to/b-uber/jar. The following sample code shows the configurations for a DataStream program:

   Configuration conf = new Configuration();
   conf.setString("pipeline.classpaths", "file://" + "Absolute path of the uber JAR package.
   StreamExecutionEnvironment env =
     	StreamExecutionEnvironment.getExecutionEnvironment(conf);

   The following sample code shows the configurations for a Table API program:

   Configuration conf = new Configuration();
   conf.setString("pipeline.classpaths", "file://" + "Absolute path of the uber JAR package.
   EnvironmentSettings envSettings =
     	EnvironmentSettings.newInstance().withConfiguration(conf).build();
   TableEnvironment tEnv = TableEnvironment.create(envSettings);

Step 2: Configure the ClassLoader JAR package that is required to run the job

To enable Flink to load the runtime classes of the connector, add the ClassLoader JAR package to the configuration:

1. Click to download the ClassLoader JAR package based on your VVR version:

   • ClassLoader JAR for VVR 6.x

   • ClassLoader JAR for VVR 8.x

   • ClassLoader JAR for VVR 11.x

2. Open the program file in IntelliJ IDEA.

3. Click the green icon to the left of the entry class to expand the menu.

   

4. Select Modify Run Configuration....

   

5. In the window that appears, click Modify options.

   

6. In the Add Run Options drop-down list, select Modify classpath in the Java section.

   The Modify classpath section appears in the window.

7. In the Modify classpath section, click the + icon, choose Include, and select the downloaded ClassLoader JAR.

8. Save the configuration.

   Note

   If an error message indicates missing common Flink classes, click Modify options and select Add dependencies with "provided" scope to classpath.