Reference

How do I upload a JAR package in the Object Storage Service (OSS) console?

1. In the console of fully managed Flink, view the OSS bucket of the current cluster.
   The following figure shows information about the OSS bucket.
2. Log on to the OSS console and upload the JAR package to the /artifacts/namespaces directory of the OSS bucket.
3. In the left-side navigation pane of the console of fully managed Flink, click Artifacts to view the JAR package that you uploaded in the OSS console.

How do I configure the parameters that are related to checkpoints and state backends?

In the upper-right corner of the job details page, click Configure. On the right side of the Draft Editor page, click the Advanced tab. In the panel that appears, add the code shown in the following figure to the Additional Configuration section and save the code to make it take effect. The following figure shows the sample code.

state.backend: rocksdb
state.backend.incremental: true
table.exec.state.ttl: 129600000

The following table describes the parameters that you can configure for fully managed Flink in addition to the parameters that are provided by Apache Flink.

Parameter	Description	Remarks
state.backend	The stream state storage system.	Valid values: GeminiStateBackend: specifies the stream state storage system provided by Alibaba Cloud. This is the default value. RocksDB: specifies the stream state storage system provided by Apache Flink.
table.exec.state.ttl	The time-to-live (TTL) of state data in SQL jobs.	Unit: milliseconds. For example, if you set this parameter to 129600000, the TTL of the state data is 1.5 days. The default value of this parameter varies based on the Ververica Runtime (VVR) version of Flink: If the VVR version is 4.0.12 or later, the default value is 129600000. This value is equal to 1.5 days. If the VVR version is earlier than 4.0.12, this parameter is left empty by default. This indicates that the state data does not expire.
state.backend.gemini.savepoint.external-sort.local-storage.enabled	Specifies whether the temporary data generated during the savepoint operation is stored on a local disk.	Valid values: false: Temporary data is stored on a Distributed File System (DFS). This is the default value. true: Temporary data is stored on a local disk to minimize the interactions with the DFS and speed up the savepoint operation. If the storage usage of your local disk is less than 30% and your local disk has sufficient space to store the temporary data, we recommend that you set this parameter to true. Note Only VVR 4.0 and later support this parameter.

For more information about how to configure the TTL of state data in DataStream jobs or Python jobs, see State Time-To-Live (TTL). For more information about the parameters that are related to checkpoints and state backends in Flink, see Checkpoints and State Backends.

How do I find the job that triggers an alert?

The alert event contains JobID and Deployment ID. However, JobID changes after a failover is performed. In this case, you must use Deployment ID to find the job for which the error is returned. Deployment ID is not displayed in the console of fully managed Flink. You must obtain this information from the URL of the job.

How do I view information about a workspace, such as the workspace ID?

Log on to the Realtime Compute for Apache Flink console, find the workspace whose information you want to view, and then choose More > Workspace Details in the Actions column.

How do I query the engine version of Flink that is used by a job?

Can I directly deactivate Prometheus Service that is automatically activated with fully managed Flink?

No, you cannot directly deactivate Prometheus Service. However, you can uninstall the Prometheus Service plug-in. For more information, see Uninstall the Prometheus agent. If you have questions about how to uninstall the Prometheus Service plug-in, submit a ticket.

How do I configure a whitelist?

How does a fully managed Flink cluster access the Internet?

• Background information
  By default, fully managed Flink clusters cannot access the Internet. Therefore, Alibaba Cloud provides NAT gateways to enable communications between virtual private clouds (VPCs) and the Internet. This way, users of fully managed Flink clusters can access the Internet by using user-defined functions (UDFs) or DataStream code.
• Solution
  Create a NAT gateway in the VPC. Then, create a source network address translation (SNAT) entry to bind the vSwitch that is associated with the fully managed Flink cluster to an elastic IP address (EIP). This way, the cluster can access the Internet by using the EIP. To access the Internet by using the EIP, perform the following steps:

  1. Create a NAT gateway. For more information, see Create a NAT gateway.
  2. Create an SNAT entry. For more information, see Create an SNAT entry.
  3. Bind the vSwitch that is associated with the fully managed Flink cluster to an EIP. For more information, see Associate an EIP with an Internet NAT gateway.

How does Realtime Compute for Apache Flink access storage resources across VPCs?

How do I troubleshoot dependency conflicts of Flink?

• Problem description
  • An error caused by an issue in Flink or Hadoop is reported.

    java.lang.AbstractMethodError
    java.lang.ClassNotFoundException
    java.lang.IllegalAccessError
    java.lang.IllegalAccessException
    java.lang.InstantiationError
    java.lang.InstantiationException
    java.lang.InvocationTargetException
    java.lang.NoClassDefFoundError
    java.lang.NoSuchFieldError
    java.lang.NoSuchFieldException
    java.lang.NoSuchMethodError
    java.lang.NoSuchMethodException

  • No error is reported, but one of the following issues occur:
    • Logs are not generated or the Log4j configuration does not take effect.
      In most cases, this issue occurs because the dependency contains the Log4j configuration. To resolve this issue, you must check whether the dependency in the JAR file of your job contains the Log4j configuration. If the dependency contains the Log4j configuration, you can configure exclusions in the dependency to remove the Log4j configuration.

      Note If you use different versions of Log4j, you must use maven-shade-plugin to relocate Log4j-related classes.
    • The remote procedure call (RPC) fails.

      By default, errors caused by dependency conflicts during Akka RPCs of Flink are not recorded in logs. To check these errors, you must enable debug logging.

      For example, a debug log records Cannot allocate the requested resources. Trying to allocate ResourceProfile{xxx}. However, the JobManager log stops at the message Registering TaskManager with ResourceID xxx and does not display any information until a resource request times out and displays the message NoResourceAvailableException. In addition, TaskManagers continuously report the error message Cannot allocate the requested resources. Trying to allocate ResourceProfile{xxx}.

      Cause: After you enable debug logging, the RPC error message InvocationTargetException appears. In this case, slots fail to be allocated for TaskManagers and the status of the TaskManagers becomes inconsistent. As a result, slots cannot be allocated and the error cannot be fixed.

• Cause
  • The JAR package of your job contains unnecessary dependencies, such as the dependencies for basic configurations, Flink, Hadoop, and Log4j. As a result, dependency conflicts occur and cause some issues.
  • The dependency that corresponds to the connector that is required for your job is not included in the JAR package.
• Troubleshooting
  • Check whether the pom.xml file of your job contains unnecessary dependencies.
  • Run the jar tf foo.jar command to view the content of the JAR package and determine whether the package contains the content that causes dependency conflicts.
  • Run the mvn dependency:tree command to check the dependency relationship of your job and determine whether dependency conflicts exist.
• Solution
  • We recommend that you set scope to provided for the dependencies for basic configurations. This way, the dependencies for basic configurations are not included in the JAR package of your job.
    • DataStream Java

      <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-streaming-java_2.11</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
      </dependency>

    • DataStream Scala

      <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-streaming-scala_2.11</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
      </dependency>

    • DataSet Java

      <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-java</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
      </dependency>

    • DataSet Scala

      <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-scala_2.11</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
      </dependency>

  • Add the dependencies that correspond to the connectors required for the job, and set scope to compile. This way, the dependencies that correspond to the required connectors are included in the JAR package. The default value of scope is compile. In the following code, the Kafka connector is used as an example.

    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-connector-kafka_2.11</artifactId>
        <version>${flink.version}</version>
    </dependency>

  • We recommend that you do not add the dependencies for Flink, Hadoop, or Log4j. Take note of the following exceptions:
    • If the job has direct dependencies for basic configurations or connectors, we recommend that you set scope to provided. Sample code:

      <dependency>
          <groupId>org.apache.hadoop</groupId>
          <artifactId>hadoop-common</artifactId>
          <scope>provided</scope>
      </dependency>

    • If the job has indirect dependencies for basic configurations or connectors, we recommend that you configure exclusions to remove the dependencies. Sample code:

      <dependency>
          <groupId>foo</groupId>
            <artifactId>bar</artifactId>
            <exclusions>
              <exclusion>
              <groupId>org.apache.hadoop</groupId>
              <artifactId>hadoop-common</artifactId>
             </exclusion>
          </exclusions>
      </dependency>

How do I resolve the domain name of the service on which a Flink job depends?