Realtime Compute for Apache Flink:Network connectivity

• How do I troubleshoot network issues?

• How do I use the network detection feature?

• How do I obtain the endpoint of a Hologres instance?

• How do I access the Internet?

• How do I view the public bandwidth?

• How do I access other services across VPCs?

• How do I configure a whitelist?

• How do I resolve the domain name of a service that a Flink job depends on?

• Error: JobManager heartbeat timeout

• Why does the 'timeout expired while fetching topic metadata' error occur even if the network between Flink and Kafka is connected?

How do I troubleshoot network issues?

Realtime Compute for Apache Flink workspaces are deployed in a virtual private cloud (VPC). You cannot change the VPC after you create a workspace. If a data source or destination is not in the same VPC as the Flink workspace, the network is disconnected, and data cannot be read or written. If you cannot read or write data, check for network issues by following these steps:

1. Check the network connectivity between your upstream and downstream services and the Flink workspace. You can test network connectivity in the Flink console. For more information, see How do I use the network detection feature?.

   By default, Realtime Compute for Apache Flink can access only services in the same region and VPC. To access resources across VPCs or over the Internet:

   • For more information about cross-VPC access, see How do I access other services across VPCs?.

   • To access the Internet, use an Alibaba Cloud NAT Gateway to connect your VPC to the Internet. For more information, see How do I access the Internet?.

2. Check if the upstream and downstream services have whitelists configured. For more information, see How do I configure a whitelist?.

3. If a network timeout error still occurs, it may be caused by a connection timeout. Increase the value of the connect.timeout parameter in the DDL `WITH` clause. The default value is 30 seconds.

How do I use the network detection feature?

Realtime Compute for Apache Flink provides a network detection feature. You can use the network detection feature in the Flink development console by following these steps:

1. Log on to the Realtime Compute for Apache Flink console.

2. In the Actions column of the target workspace, click Console.

3. In the upper-right corner of the top navigation bar, click the Network Detection icon.

   

4. Enter an endpoint or IP address to check if the Flink job environment is connected to your upstream and downstream systems.

   Important

   When you enter an endpoint, remove the :<port> suffix and enter the port number in the Port field.

   

   If a connect timed out error occurs, confirm whether the domain you are accessing is on the public network or in another VPC. By default, Realtime Compute for Apache Flink can access only services in the same VPC. To access resources across VPCs or over the public network, see How do I access other services across VPCs? and How do I access the Internet?.

How do I obtain the endpoint of a Hologres instance?

1. Log in to the Hologres console. On the Instance List page, click the target instance.

2. On the Instance Details page, you can find the endpoint in the Network Information section.

   You can obtain the corresponding endpoint based on the network type.

   Network type	Scenarios
   Specified VPC (Recommended)	A private network connected to the specified VPC. Same VPC (Recommended): The Hologres instance and the Realtime Compute for Apache Flink workspace are in the same VPC and can connect directly. Different VPCs: The Hologres instance and the workspace are in different VPCs. You must configure network settings to access cross-VPC resources. For more information, see Cross-VPC services.
   Public network	A public network with no access restrictions. Compared to an internal network, latency can be unpredictable. You must use a NAT Gateway to connect the VPC to the public network. For more information, see Configure public network access.

3. (Optional) Test the network connectivity in the Flink console. For more information, see How do I use the network detection feature?.

   • Detection successful: The endpoint is correct.

   • Detection failed: Confirm whether the Hologres instance and the Realtime Compute for Apache Flink workspace are in different VPCs or require public network access. You must configure the network as needed before you can access the instance. For more information, see Cross-VPC services and Configure public network access.

How do I access the Internet?

Accessing the Internet can have unpredictable latency compared to an internal network. If your business requires low latency and high stability, we recommend using internal network access.

Alibaba Cloud NAT Gateway connects a VPC to the Internet. This allows Realtime Compute for Apache Flink to access public data sources. For more information, see Public data sources.

How do I view the public bandwidth?

When reading from or writing to a job over the public network, if all job metrics are normal and there is no backpressure, you can check the public bandwidth to identify potential bottlenecks. Perform the following steps:

1. In the Realtime Compute for Apache Flink console, obtain the VPC ID from the Workspace Details.

2. In the VPC console, click the ID of the target VPC.

3. On the Resource Management tab, click the number next to Internet NAT Gateway.

   Note

   If the number next to Internet NAT Gateway is 0, create an Internet NAT gateway. For more information, see Internet NAT Gateway.

4. Click the instance ID of the Internet NAT gateway.

5. On the Attached EIP tab, click the instance name.

6. On the Public Network Access > Elastic IP Address page, click Monitoring to view the public bandwidth information.

How do I access other services across VPCs?

If the other service is in the early planning stage or can be replaced, we recommend purchasing a service in the same VPC as Realtime Compute for Apache Flink. Alternatively, release the current Flink workspace and create a new one in the same VPC as the other service.

Choose a suitable method to access services across VPCs based on your business needs. For more information about the solutions, see Cross-VPC services.

How do I configure a whitelist?

By default, upstream and downstream services for Realtime Compute for Apache Flink deny access from external devices. You must add the CIDR block of the virtual switch configured for the Flink workspace to the whitelist of the target service. Perform the following steps:

1. Log on to the Realtime Compute for Apache Flink console.

2. For the target workspace, in the Actions column, choose More > Workspace Details.

3. In the Workspace Details dialog box, view the CIDR block information of the workspace's virtual switch.

4. Add the CIDR block of the Flink workspace's virtual switch to the whitelist of your target upstream and downstream services.

   For example, to set a whitelist for an ApsaraDB RDS for MySQL database, see Configure an IP address whitelist.

   Note

   • If you add a new virtual switch later, you must also add its CIDR block to the target service's whitelist.

   • If your virtual switch and the upstream or downstream service are in different zones, the network can be connected after you add the virtual switch's CIDR block to the whitelist.

How do I resolve the domain name of a service that a Flink job depends on?

If a Flink job you built depends on a service specified by a domain name, you might encounter domain name resolution errors when you migrate the job to Realtime Compute for Apache Flink. In this case, use one of the following solutions to resolve the domain name:

• You have an on-premises DNS, the Flink VPC can connect to this on-premises DNS service, and the on-premises DNS can resolve the domain names correctly.

  You can use a Realtime Compute for Apache Flink job template for domain name resolution. For example, if your on-premises DNS IP address is 192.168.0.1, perform the following steps:

  1. Log on to the Realtime Compute for Apache Flink console.

  2. In the Actions column of the target workspace, click Console.

  3. On the Configuration Management page, go to the Default Job Configurations tab and add the following code to the Other Configurations text box.

     env.java.opts: >-
       -Dsun.net.spi.nameservice.provider.1=default
       -Dsun.net.spi.nameservice.provider.2=dns,sun
       -Dsun.net.spi.nameservice.nameservers=192.168.0.1

     Note

     • This method applies only to engines that do not use JDK 11. It does not work if the database engine version includes jdk-11.

     • If your on-premises DNS has multiple IP addresses, separate them with commas (,).

  4. Click Save Changes.

  5. In the Realtime Compute for Apache Flink development console, create and run a new job.

     • If an UnknownHostException error still occurs, it means the domain name cannot be resolved. In this case, contact us.

     • If the job frequently fails over and a 'JobManager heartbeat timeout' error occurs after you configure on-premises DNS resolution, see Error: JobManager heartbeat timeout for a solution.

• If you do not have an on-premises DNS, or if the Flink VPC cannot connect to your on-premises DNS, you can use Alibaba Cloud DNS to establish a connection.

Error: JobManager heartbeat timeout

• Error details

  After you configure on-premises DNS resolution, the job frequently fails over and a 'JobManager heartbeat timeout' error occurs.

• Cause

  This may be caused by high connection latency to the on-premises DNS.

• Solution

  You must disable domain name resolution for the TaskManagers (TMs) in the job. To do this, set jobmanager.retrieve-taskmanager-hostname to `false`. This configuration does not affect the job's ability to connect to external services using domain names. For information about how to configure this parameter, see How do I configure custom running parameters for a job?.

Why does the "timeout expired while fetching topic metadata" error occur even if the network between Flink and Kafka is connected?

A network connection between Flink and Kafka does not guarantee that data can be read. Flink can connect to Kafka and read data only if it can connect to the endpoint described in the cluster metadata that the Kafka broker returns during the bootstrap process. For more information, see Kafka client cannot connect to broker. To check the connection, perform the following steps:

1. Use the zkCli.sh or zookeeper-shell.sh tool to log on to the ZooKeeper used by Kafka.

2. Run the ls /brokers/ids command to list all Kafka broker IDs.

3. Run the get /brokers/ids/{your_broker_id} command to view the broker metadata.

   The endpoint information is in `listener_security_protocol_map`.

4. Confirm that Flink can connect to this endpoint.

   If the endpoint uses a domain name, configure the corresponding domain name resolution service for Flink. For more information about how to resolve domain names, see How do I resolve the domain name of a service that a Flink job depends on?.