This topic outlines the process for configuring resources in Ververica Flink's unified stream-batch tasks for offline mode, including setting up production and development resource clusters, engine versions, and specifying the task's degree of parallelism, Task Manager quantity, Job Manager Memory, and Task Manager Memory.
Procedure
Navigate to the Development option in the top menu bar from the Dataphin home page and click it. You will be directed to the Data Development page by default.
To access the configuration panel for the unified stream-batch task offline mode, follow these steps:
Select the project (environment selection is required for Dev-Prod mode) -> click Script Task -> choose and click the compute task -> click Configure -> click Offline Mode.
In the Resource Configuration section of the configuration panel, set the Resource Configuration parameters for the task's offline mode.
Parameter
Description
Production Environment Cluster
The default cluster is the one selected during the creation of the Flink SQL task. You have the option to switch clusters if necessary.
ImportantBe cautious when switching clusters as it may affect task execution post-submission.
Production Environment Engine Version
The default engine version is the one selected during the creation of the Flink SQL task. You have the option to switch engine versions if necessary.
Development Environment Cluster And Engine Version
Choose between the system default configuration and a custom configuration.
System Default Configuration: Automatically uses the same cluster and engine version as the production environment.
Custom Configuration: Allows you to select a different cluster and engine version for the development environment task.
NoteThis configuration is not necessary if your project space is in Basic mode.
Parallelism
Set the task's degree of parallelism. The default is 1, but you can specify any integer greater than 0. Use -1 for automatic inference.
Task Manager Quantity
By default, this matches the degree of parallelism, and you can enter any integer greater than 0.
Job Manager Memory
This is the memory allocation for the Flink cluster's Job Manager. The default is 1Gi, and it is recommended to use Gi/Mi units. You can enter a value in bytes or include the memory units Gi/Mi.
Task Manager Memory
This is the memory allocation for the Task Manager in the Flink cluster. The default is 1Gi, and it is recommended to use Gi/Mi units. You can enter a value in bytes or include the memory units Gi/Mi.
Click OK to finalize the resource configuration for the task's offline mode.