In real-time mode for Alibaba Blink stream-batch unified tasks, you can configure the resource queue, engine version, resource configuration type, and auto-tuning. This topic describes how to configure these resources.
Procedure
In the top menu bar of the Dataphin home page, choose Development > Data Development.
In the top menu bar, select a Project. If you are in Dev-Prod mode, also select an Environment.
In the navigation pane on the left, choose Data Processing > Script Task. In the list of compute tasks on the right, click the target compute task.
In the sidebar on the right, click Configure. In the panel that appears, click the Real-time Mode tab.
In the Resource Configuration section of the real-time mode configuration panel, configure the Resource Configuration settings.
Parameter
Description
Resource Queue
The default is the resource queue selected when you created the Blink SQL task. You can switch the resource queue here.
ImportantTask execution may differ after you switch the queue. The change takes effect after you submit the task. Proceed with caution.
Engine Version
The default is the engine version selected when you created the Blink SQL task. You can switch the engine version here.
Resource configuration type
System-recommended configuration
This is the intelligent mode (BETA) for fine-grained resource configuration in real-time computing Flink. Intelligent mode is an upgrade to expert mode. In intelligent mode, a job uses the resource configuration from expert mode and enables the AutoPilot auto-tuning feature.
In auto configuration mode, you do not need to configure resources. Autopilot automatically generates resource configurations for the job and tunes them based on the job's runtime status. This optimizes resource usage while ensuring the job remains healthy. For more information about Autopilot, see Configure auto-tuning.
Custom configuration
This is the expert mode (BETA) for fine-grained resource configuration in real-time computing Flink. Fully managed Flink introduces this new resource configuration mode. It supports fine-grained control over job resources to meet throughput requirements.
The system automatically runs jobs in Native K8s mode based on your configured resource requirements, and automatically determines the specifications and number of TMs based on Slot specifications and job concurrency. For more information about configuring fine-grained resources, see Configure job deployment information.
To customize Blink SQL task resources in Dataphin, see Customize Alibaba Blink real-time mode resources.
Resource configuration record
When Resource Configuration information changes, click Save current configuration as new record. For a saved resource configuration record, you can view details, start, or delete it.
Auto-tuning
Click to enable auto-tuning. You can set the Maximum CUs and Expected maximum memory for the current task at runtime.
Maximum CUs: The upper limit of compute unit (CU) resources available to the job. 1 CU = 1 Core + 4 GB memory. The maximum number of CUs must be less than the number of CUs available for the project.
Expected maximum memory: The upper limit of memory resources that the job is expected to use.
Click OK to complete the resource configuration for the real-time mode.