Dataphin:Ververica Flink real-time task resource customization

Ververica Flink real-time task resource customization entry

1. On the Dataphin home page, navigate to the top menu bar and select Development > Data Development.

2. At the top menu bar of the data Development page, choose Project.

3. In the left-side navigation pane, select Data Processing > Script Task. Then, in the right-side computing task list, click the desired task.

4. Under the selected computing task tab, click Configuration in the right sidebar to open the configuration panel.

5. In the configuration panel, select the Real-time Mode tab. Under the Real-time Mode tab, in the Resource Configuration section, set the Resource Configuration Type to Custom Configuration.

6. Click Go To Configure after selecting Custom Configuration.

Procedure

The resource configuration page displays a topology diagram, where each box represents an independently configurable computing task. Each Group indicates that the nodes within it can be placed on a single machine for computation, minimizing network data transmission and enhancing performance. The diagram's current resource configuration is the system's default recommendation for you.

1. Click the icon in the upper right corner of the Group to configure its resources. Configure the parameters in the Custom Configuration Group Runtime Parameters and Custom Configuration StreamExecCalc Runtime Parameters dialog boxes.

   • Custom Configuration Group Runtime Parameters

     Parameter	Description
     CPU	Specifies the CPU resources for the current node's operation.
     Heap Memory	Heap Memory refers to the memory used by Java applications, measured in MB or GB. It can be controlled through JVM command line parameters. For Blink programs, a portion of Heap Memory is typically required for overhead, such as caching. Adjust its size based on the program's scale. Heap Memory size and its components are adjustable via JVM command line parameters. Typically, Blink programs necessitate a specific amount of Heap Memory overhead, for instance, allocating Heap Memory for program cache. Consequently, its size should be configured based on the program's scale.
     Off-heap Memory	Sets the size of off-heap memory, measured in MB or GB.
     Concurrency	Batch update the concurrency for all OPERATOR nodes under the current SLOT node. Click the SLOT node (outermost layer) Edit icon to configure concurrency in the pop-up dialog box. Accepts positive integers between 1 and 32768. If left empty, no update is performed. Accepts positive integer inputs ranging from 1 to 32768. If left empty, no update will be carried out. Note Note: Only SLOT nodes support batch modification of concurrency.

   • Custom Configuration StreamExecCalc Runtime Parameters

     Parameter	Description
     Concurrency	The default concurrency is 1. Enter any integer greater than 0, or -1 for automatic inference.
     Chaining Policy	ALWAYS: Ensure chaining with both upstream and downstream operators. HEAD: The current operator serves as the head node in the chain, breaking only the links with upstream operators, while maintaining the chain with downstream nodes. NEVER: This operator does not chain with either upstream or downstream operators.
     Operator State Expiration Time Setting	If the current OPERATOR node is stateful, it allows for setting individual expiration times for each state. You can set the expiration time in days, hours, minutes, or seconds.

2. Once the parameter configuration is complete, click OK.

3. After configuring, click Save in the upper right corner of the page.

   Important

   To revert to the system-recommended resource configuration, click Reset To System Initial in the upper right corner of the page.

   Additionally, you have the option to save resource configuration records. To do this, click Save Current Configuration As New Record following the Resource Information Record. In the Save Resource Record dialog box, provide a Resource Record Name and then click OK.

   For existing resource information records, you can perform the following operations:

   Operation	Description
   View Version Information	To view version information, click the Actions icon under the record's column.
   Enable Record	Click the Actions icon under the record's column. In the Prompt dialog box, click OK.
   Delete Record	Click the Actions icon under the record's column. In the Prompt dialog box, click OK.