In real-time mode, you can define properties for a real-time node. These properties include node resources, time parameters, and checkpoints. This topic describes how to configure the real-time mode for a real-time node.
Access the real-time mode configuration
On the Dataphin home page, choose R&D > Data Development from the top menu bar.
From the top menu bar, select a project. If you are in Dev-Prod mode, you must also select an environment.
In the left-side menu bar, choose Data Processing > Compute Task, and then select the target compute task from the list.
In the right sidebar, click Configuration. On the configuration panel, configure the parameters on the Real-time Mode tab.
Configure real-time mode
On the real-time mode configuration panel, configure the resource and dependency parameters for the real-time node as described in the following table.
Configuration item | Description |
Resource configuration |
|
Variable configuration | Used to configure the variable parameters for a real-time computing node. For more information, see Real-time mode variable configuration. |
Checkpoint configuration | Checkpoints for real-time tasks save the task state to persistent storage. This helps you restore the task to its pre-crash state by rerunning the program after an unexpected crash. For configuration instructions, see Real-time mode checkpoint configuration. |
State configuration | State is a mechanism that Flink real-time tasks use to maintain and manage data state. By properly configuring the time-to-live (TTL) for Flink State data, you can improve resource utilization, memory management, data consistency, and fault tolerance. For configuration instructions, see Real-time mode State configuration. |
Runtime Parameter | When you develop a real-time node, you often need to configure runtime parameters to control its behavior and performance. Examples include the memory size of the JobManager process and the slot timeout in the TaskManager. |
Dependency files | If your real-time node depends on external resource files, such as text files, Python files, or Jar files, you can upload the corresponding resource files to Dataphin and add them as dependency files to the node to ensure that it runs properly. For configuration details, see Configure dependency files in real-time mode. |
Dependency | Configuring dependencies for real-time nodes helps you quickly understand the upstream and downstream nodes of the data when you troubleshoot and test. For more information, see Configure dependencies in real-time mode. |