Dataphin:Runtime settings

Background information

Dataphin assigns independent containers to each Shell or Python task by default to prevent resource or status conflicts, such as global environment variable modifications or local shared file removals. However, this can lead to high resource consumption when numerous tasks run concurrently, potentially delaying the scheduling of tasks that share runtime resources, like SQL tasks, and affecting business data timeliness. Sharing runtime resources among multiple Shell and Python tasks that use the same offline computing template can enhance resource efficiency.

For security and access control, the operations tenant must first activate the global switch for runtime resource sharing in the runtime configuration section of the Management Center. Once activated, the super administrator can decide whether code tasks based on each template should share runtime resources. Since tasks from the same template typically have similar scenarios and consistent parameters, the risks associated with sharing runtime resources can be effectively managed and controlled. For instructions on enabling the shared runtime resource switch for offline computing templates, see Step 3: Configure offline computing templates.

Permissions

Only super administrators and system administrators of the metadata warehouse tenant have the authority to toggle the global shared resource switch.

Precautions

Prior to enabling the shared resource switch, it is crucial to assess data security and verify that no conflicts will arise from tasks sharing runtime resources to prevent abnormal task execution.

Procedure

1. Navigate to the Dataphin home page and single click the Management Center from the top menu bar.

2. Refer to the figure below, on the Runtime Settings page, to enable the global shared resource switch for Shell and Python template tasks.

   

3. To disable the global shared resource switch for Shell and Python template tasks, switch it off manually or use the one-click Restore Default Values option.

4. To apply the changes, single click Save.

   Important

   Configurations will not affect running instances generated before each change.