Dataphin:Runtime settings

Background information

By default, Dataphin assigns an independent container to each Shell or Python task to prevent resource or status conflicts, such as global environment variable modifications or local shared file removals. However, running many tasks concurrently can lead to high resource consumption, potentially delaying the scheduling of tasks that share runtime resources, such as SQL tasks, and affecting business data timeliness. Allowing multiple Shell and Python tasks that use the same offline computing template to share runtime resources can improve resource efficiency.

For security and access control, the operations tenant must first activate the global switch for runtime resource sharing in the runtime configuration section of the Management Center. The super administrator can then decide whether code tasks based on each template share runtime resources. Because tasks from the same template typically have similar scenarios and consistent parameters, the risks of sharing runtime resources can be effectively managed. For instructions on enabling the shared runtime resource switch for offline computing templates, see Step 3: Configure offline computing templates.

Permissions

Only super administrators and system administrators of the metadata warehouse tenant can toggle the global shared resource switch.

Precautions

Before you enable the shared resource switch, assess data security and verify that no conflicts will arise from tasks sharing runtime resources to prevent abnormal task execution.

Procedure

1. Navigate to the Dataphin home page and single click the Management Center from the top menu bar.

2. Refer to the figure below, on the Runtime Settings page, to enable the global shared resource switch for Shell and Python template tasks.

   

3. To disable the global shared resource switch for Shell and Python template tasks, switch it off manually or use the one-click Restore Default Values option.

4. To apply the changes, single click Save.

   Important

   Configurations will not affect running instances generated before each change.