Throttling configuration - Dataphin - Alibaba Cloud Documentation Center

If a compute engine experiences performance bottlenecks, lacks resources, or requires controlled task execution, you can configure throttling rules. These rules apply to tasks in both development and production environments and help ensure system stability and timely data output. This topic describes how to configure throttling.

Background information

Without throttling, task execution is determined by upstream dependencies, the scheduled runtime, and the availability of schedule resources. Tasks run as soon as these conditions are met, which can cause the following problems:
- Batch data backfills that cover a long period can monopolize schedule resources, which affects recurring tasks and ad hoc queries.
- Low-priority tasks might run before high-priority tasks, consuming computing resources and slowing down important tasks.
- If schedule or computing resources are limited, many tasks may be forced to wait, which can lead to a compute engine crash.
Rate limiting lets you achieve the following goals:
- Control the number of concurrent tasks sent to the compute engine. This prevents system crashes that can be caused by high concurrency or an excessive number of requests and ensures system stability.
- Ensure that high-priority tasks are given priority access to resources for execution. This ensures that data is processed in a timely and orderly manner.

Limits

You have purchased the artificial intelligence for IT operations (AIOps) value-added service, and the AIOps module has been enabled for the current tenant.
Before you configure throttling, note the following limits:
- Throttling is supported only for offline tasks. It is not supported for real-time tasks, such as real-time computing and real-time integration tasks.
- You can create a maximum of 50 throttling rules.
If you enable high availability (HA) for DataService Studio or Tag Online Service without purchasing AIOps, only one throttling rule is supported.

Permission description

The following table describes the permissions required for throttling configuration.

Operation	Role
Add, batch operations	super administrator, system administrator
Edit, delete	super administrator, system administrator, rule owner
View rule details	No permission control

Throttling configuration overview

The throttling configuration page contains an operation area, a throttling rule list, and batch operations. You can use this page to create and manage throttling rules.

Area	Description
①Operation area	Enabled: Displays throttling rules that are enabled. Search: Search by throttling rule name. Filter: Click the filter icon to filter rules by tag, owner, or effective environment. Manage rule tags: Rule tags identify and classify throttling rules. Create tags and set keywords to find rules easily. For more information, see Manage throttling rule tags. Create throttling rule: Create rules to throttle compute tasks. For specific steps, see Create a throttling rule. Refresh: Click the refresh icon to refresh the rule list.
②Throttling rule list	Displays the created throttling rules. The list includes Rule Name, Effective Environment, Effective Time, Concurrent Runs, Last Updated, Owner, Enabling Status, and available Operations. Important After you create a throttling rule, its Enabling Status is on by default. If you turn off the status, the rule becomes inactive. Details: View the details of the rule. Edit: Edit the throttling rule. You cannot change the Effective Environment. Copy: Copy the current rule to quickly create a new one. Delete: Delete the current throttling rule.
③Batch operations	Enable: Enable the selected throttling rules. After you enable them, the rules apply to all matching tasks in the current tenant. This might affect normal task execution. Confirm that you want to enable them. Disable: Disable the selected throttling rules. After you disable them, the rules become inactive. High task concurrency might pressure the compute engine and external data sources. Evaluate the impact carefully. Delete: Delete the selected throttling rules. After deletion, these rules no longer control matching tasks that have not yet run. Evaluate the impact on core data output and system pressure from high concurrency. Change owner: Change the owner for the selected throttling rules. The rule owner can edit or delete the rules they own. Changes take effect immediately. Modify tags: Modify the tags of the selected rules or clear their existing tags.

Create a throttling rule

Important

Throttling rules take effect immediately after they are configured. Therefore, you must evaluate the potential impact on your business data before you proceed.

On the Dataphin home page, go to the top menu bar and choose Develop > O&M.
In the navigation pane on the left, choose System Configuration > Throttling Configuration.
On the Throttling Configuration page, click + Create Throttling Rule.

In the Create Throttling Rule dialog box, configure the parameters.

Parameter	Description
Rule Name	Enter a custom name for the throttling rule. The name can be up to 128 characters long.
Description	Enter a brief description of the rule. The description can be up to 256 characters long.
Owner	The default owner is the user who creates the rule. Click the drop-down list to select a different owner.
Tag	Click the drop-down list to select a created tag. For more information, see Manage throttling rule tags.
Effective Environment	The environment where the rule takes effect. Options are Production Environment and Development Environment. Note The available objects for configuration differ by environment. If you switch the environment, the current rule settings are cleared. You must configure the rule again.
Rule Configuration	Global Throttling: Throttles all tasks in the target environment. Custom Rule: Throttles tasks that match the specified filter conditions. Supported task properties include Project, Schedule type, Task type, Business type, and Task name. The available conditions and values vary by task property. Project: Condition: Supports Belongs to and Does not belong to. Condition value: If Effective Environment is set to Development Environment, you can search for all Dev projects in the current tenant. If Effective Environment is set to Production Environment, you can search for all Basic and Prod projects in the current tenant. Schedule Type: Condition: Supports Equals and Does not equal. Value: Recurring Instance: Instances that are automatically generated by auto triggered tasks based on their schedules. One-time Instance: Instances generated by compute tasks, integration tasks, sync tasks, data backfill tasks for extraction, and test runs of tag-related tasks. Data Backfill Instance: An instance created by a data backfill operation. One-time Run: Includes ad hoc queries, trial runs of compute tasks, smoke testing for logical tables, and trial runs of integration tasks. Task Type: Condition: Supports Belongs to and Does not belong to. Value: Different compute engines support different task types. For the specific types supported, see the UI. Business Type: Condition: Supports Equals and Does not equal. Value: Integration Sync: Corresponds to offline integration tasks. Offline Data Processing: Corresponds to offline computing tasks, sync tasks, and Flink Batch tasks. Modeling R&D: Corresponds to all logical table tasks. This is supported only in the Enterprise Edition for Intelligent Data Development. Task name: Condition: Supports Equals, Does not equal, Contains, Does not contain, Belongs to, Does not belong to, Starts with, Does not start with, Ends with, and Does not end with. Value: Enter any filter value. If the condition is Belongs to or Does not belong to, you can enter multiple task names. Enter one task name per line.
Concurrent Runs	Set the number of concurrent runs based on your throttling scenario. Note A concurrency of 0 means no tasks are run.
Effective Time	Set the effective period for the throttling rule. You can select All Time, Specific Time Range, or Specific Time Interval. All Time: The rule is always active. Specific Time Range: The rule is active within the specified date and time range. If the system time zone (in User Center) is different from the schedule time zone, both are displayed. When you select a time range, the system automatically calculates the corresponding time in the schedule time zone. For example, if the system time zone is GMT+08:00 and the schedule time zone is GMT+01:00, a specified range of 2025-06-10 00:00:00 to 2025-06-10 01:00:00 corresponds to `2025-06-09 17:00:00 ~ 2025-06-09 18:00:00` in the schedule time zone. Note This is useful for scenarios that require throttling during a specific time range. Specific Time Interval: The rule is active during a specific time interval each day within a selected date range. For example, `00:00:00 to 11:00:00 from 2022-09-11 to 2022-09-12`. If the system time zone is different from the schedule time zone, both are displayed. When you select a start and end time, the system calculates the corresponding time in the schedule time zone. If the calculated time falls on a different day, `(-1)` indicates the previous day and `(+1)` indicates the next day. For example, if the system time zone is GMT+08:00 and the schedule time zone is GMT+01:00, a specified interval of `00:00 to 01:00 from 2025-06-10 to 2025-06-11` corresponds to `17:00(-1) to 18:00(-1) from 2025-06-10 to 2025-06-11` in the schedule time zone. Note This is useful for scenarios that require throttling during a specific time interval each day.

Click OK to create the throttling rule.

Manage throttling rule tags

Throttling rule tags are used to identify and classify rules. You can create tags and set keywords to easily find specific rules. You can create and manage tags in the Manage Throttling Rule Tags dialog box.

Area	Description
①Operation area	Search: Search by tag name. Create tag: Follow these steps to create a tag: Click Create Tag. In the input fields, enter the tag parameters. Tag Name: Enter a name for the tag, up to 128 characters. Description: Enter a brief description of the tag, up to 128 characters. Click the confirm icon to create the tag.
②Tag list	For created tags, you can perform Edit and Delete operations.