Group LLM data processing (DLC) components in Machine Learning Designer to run them as a single batch job — no intermediate data is written to storage between components, which reduces I/O overhead and speeds up your pipeline.
How it works
A group acts as a single execution unit that wraps multiple LLM-DLC components. When the group runs, all enclosed components execute in sequence without persisting intermediate results to storage. The group provides a unified set of configurations and produces a collective output consumed by downstream components.
-
LLM Data Processing (DLC)
-
LVM Data Processing (DLC)
Limitations
-
Currently, LLM-Risk Content Filtering (DLC) and LLM-Quality Predict (DLC) under the Large Model Data Preprocessing folder do not support grouping.
-
Some components do not support multi-node distributed operation. If a group contains such components, multi-node tasks fail. To check whether a component supports multi-node distributed operation, open the component's Tuning tab. If the Nodes parameter can be set to a value greater than 1, the component supports multi-node distributed operation.
Configuration hierarchy
Group-level configurations take precedence over individual component configurations. Within a group, settings are split across two levels:
| Level | What to configure |
|---|---|
| Group level | Text fields, image fields, video fields, computing resources, data output paths |
| Component level | Tuning parameters specific to each component |
Component-level tuning parameters are not overridden by the group. Configure them individually on each component.
Group output behavior
Grouped components do not produce individual outputs. The group emits a single collective output that combines the results of all enclosed components. In the example below, the connection originates from the center of the group, indicating that the output includes results from both LLM-Text Normalizer (DLC) and LLM-Clean Special Content (DLC)-1.
Create a group
Intelligent aggregation
Machine Learning Designer automatically detects canvas nodes that can be grouped. Click
to aggregate them into a group, then click
to configure resources for the group.
The Custom Experiment Settings panel contains the following fields:
-
Alink Group Name (for example,
LLMDataProcessGroup1) -
Computing Resource Type (DLC)
-
Resource group (Public resource group)
-
Node Type (for example, CPU 4vCPU+16GB Mem ecs.g6.xlarge)
-
Nodes
-
VPC configuration
-
Maximum runtime (seconds)
-
Text field (name of the text field to process)
-
Data output OSS path (required; OSS directory for storing processed output)
Manual aggregation
Click
or use Shift+left-click to select multiple components. Right-click in an empty area and choose Group Selected Nodes. Then click
to configure resources for the group.
In the Custom Experiment Settings panel, configure the following parameters:
-
Alink Group Name: Enter a custom group name, for example,
LLMDataProcessGroup1. -
Computing Resource Type: Select DLC.
-
Resource group: Select a public resource group or a dedicated resource group.
-
CPU/GPU: Select CPU or GPU.
-
Node Type: Select a computing instance specification, for example, 4vCPU+16GB Mem ecs.g6.xlarge.
-
Nodes: Set the number of nodes.
-
VPC: Select a VPC.
-
Maximum runtime (seconds): Set the maximum runtime duration.
-
Text field: Enter the text field name, for example,
text. -
Data output OSS path: Set the data output path.