The LLM data processing components can be grouped to support batch execution and minimize data storage configurations, thus significantly boosting performance.
Overview
The LLM data processing (DLC) components can be grouped to execute batch tasks without intermediate data storage, thus enhancing efficiency and resource utilization.
LLM data processing (DLC) components in Machine Learning Designer include:
Currently, the LLM-Risk Content Filtering (DLC) and LLM-Quality Predict (DLC) components under the Large Model Data Preprocessing folder do not support grouping.
Usage notes
Consider the following when aggregating components into groups:
The configurations of the group take precedence over the individual configuration of components within the group.
Currently, the LLM-Risk Content Filtering (DLC) and LLM-Quality Predict (DLC) components under the Large Model Data Preprocessing folder do not support grouping.
The tuning parameters of the components should be configured individually, while global fields like text, image, video fields, computing resources, and data output paths should be set at the group level.
Some components may not support multi-node distributed operation. If a group includes such components, multi-node tasks will fail. To check whether a component supports multi-node distributed operation, go to the Tuning tab of the component. If the Nodes parameter can be set to a value larger than 1, the component supports multi-node distributed operation.
Grouped components will not output results individually. The group output is a collective result of all components. For example, the workflow below shows a connection that starts from the center of the group, indicating that the group output includes the outputs from both LLM-Text Normalizer (DLC) and LLM-Clean Special Content (DLC)-1.
Group components
Intelligent aggregation
The system can automatically detect nodes on the canvas suitable for grouping. Click 
to aggregate them, then click 
to configure resources for the group.
Manual aggregation
Click 
or use Shift+left-click to select multiple components. Then, right-click in an empty area and choose Group Selected Nodes. Click to configure resources for the group.
