The LVM-Image-NSFW Filter (DLC) component scores images for Not Safe For Work (NSFW) content and removes images that exceed a configured threshold. Use it as a data-cleaning step before training image generation models to improve training data quality.
How it works
The component reads image data from Object Storage Service (OSS) — either from a directory path or a metadata file.
It calculates an NSFW score for each image.
Images with a score above the configured NSFW Score Threshold are filtered out.
The retained images and statistics are written to the specified OSS output directory.
Prerequisites
Before you begin, ensure that you have:
An OSS bucket containing the image data to filter
An OSS directory available to store the output
A GPU instance type available in your resource group (required by the scoring algorithm)
Input ports
The component accepts input from one of the following sources:
| Scenario | Input source |
|---|---|
| First run (no upstream component) | Set the Image Data OSS Path parameter to point to the OSS directory containing your images |
| Subsequent runs | Set Image Data OSS Path to the meta.jsonl file generated in the parent directory on the first run |
| Pipeline integration | Connect any image data preprocessing component as the upstream input |
When the component runs for the first time with a directory path, it generates a meta.jsonl metadata file in the parent directory of that path. On subsequent runs, pass meta.jsonl directly to skip re-scanning the directory.
Output port
The component writes filtering results to the OSS directory specified by Output File OSS Path. The output includes:
| File | Description |
|---|---|
{name}.jsonl | The filtered output dataset. The filename is set by Output Filename (default: result.jsonl) |
{name}_stats.jsonl | The state file |
dj_run_yaml.yaml | The YAML configuration used when the algorithm ran |
Configure the component
Configure the following parameters in Machine Learning Designer.
Field settings
| Parameter | Required | Default | Description |
|---|---|---|---|
| Image Data OSS Path | No | — | The OSS directory containing your images, or the meta.jsonl file from a previous run. See Input ports for when to use each. |
| Output File OSS Path | Yes | — | The OSS directory where filtering results are stored. |
| Output Filename | Yes | result.jsonl | The base name for the output file. The stats file uses the same base name with a _stats suffix. |
Parameter settings
| Parameter | Required | Default | Description |
|---|---|---|---|
| NSFW Score Threshold | Yes | 0.5 | Images with an NSFW score above this value are filtered out. A lower threshold is stricter and retains fewer images; a higher threshold is more permissive and retains more images. |
Execution tuning
| Parameter | Required | Default | Description |
|---|---|---|---|
| Number of Processes | Yes | 4 | The number of processes. |
Select resource group
| Parameter | Required | Default | Description |
|---|---|---|---|
| Public Resource Group | No | — | The instance type (CPU or GPU) and virtual private cloud (VPC) to use. Select a GPU instance type — the scoring algorithm requires GPU compute. |
| Dedicated resource group | No | — | The number of vCPUs, memory, shared memory, and GPUs to allocate. |
| Maximum Running Duration (seconds) | No | — | The job is terminated if it runs longer than this value. |
You must select the GPU instance type for the algorithm. Select a GPU instance type under Public Resource Group or configure a GPU-enabled dedicated resource group.