Filters videos with high NSFW (Not Safe For Work) scores from your dataset. Only MP4 videos are supported.
Supported computing resources
How it works
Samples frames from each video and calculates NSFW scores per frame. Videos exceeding the score threshold are filtered out. Typically used to curate training data for video generation models.
Inputs and outputs
Input ports
-
Use the Read File Data component to read training data from OSS.
-
Configure OSS Data Path to select the OSS directory containing video data or the video metadata file. See the parameter table below.
-
Any LVM Data Processing (DLC) component can serve as input.
Output port
The filtering results. See the parameter table below for output details.
Configure the component
Configure the LVM-NSFW Filter (DLC) component in Machine Learning Designer with the following parameters.
|
Tab |
Parameter |
Required |
Description |
Default value |
|
|
Field Settings |
Video Data OSS Path |
No |
If no upstream component exists on first run, manually select the OSS directory storing video data. The component generates a video_meta.jsonl metadata file in the OSS bucket root directory. For subsequent runs, select video_meta.jsonl directly. |
No default value |
|
|
Output File OSS Path |
Yes |
The OSS directory for filtering results. Output files:
|
No default value |
||
|
Output Filename |
Yes |
File name for the filtering results. |
result.jsonl |
||
|
Parameter Settings |
NSFW Score Threshold |
Yes |
The maximum NSFW score. |
0.5 |
|
|
Number of Sampled Frames |
Yes |
Number of frames to sample. Frames are evenly distributed across the video duration. |
1 |
||
|
Execution Tuning |
Number of Processes |
Yes |
The number of processes. |
4 |
|
|
Select Resource Group |
Public Resource Group |
No |
Instance type (CPU or GPU), number of instances, and VPC. |
No default value |
|
|
Dedicated resource group |
No |
vCPUs, memory, shared memory, GPUs, and number of instances. |
No default value |
||
|
Maximum Running Duration (seconds) |
No |
Maximum runtime in seconds. The job terminates if this limit is exceeded. |
No default value |
||