Live Video Stream Moderation API v2.0 - Content Moderation

ApsaraVideo Live stream moderation 2.0 helps you detect risky content in the video and audio of live streams. This topic describes how to use API operations to perform live stream moderation.

Onboarding guide

Register an Alibaba Cloud account and follow the instructions to complete the registration.
Enable pay-as-you-go for Content Moderation. Make sure that you have enabled the service. For more information, see Enable Service. No fees are charged for activating the service. After you use the API, the system automatically bills you based on your usage. For more information, see Billing description.
Create an AccessKey in the RAM console. For more information, see Create an AccessKey. If you use the AccessKey of a Resource Access Management (RAM) user, you must use your Alibaba Cloud account to grant the AliyunYundunGreenWebFullAccess permission to the RAM user. For more information, see Grant permissions to a RAM user.
Develop and integrate the service. We recommend that you call the operation using a software development kit (SDK). For more information, see Video Moderation SDK v2.0 and integration guide.
The live video stream moderation service includes the following three API operations:
- VideoModeration: Submit a live stream moderation job
- VideoModerationResult: Query the result of a live stream moderation job
- VideoModerationCancel: Cancel a live stream moderation job

Submit a moderation job

API description

API operation: VideoModeration. This operation supports only asynchronous moderation for live video streams.

Supported regions and endpoints:

Region	Public endpoint	VPC endpoint	Supported services
Singapore	green-cip.ap-southeast-1.aliyuncs.com	green-cip-vpc.ap-southeast-1.aliyuncs.com	liveStreamDetection_global, liveStreamDetectionByVL_global
US (Virginia)	https://green-cip.us-east-1.aliyuncs.com	https://green-cip-vpc.us-east-1.aliyuncs.com	liveStreamDetection_global
US (Silicon Valley)	https://green-cip.us-west-1.aliyuncs.com	Not available
Germany (Frankfurt)	green-cip.eu-central-1.aliyuncs.com	Not available

Billing:
This is a paid operation. You are charged based on the moderation policies that you configure for live video frames and audio. For live video frames, you can select multiple services. The total fee is calculated by multiplying the number of video snapshots by the unit price of each service. If you also moderate the audio in the live stream for violations, you are charged an additional fee. This fee is calculated by multiplying the video duration by the unit price of the audio moderation feature. For more information about billing methods, see Billing.
Detection object: Live video streams.
Response: Asynchronous moderation jobs do not return results in real time. You must obtain the moderation result using a callback or polling.
- Use a callback to obtain the moderation result: When you submit an asynchronous moderation job, you can specify the callback parameter in the request to automatically receive the moderation result.
- Use polling to obtain the moderation result: When you submit an asynchronous moderation job, do not specify the callback parameter. After you submit the job, you can call the result query operation to obtain the moderation result.
Live stream requirements:
- Supported protocols: RTMP, HLS, HTTP-FLV, and RTSP.
- Live stream duration limit: A single video stream moderation job can last for a maximum of 24 hours. A job automatically ends if it exceeds this limit.

Detection rule configuration:

For your first call, you can configure live stream moderation rules in the Content Moderation console.

If you do not set any rules, the Live Stream Moderation V2.0 API uses the following default configurations:

Service	Default configurations
Live stream detection (liveStreamDetection_global)	Fixed frame capture frequency: 1 frame/second Video frame detection service: General baseline detection (baselineCheck_global) Video audio detection: Enabled Video audio detection service: Social and entertainment live stream detection (live_stream_detection_global) Result return method: Returns only results with detected threats
Live stream detection (large model) (liveStreamDetectionByVL_global) Note Currently supported only in the Beijing and Shanghai regions. The large model version is limited to 10 concurrent ingest endpoints.	Fixed frame capture frequency: 1 frame/second Video frame detection service: Image moderation service with large and small models (postImageCheckByVL) Video audio detection: Enabled Video audio detection service: Social and entertainment live stream detection for regions outside China (live_stream_detection_cb) Result return method: Returns only results with detected threats

QPS limits

The queries per second (QPS) limit for this operation is 100 for each user. The number of concurrent moderation streams is limited to 50. This means a maximum of 50 jobs can be processed at the same time. To increase the limit, contact your business manager. If the limit is exceeded, the API call is throttled. This may affect your business. Call the operation in a reasonable manner.

Debugging

Before integration, you can also use Alibaba Cloud OpenAPI to debug the VideoModeration API online, view sample code and SDK dependency information, and obtain an overview of the API methods and parameters.

Important

The online debugging feature uses the currently logged-on account to call the Content Moderation API operations. Therefore, the number of calls is included in the billable usage of the account.

Request parameters

Name

Type

Required

Example

Description

Service

String

Yes

liveStreamDetection_global

The type of the moderation service. Valid values:

liveStreamDetection_global: live video stream moderation
liveStreamDetectionByVL_global: live video stream moderation (large model edition)

ServiceParameters

JSONString

Yes

The parameters required by the moderation service. This parameter is a JSON string. For more information about the parameters, see Table 1. ServiceParameters.

Table 1. ServiceParameters

Name	Type	Required	Example	Description
url	String	Yes	http://www.aliyundoc.com/a.flv	The URL of the object to moderate. Make sure that the URL is accessible over the Internet. The URL can be up to 2,048 characters in length. Note The URL cannot contain Chinese characters. Make sure that you specify only one URL in each request.
callback	String	No	http://www.aliyundoc.com	The URL to which the moderation result is sent in a callback. HTTP and HTTPS are supported. If you leave this parameter empty, you must poll for the moderation result. The callback interface must support POST requests, UTF-8 encoded data, and the checksum and content form parameters. Content Moderation sets the checksum and content parameters based on the following rules and format, and calls your callback interface to return the moderation result. checksum: A string that is generated by applying the SHA256 algorithm to a new string created by concatenating `user UID + seed + content`. The user UID is your Alibaba Cloud account ID, which you can find in the Alibaba Cloud Management Console. To prevent tampering, when you receive the pushed result, you can generate a string using the same algorithm and verify it against the checksum. Note The user UID must be the UID of an Alibaba Cloud account, not the UID of a RAM user. content: The value is a JSON-formatted string. You must parse and invert the string to obtain a JSON object. For an example of the content result, see the sample response for a detection query. Note After your callback server receives the result pushed by Content Moderation, if the returned HTTP status code is 200, the result is received successfully. Other HTTP status codes indicate a failure. If the result fails to be received, Content Moderation retries pushing the moderation result up to 16 times until the result is successfully received. If the result is still not received after 16 retries, the result is no longer pushed. We recommend that you check the status of your callback interface.
seed	String	No	abc****	A random string used for the signature in the callback notification request. The string can contain letters, digits, and underscores (_), and can be up to 64 characters in length. You can customize this value to verify that the callback notification request is initiated by the Alibaba Cloud Content Moderation service. Note This parameter is required when you use a callback.
cryptType	String	No	SHA256	The encryption algorithm for the signature in the callback notification. Content Moderation calculates the signature for the returned result (the `user uid + seed + content` string) based on the encryption algorithm you specify, and then sends the signature to your callback URL. Valid values: SHA256 (default): uses the SHA256 encryption algorithm. SM3: uses the Chinese cryptographic HMAC-SM3 algorithm. A hexadecimal string consisting of lowercase letters and digits is returned. For example, after `abc` is encrypted using the SM3 algorithm, `66c7f0f462eeedd9d1f2d46bdc10e4e24167c4875cf2f7a2297da02b8f4ba8e0` is returned.
dataId	String	No	videoId****	The data ID of the object. The ID can contain uppercase letters, lowercase letters, digits, underscores (_), hyphens (-), and periods (.), and can be up to 128 characters in length. You can use this ID to uniquely identify your business data.
liveId	String	No	liveId****	The ID of the live video stream. This parameter is used to remove duplicate live video stream moderation jobs and prevent repeated moderation. If you specify this parameter, the system checks whether a live stream moderation job is in progress based on the `uid+service+liveId` combination. If a job is in progress, the system returns the taskId of the existing job and does not start a new one.

Note

After your callback server receives the result pushed by Content Moderation, an HTTP status code of 200 indicates that the result was received successfully. Other HTTP status codes indicate a failure. If the result is not received, Content Moderation retries pushing the moderation result up to 16 times. If the result is still not received after 16 retries, Content Moderation stops pushing the result. We recommend that you check the status of your callback interface.

Response parameters

Name	Type	Example	Description
Code	Integer	200	The status code. For more information, see Code description.
Data	JSONObject	{"TaskId": "AAAAA-BBBBB"}	The moderation result data.
Message	String	OK	The response message for the request.
RequestId	String	ABCD1234-1234-1234-1234-123****	The request ID.

Examples

Sample request

{
    "Service": "liveStreamDetection_global",
    "ServiceParameters": {
        "url": "http://www.aliyundoc.com/a.flv",
        "dataId": "videoId****",
        "liveId": "liveId****"
    }
}

Sample success response

{
    "Msg": "OK",
    "Code": 200,
    "Data":
    {
        "TaskId": "AAAAA-BBBBB",
        "DataId": "videoId****"
    },
    "RequestId": "ABCD1234-1234-1234-1234-123****"
}

Query the result of a live stream moderation job

API description

API operation: VideoModerationResult.

Billing: This operation is not billed.
Query timeout: You can set the query interval to 30 seconds. This means you can query the result 30 seconds after you submit the asynchronous moderation job. The maximum query period is 24 hours. After 24 hours, the result is automatically deleted.
Result return: If the live stream is in progress, the last 10 moderation results are returned for each query. If the live stream has ended, all moderation results are returned.

QPS limits

The QPS limit for this operation is 100 for each user. If the limit is exceeded, the API call is throttled. This may affect your business. Call the operation in a reasonable manner.

Debugging

Before integration, you can also use Alibaba Cloud OpenAPI to debug the VideoModerationResult API online. This lets you view sample code and SDK dependency information and obtain an overview of the API methods and parameters.

Request parameters

Name

Type

Required

Example

Description

Service

String

Yes

liveStreamDetection_global

The type of the moderation service. The value must be the same as the service type you specified when you submitted the moderation job.

ServiceParameters

JSONString

Yes

The parameters required by the moderation service. This parameter is a JSON string. For more information about the parameters, see Table 1. ServiceParameters.

Table 1. ServiceParameters

Name	Type	Required	Example	Description
taskId	string	Yes	abcd****	The taskId of the moderation job to query. You can specify only one taskId at a time. Note You can obtain the taskId of a moderation job from the response data after you submit the job.

Response parameters

Name	Type	Example	Description
RequestId	String	ABCD1234-1234-1234-1234-123****	The ID of the request. This is a unique identifier generated by Alibaba Cloud for the request and can be used for troubleshooting.
Data	Object		The result of the live stream content moderation. For more information, see Table 2. Data.
Code	String	200	The status code. For more information, see Code description.
Message	String	OK	The response message for the request.

Table 2. Data

Name	Type	Example	Description
DataId	String	videoId****	The data ID of the object. Note If you specified the dataId parameter in the moderation request, the corresponding DataId is returned.
LiveId	String	liveId****	The ID of the live stream. Note If you specified the liveId parameter in the moderation request, the corresponding LiveId is returned.
TaskId	String	AAAAA-BBBBB	The ID of the moderation job.
RiskLevel	String	high	The risk level of the live stream. The value is determined based on the moderation results of both the video frames and audio. Valid values: high: high risk medium: medium risk low: low risk none: no risk detected Note We recommend that you handle high-risk content directly, manually review medium-risk content, and handle low-risk content only when high recall is required. In other cases, we recommend that you handle low-risk content in the same way as content with no detected risks. You can configure the risk scores for video frames in the Content Moderation console.
FrameResult	JSONObject		The moderation result of the video frames. If the call is successful (code=200), a struct is returned. For more information about the struct, see Table 3. FrameResult. Note For video stream moderation, the status code 280 indicates that the moderation is in progress, and 200 indicates that the moderation is complete. If the moderation is in progress, the moderation result contains the results detected from the start of the moderation to the current time.
AudioResult	JSONObject		The moderation result of the audio. A struct is returned. For more information about the struct, see Table 8. audioResult.

Table 3. FrameResult

Name	Type	Example	Description
FrameNum	Integer	200	The number of snapshots returned for the video frames.
FrameSummarys	JSONArray		A summary of the labels for the video frames. For more information about the struct, see Table 4. FrameSummarys.
RiskLevel	String	high	The risk level of the video frames. The value is calculated based on all live stream snapshots. Valid values: high: high risk medium: medium risk low: low risk none: no risk detected
Frames	JSONArray		Information about the live stream frames that hit a label. For more information about the struct, see Table 5. Frames.

Table 4. FrameSummarys

Name	Type	Example	Description
Label	String	violent_armedForces	The label of the video frame.
Description	String	Suspected to contain fireworks	The description of the Label field. Important This field provides an explanation of the Label field and may be subject to changes. We recommend that you handle the results based on the Label field, not this field.
LabelSum	Integer	8	The number of times the label appears.

Table 5. Frames

Name	Type	Example	Description
TempUrl	String	http://www.aliyundoc.com/test.jpg	The temporary URL of the live stream frame. The URL is valid for 30 minutes. Save the frame promptly. Note If video evidence storage is enabled, the OSS URL of the stored video frame is returned.
Offset	Float	50.5	The timestamp of the live stream frame relative to the start of the live stream. Unit: seconds.
Timestamp	Integer	1684559739000	The absolute timestamp when the frame was processed. Unit: milliseconds.
RiskLevel	String	high	The risk level of the live stream frame, returned based on the configured high and low risk scores. Valid values: high: high risk medium: medium risk low: low risk none: no risk detected Note We recommend that you handle high-risk content directly, manually review medium-risk content, and handle low-risk content only when high recall is required. In other cases, we recommend that you handle low-risk content in the same way as content with no detected risks. You can configure the risk scores for video frames in the rule configuration of the Content Moderation console.
Results	JSONArray		The results of the live stream frame moderation, including risk labels and confidence scores. For more information, see Table 6. Results.

Table 6. Results

Name	Type	Example	Description
Service	String	baselineCheck_global	The frame moderation service that is called.
Result	Array		The results of the live stream frame moderation, including risk labels and confidence scores. For more information, see Table 7. Result.

Table 7. Result

Name	Type	Example	Description
Label	String	violent_explosion	The label returned after the live stream frame is moderated. A frame may have multiple labels and scores. The following labels are supported: General baseline detection (baselineCheck_global) supported labels
Confidence	Float	81.22	The confidence score, ranging from 0 to 100. The value is accurate to two decimal places.
Description	String	Suspected to contain fireworks	The description of the Label field. Important This field provides an explanation of the Label field and may be subject to changes. We recommend that you handle the results based on the Label field, not this field.

Table 8. audioResult

Name	Type	Example	Description
AudioSummarys	JSONArray		A summary of the audio labels. For more information about the struct, see Table 9. AudioSummarys.
RiskLevel	String	high	The risk level of the audio. The value is calculated based on all audio segments. Valid values: high: high risk medium: medium risk low: low risk none: no risk detected
SliceDetails	JSONArray		The details of the text converted from the audio. Each element corresponds to a sentence. The value contains one or more elements. For more information about the struct, see Table 10. SliceDetails.

Table 9. AudioSummarys

Name	Type	Example	Description
Label	String	profanity	The label of the video audio.
LabelSum	Integer	8	The number of times the label appears.

Table 10. SliceDetails

Name	Type	Example	Description
StartTime	Integer	0	The start time of the sentence. Unit: seconds.
EndTime	Integer	4065	The end time of the sentence. Unit: seconds.
StartTimestamp	Integer	1678854649720	The start timestamp of the segment. Unit: milliseconds.
EndTimestamp	Integer	1678854649720	The end timestamp of the segment. Unit: milliseconds.
Text	String	disgusting	The text content converted from the audio.
Url	String	https://aliyundoc.com/test.wav	If the moderated content is an audio stream, this parameter indicates the temporary URL of the audio stream corresponding to the text. The URL is valid for 30 minutes. Save the audio stream promptly.
Labels	String	political_content,xxxx	The details of the labels. Multiple labels are separated by commas. Valid values: ad: Ad-related content violence: Violent or terrorist content political_content: Political content specified_speaking: Specific spoken content specified_lyrics: Specific song lyrics sexual_content: Sexual content sexual_sounds: Sexual sounds contraband: Contraband content profanity: Abusive content religion: Religious content cyberbullying: Cyberbullying negative_content: Negative content nontalk: Silent audio C_customized: Custom library hit
RiskLevel	String	high	The risk level of the audio segment. Valid values: high: high risk medium: medium risk low: low risk none: no risk detected
RiskWords	String	AAA,BBB,CCC	The risk words that are hit. Multiple words are separated by commas.
RiskTips	String	porn_vulgar_words,porn_description	The sub-labels. Multiple labels are separated by commas.
Extend	String	{"riskTips":"porn_vulgar_words","riskWords":"pornographic_service"}	A reserved field.

Examples

Sample request

{
    "Service": "liveStreamDetection_global",
    "ServiceParameters": {
        "taskId": "abcd****"
    }
}

Sample success response

Moderating only live stream video

{
    "Code": 200,
    "RequestId": "25106421-XXXX-XXXX-XXXX-15DA5AAAC546",
    "Message": "success finished",
    "Data": {
        "DataId": "dc16c28f-xxxx-xxxx-xxxx-51efe0131080",
        "LiveId": "live0307-0728-****",
        "RiskLevel": "high",
        "FrameResult": {
            "FrameNum": 2,
            "FrameSummarys": [
                {
                    "Label": "violent_explosion",
                    "Description": "Suspected to contain fireworks",
                    "LabelSum": 8
                },
                {
                    "Label": "sexual_cleavage",
                    "Description": "Suspected to contain nudity or sexual content",
                    "LabelSum": 8
                }
            ],
            "RiskLevel": "medium",
            "Frames": [
                {
                    "Offset": 1,
                    "RiskLevel": "none",
                    "Results": [
                        {
                            "Result": [
                                {
                                    "Label": "nonLabel",
                                    "Description": "No risk detected"
                                }
                            ],
                            "Service": "baselineCheck_global"
                        }
                    ],
                    "TempUrl": "http://abc.oss-ap-southeast-1.aliyuncs.com/test1.jpg",
                    "Timestamp": 1704769999000
                },
                {
                    "Offset": 2,
                    "RiskLevel": "medium",
                    "Results": [
                        {
                            "Result": [
                                {
                                    "Confidence": 78.88,
                                    "Label": "sexual_cleavage",
                                    "Description": "Suspected to contain nudity or sexual content"
                                },
                              	{
                                    "Confidence": 74.18,
                                    "Label": "violent_explosion",
                                    "Description": "Suspected to contain fireworks"
                                }
                            ],
                            "Service": "baselineCheck_global"
                        }
                    ],
                    "TempUrl": "http://abc.oss-ap-southeast-1.aliyuncs.com/test2.jpg",
                    "Timestamp": 1704769989000
                }
            ]
        }
    }
}

Moderating both live stream video and audio

{
    "Code": 200,
    "RequestId": "25106421-XXXX-XXXX-XXXX-15DA5AAAC546",
    "Message": "success finished",
    "Data": {
        "DataId": "dc16c28f-xxxx-xxxx-xxxx-51efe0131080",
        "LiveId": "live0307-0728-****",
        "RiskLevel": "high",
        "AudioResult": {
            "AudioSummarys": [
                {
                    "Label": "sexual_sounds",
                    "LabelSum": 3
                }
            ],
            "RiskLevel": "high",
            "SliceDetails": [
                {
                    "EndTime": 60,
                    "EndTimestamp": 1698912813192,
                    "Labels": "",
                    "RiskLevel": "none",
                    "StartTime": 30,
                    "StartTimestamp": 1698912783192,
                    "Text": "Content Moderation",
                    "Url": "http://abc.oss-ap-southeast-1.aliyuncs.com/test.wav"
                },
                {
                    "EndTime": 30,
                    "EndTimestamp": 1698912813192,
                    "Extend": "{\"customizedWords\":\"service\",\"customizedLibs\":\"test\"}",
                    "Labels": "C_customized",
                    "RiskLevel": "high",
                    "StartTime": 0,
                    "StartTimestamp": 1698912783192,
                    "Text": "Welcome to Alibaba Cloud Content Moderation service",
                    "Url": "http://abc.oss-ap-southeast-1.aliyuncs.com/test.wav"
                }
            ]
        },
        "FrameResult": {
            "FrameNum": 2,
            "FrameSummarys": [
                {
                    "Label": "violent_explosion",
                    "Description": "Suspected to contain fireworks",
                    "LabelSum": 8
                },
                {
                    "Label": "sexual_cleavage",
                    "Description": "Suspected to contain nudity or sexual content",
                    "LabelSum": 8
                }
            ],
            "RiskLevel": "medium",
            "Frames": [
                {
                    "Offset": 1,
                    "RiskLevel": "none",
                    "Results": [
                        {
                            "Result": [
                                {
                                    "Label": "nonLabel",
                                    "Description": "No risk detected"
                                }
                            ],
                            "Service": "baselineCheck_global"
                        }
                    ],
                    "TempUrl": "http://abc.oss-ap-southeast-1.aliyuncs.com/test1.jpg",
                    "Timestamp": 1704769999000
                },
                {
                    "Offset": 2,
                    "RiskLevel": "medium",
                    "Results": [
                        {
                            "Result": [
                                {
                                    "Confidence": 74.18,
                                    "Label": "violent_explosion",
                                    "Description": "Suspected to contain fireworks"
                                },
                                {
                                    "Confidence": 78.88,
                                    "Label": "sexual_cleavage",
                                    "Description": "Suspected to contain nudity or sexual content"
                                }
                            ],
                            "Service": "baselineCheck_global"
                        }
                    ],
                    "TempUrl": "http://abc.oss-ap-southeast-1.aliyuncs.com/test2.jpg",
                    "Timestamp": 1704769989000
                }
            ]
        }
    }
}

Cancel a live stream moderation job

API description

You can call this operation to cancel a live stream moderation job.

API operation: VideoModerationCancel.

Billing: This operation is not billed.

Debugging

Before integration, you can also use Alibaba Cloud OpenAPI to debug the VideoModerationCancel API online and view sample call code and SDK dependency information for an overview of the API's usage and parameters.

Request parameters

Name

Type

Required

Example

Description

Service

String

Yes

liveStreamDetection_global

The type of the moderation service. The value must be the same as the service type you specified when you submitted the moderation job.

ServiceParameters

JSONString

Yes

The parameters required by the service. This parameter is a JSON string. For more information about the parameters, see Table 11. ServiceParameters.

Table 11. ServiceParameters

Name

Type

Required

Example

Description

taskId

string

Yes

abcd****

The taskId of the moderation job to cancel. You can specify only one taskId at a time.

Note: You can obtain the taskId of a moderation job from the response data after you submit the job.

Response parameters

Name	Type	Example	Description
Code	Integer	200	The status code. For more information, see Code description.
Message	String	OK	The response message for the request.
RequestId	String	AAAAAA-BBBB-CCCCC-DDDD-EEEEEEEE****	The request ID.

Example

Sample request

{
    "Service": "liveStreamDetection",
    "ServiceParameters": {
        "taskId": "abcd****"
    }
}

Sample response

{
    "Msg": "OK",
    "Code": 200,
    "RequestId": "70ED13B0-BC22-576D-9CCF-1CC12FEAC477"
}

Code description

This topic describes the response codes for the ApsaraVideo Live stream auditing 2.0 API. You are charged only for requests that return a response code of 200 or 280. Requests that return other response codes are not charged.

Code	Description
200	The request is successful or the moderation is complete.
280	Moderation in progress.
400	The request parameters are empty.
401	The request parameters are invalid.
402	The length of a request parameter does not meet the requirements. Check the parameter and try again.
403	The number of queries per second (QPS) exceeds the limit. Check the number of concurrent requests and try again.
404	An error occurred while downloading the video. Check the video and try again.
405	The video download timed out. The video may be inaccessible. Check the video and try again.
406	The video is too large. Check the video size and try again.
407	The video format is not supported. Check the format and try again.
408	You are not authorized to call this operation. Your account may not have the service activated, have an overdue payment, or is not granted the required permissions.
409	The specified TaskId does not exist. The result may have expired because it was retained for more than 24 hours.
480	The number of concurrent moderation streams exceeds the limit. Check the number of concurrent requests and try again.
500	A system error occurred.