how to configure notifications in content moderation - AI Guardrails

Content Moderation delivers content detection results and self-review results as asynchronous callback notifications. Configure callback notifications for the Content Moderation API to receive this data on your own server and integrate it into your business.

Background

Content Moderation API callback notifications include scan result callback notifications and review result callback notifications. Both types are sent to the HTTP callback notification address that you configure.

Scan result callback notification — After Content Moderation completes a detection request, the server sends the detection results in a POST request to the callback address that you specify.
Review result callback notification — After you modify detection results by performing a self-review or by calling the feedback operation, the Content Moderation server sends the review results in a POST request to the callback address that you specify. For more information, see Self-review.

Prerequisites

A callback address that meets the following requirements:
- Must be a publicly accessible URL that uses the HTTP or HTTPS protocol.
- Must support the POST method.
- Must support UTF-8 encoding for transmitted data.
- Must support receiving data in the application/x-www-form-urlencoded format.
- Must support the checksum and content form parameters.
(Optional) To receive Alibaba Cloud human review callbacks, activate the Alibaba Cloud human review service.

Scan result callback notification

All asynchronous detection operations of the Content Moderation API support scan result callbacks, such as Asynchronous image moderation and Asynchronous video moderation. When you make an asynchronous call and want Content Moderation to return scan results to you, specify the callback (callback address) and seed (verification string) request parameters.

Note

If you do not use scan result callbacks for asynchronous calls, you can obtain asynchronous detection results only by polling on a regular basis.

Procedure

Prepare an HTTP callback address and a seed parameter to receive scan results.
When you call an asynchronous Content Moderation API operation, pass the corresponding callback and seed request parameters. For more information, see the parameter descriptions in the API reference.

Callback notifications for human review

The API operations for human review do not return detection results in real time. Instead, the system notifies the caller by using callbacks. The configuration method depends on the human review service that you use.

Machine-assisted human review

If you use the Alibaba Cloud human review service together with the machine moderation service, configure notifications in the console to enable callbacks for the human review module.

Log on to the Content Moderation console.
In the left-side navigation pane, choose Machine Moderation V1.0 > Settings.
On the Machine Moderation page, click the Notification tab, and then click Create New Notification.
In the Create New Notification dialog box, enter the Title and Callback URL, and select the Encryption algorithm, Notification type, and Moderation Result. Then, click OK.
On the BizType Manage tab, in the Actions column to the right of the target business scenario, click Choose Notification Plan and associate the Callback notification scheme that you configured.
For the notification type, select Manual Review Results by AlibabaCloud.

The following encryption algorithms are supported:

SHA256 — Uses the HMAC-SHA256 encryption algorithm.
SM3 — Uses the SM3 Chinese cryptographic algorithm. A hexadecimal string is returned, and the string consists of lowercase letters and digits.
For example, abc returns 66c7f0f462eeedd9d1f2d46bdc10e4e24167c4875cf2f7a2297da02b8f4ba8e0 after it is encrypted by using the SM3 Chinese cryptographic algorithm.

After you save the configuration, the system automatically generates a seed. The seed value is used to verify that the requests received by your callback operation come from the Content Moderation server. Save the automatically generated seed.

Important

If you have already configured the callback notification service for machine moderation, you can reuse the existing configuration or reconfigure it based on your business requirements.
The system supports callbacks for all review results. You can also select some of the review results for callbacks based on your business scenarios.

Callback data structure

After you enable callback notifications, Content Moderation sends callback notifications based on your callback configuration. Each callback request is sent as form data that contains the following parameters.

Parameter	Type	Description
`checksum`	String	The checksum. It is generated by applying the configured encryption algorithm to the string that is concatenated from `<user UID> + <seed> + <content>`. The available encryption algorithms are HMAC-SHA256 and SM3. The user UID is your Alibaba Cloud account ID. To query your Alibaba Cloud account ID, go to the Alibaba Cloud Management Console and move the pointer over the profile picture in the upper-right corner. Note Note: To prevent tampering, when you receive a pushed result, generate a string by using the preceding algorithm and verify it against the `checksum`.
`content`	String	A JSON object that is stored in the string format. Parse the string back into a JSON object. For more information about the structure of `content` after it is parsed into JSON, see Description of the content parameter in this topic. Note The parameter structure of `content` differs between the Content Moderation API and OSS content violation detection.

The content parameter is a JSON object. The following table describes its structure.

Parameter	Type	Required	Description
`scanResult`	JSONObject	No	The scan result. The structure varies based on the detected object, such as an image or a video. For an image object, the structure is the same as the `results` response parameter in synchronous image moderation. For more information, see Synchronous image moderation. For a video object, the structure is the same as the `results` response parameter in asynchronous video moderation. For more information, see Asynchronous video moderation.
`auditResult`	JSONObject	No	The result of your self-review. This field is returned only when a self-review is performed. For more information, see auditResult. Note Note: This field is not returned if only scan results are pushed.
`humanAuditResult`	JSONObject	No	The Alibaba Cloud human review result. If you have purchased the Alibaba Cloud human review service, the human review result is returned in this field. For more information, see humanAuditResult.

The following table describes the auditResult structure.

Parameter	Type	Required	Description
`suggestion`	String	Yes	The self-review result. Valid values: `block` (the content is set as a violation during the review), `pass` (the content is set as normal during the review).
`labels`	JSONArray	No	The labels that are set during the self-review. This parameter contains one or more of the following values: `porn` (pornography), `terrorism` (terrorist or politically sensitive content), `ad` (advertisements in images or text), `live` (undesirable scenes).

The following table describes the humanAuditResult structure.

Parameter	Type	Required	Description
`suggestion`	String	Yes	The Alibaba Cloud human review result. Valid values: `block` (the Alibaba Cloud human review result is a violation), `pass` (the Alibaba Cloud human review result is normal).
`taskId`	String	Yes	The ID of the detection task. You can use the task ID to associate the review result with the corresponding content.
`dataId`	String	Yes	The ID of the detected content.
`labels`	JSONArray	No	The label results of the human review. This parameter may have multiple values. Note Note: This parameter is not returned by default. Additional fees apply for configuring this field. For more information, contact your sales representative.

The following example shows the structure of content:

{
    "scanResult": {
        "code": 200,
        "msg": "OK",
        "taskId": "fdd25f95-4892-4d6b-aca9-7939bc6e9baa-1486198766695",
        "url": "http://1.jpg",
        "results": [
            {
                "rate": 100,
                "scene": "porn",
                "suggestion": "block",
                "label": "porn"
            }
        ]
    },
    "auditResult": {
        "suggestion": "block",
        "labels": [
            "porn",
            "ad",
            "terrorism"
        ]
    },
    "humanAuditResult": {
        "suggestion": "pass",
        "dataId": "yyyy",
        "labels": [
            "porn",
            "vulgar"
        ],
        "taskId": "xxxxxx"
    }
}

Name	Description
Callback address	A callback address is the server address that you configure in the Content Moderation console. It is typically the public address of your own business server.
Seed	The seed value is used to verify whether a request sent to the HTTP callback notification address that you specify originates from the Content Moderation server.
Number of callback attempts	After your server receives a callback message pushed by Content Moderation, an HTTP status code of 200 indicates that the message is received. Any other HTTP status code indicates that the message failed to be received. If a message fails to be received, the Content Moderation server pushes the callback message repeatedly until you receive it. Content Moderation pushes a callback message up to 3 times.
Callback data	Callback data is the data that the Content Moderation server returns to the callback notification address that you specify. For a description of the callback data structure, see Description of the content parameter.