AI Guardrails:Text Moderation with LLMs

Pay-as-you-go

When you activate the Content Moderation service, pay-as-you-go is the default billing method. You are billed daily based on your actual usage. You are not charged if you do not use the service.

Resource plans

For high or consistent moderation volumes, resource plans offer significant discounts. You can purchase and stack multiple plans. For more information, see Purchase a resource plan for Content Moderation 2.0.

After you purchase a resource plan, your API usage for Content Moderation 2.0 is first deducted from your resource plan. When your resource plan is depleted, subsequent usage is billed on a pay-as-you-go basis. Monitor your remaining plan balance and pay-as-you-go charges. You can set low-balance alerts in the Resource Plan system.

Label definitions

Text Moderation 2.0 supports over 30 granular labels across 6 risk categories, and returns a confidence score (0–100, where a higher score indicates greater confidence) for each. If content contains multiple risk types, the service returns multiple granular labels. The following tables list the risk label values, their corresponding confidence score ranges, and their meanings.

Configure risk labels

Enable or disable risk labels in the console. You can also adjust the detection scope for specific labels. See the Content Moderation console for details.

1. In the left navigation pane, choose Machine Moderation V2.0>Text Moderation>Rules.

2. On the Rules Management tab, find a large model moderation solution, for example, aigc_moderation_byllm_global, and click Set Thesaurus in the Operation column.

   1. Select a detection type to configure, such as inappropriate content detection.

   2. Click Edit and modify the detection settings.

   3. Click Save. The new configuration takes effect in the production environment in 2 to 5 minutes.

Step 1: Activate the service

To activate the Text Moderation 2.0 service, visit activate service.

Step 2: Grant permissions to a RAM user

Before using the SDK or calling an API, grant the required permissions to a RAM user. Create an AccessKey pair for your Alibaba Cloud account or a RAM user to authenticate API calls. For instructions, see Obtain an access key.

1. Log on to the RAM console using your Alibaba Cloud account.

2. Create a RAM user. For details, see Create a RAM user.

3. Grant the AliyunYundunGreenWebFullAccess system policy to the RAM user. This policy grants full access to Content Moderation. For details, see Manage RAM user permissions.

The RAM user can now call the Content Moderation API.

Step 3: Install and integrate the SDK

API reference

{
    "Service": "ugc_moderation_byllm_global",
    "ServiceParameters": {
        "content": "testing content",
        "dataId": "text0424****"
    }
}

{
    "Code": 200,
    "Data": {
        "Result": [
            {
                "Label": "political_entity",
                "Description": "Suspected political entity",
                "Confidence": 100.0,
                "RiskWords": "WordA,WordB",
                "RiskPositions": [
                    {
                        "EndPos": 14,
                        "RiskWord": "WordA",
                        "StartPos": 16
                    }
                ]
            },
            {
                "Label": "political_figure",
                "Description": "Suspected political figure",
                "Confidence": 100.0,
                "RiskWords": "WordB,WordC",
                "RiskPositions": [
                    {
                        "EndPos": 24,
                        "RiskWord": "WordC",
                        "StartPos": 26
                    }
                ]
            }
        ],
        "RiskLevel": "high",
        "DataId": "text0424****"
    },
    "Message": "OK",
    "RequestId": "AAAAAA-BBBB-CCCCC-DDDD-EEEEEEEE****"
}

{
    "Code": 200,
    "Data": {
        "Result": [
            {
                "Description": "Hit a custom library",
                "CustomizedHit": [
                     {
                        "LibName": "Custom Library Name 1",
                        "Keywords": "custom keyword"
                     }
                ],
                "Confidence": 100,
                "Label": "customized"
             }
        ],
        "RiskLevel": "high",
        "DataId": "text0424****"
    },
    "Message": "OK",
    "RequestId": "AAAAAA-BBBB-CCCCC-DDDD-EEEEEEEE****"
}

{
  "RequestId": "ZZZZZ-2024-0307-FORYOU-EVER",
  "Message": "OK",
  "Data": {
    "Ext": {
      "LlmContent": {
        "OutputText": "Suspected offensive or abusive content"
      }
    },
    "Result": [
      {
        "RiskWords": "risk word",
        "Description": "Suspected offensive or abusive content",
        "Confidence": 100.0,
        "Label": "inappropriate_profanity",
        "RiskPositions": [
          {
            "RiskWord": "risk word",
            "EndPos": 5,
            "StartPos": 2
          }
        ]
      }
    ],
    "RiskLevel": "high"
  },
  "Code": 200
}