Callback API - Platform For AI - Alibaba Cloud Documentation Center

PAI-Rec's callback API captures request parameters, user features, and item features at the moment of each recommendation request. Use these logs for data analytics, offline model training, or online learning.

Offline methods such as associating features by time windows are unreliable because latency between system links is hard to estimate, which causes feature leakage. The callback API avoids this by recording features at the source: when the recommendation request reaches the recommendation service, the system logs the request ID, user features, and item features to a message queue (DataHub or Kafka), then syncs the data to MaxCompute (ODPS).

Prerequisites

Before you begin, make sure you have:

A PAI-Rec deployment with the DPI engine configured
A DataHub project and the endpoint URL for your data source
(Optional) An EasyRec model, if your callback flow uses model-generated features

API reference

Endpoint

POST /api/callback

Request parameters

Parameter	Type	Required	Description	Constraints	Example
`scene_id`	string	Yes	The scene name for which to log data	—	`homepage`
`uid`	string	Yes	The user registration ID	—	`85578510`
`request_id`	string	Yes	A unique identifier for the recommendation request	—	`d9cb1c8d***`
`item_list`	json list	Yes	The list of recommended item IDs	Each item must include `item_id`	`[{"item_id":"99886867"}, {"item_id":"99888623"}]`
`features`	json string	No	User features at the time of the request	Valid JSON string	`{"age":25, "city":"beijing"}`
`request_info`	json string	No	Additional request information	Valid JSON string	`{"recom_id":"12334234"}`

Request example

curl 'http://host/api/callback' \
  -d '{"uid":"84603208","request_id":"d9cb1c8d-4d3f-491b-9ea3-380481dabde3","scene_id":"homepage","features":{"age":25, "city":"beijing"},"item_list":[{"item_id":"113939841"},{"item_id":"113764910"}],"request_info":{"recom_id":"1111111"}}'

Response

{
  "code": 200,
  "msg": "success"
}

Field	Type	Description
`code`	integer	HTTP status code. `200` indicates success.
`msg`	string	Status message. `success` indicates the callback was recorded.

Configure the callback

Callback configuration has two parts: the CallBackConfs block that defines the logging behavior, and the DatahubConfs block that defines where to write the data.

CallBackConfs

"CallBackConfs": {
  "home_feed": {
    "DataSource": {
      "Name": "pairec_callback_dh",
      "Type": "datahub"
    },
    "RankConf": {
      "RankAlgoList": [
        "ali_rnk_v2_woid_callback_public_v2"
      ],
      "ContextFeatures": [
        "none"
      ],
      "Processor": "EasyRec"
    },
    "RawFeatures": false,
    "RawFeaturesRate": 0,
    "ItemSize": 100,
    "ItemSizeRate": 10,
    "UseUserFeatures": true
  }
}

Parameter	Description	Default	Constraints
`home_feed` (scene name)	The scene for which to log data	—	Must match a scene name in `SceneConfs`
`DataSource.Type`	The message queue type	—	Currently only `datahub` is supported
`DataSource.Name`	The DataHub data source name	—	Must match a key in `DatahubConfs`
`RankConf`	The model configuration, identical to the DPI engine model config	—	Omit this field if no model-generated features are used
`RawFeatures`	Whether to record raw item features from the EasyRec model	`false`	Set to `true` to enable; must also set `RawFeaturesRate`
`RawFeaturesRate`	Sampling ratio for raw features	`0`	Integer from `0` to `100`. Takes effect only when `RawFeatures` is `true`.
`ItemSize`	Maximum number of items to process per callback	All items from the recommendation flow	Items are truncated to the first `ItemSize` items before sampling
`ItemSizeRate`	Sampling ratio applied after `ItemSize` truncation	—	Integer from `1` to `100`. If both `ItemSize` and `ItemSizeRate` are set, the system first truncates to `ItemSize` items, then samples by `ItemSizeRate`.
`UseUserFeatures`	Whether to use user features from the recommendation flow when `AutoInvokeCallBack` is enabled	—	When set to `true`, no separate user feature retrieval in `FeatureConfs` is needed for the callback flow

DatahubConfs

PAI-Rec automatically creates the DataHub topic based on the topic name and schema — no manual topic creation is required.

"DatahubConfs": {
  "pairec_callback_dh": {
    "Endpoint": "http://dh-cn-hangzhou-int-vpc.aliyuncs.com",
    "ProjectName": "${ProjectName}",
    "TopicName": "pairec_callback_log",
    "Schemas": [
      {"Field": "request_id",        "Type": "string"},
      {"Field": "module",            "Type": "string"},
      {"Field": "scene",             "Type": "string"},
      {"Field": "request_time",      "Type": "integer"},
      {"Field": "user_features",     "Type": "string"},
      {"Field": "item_features",     "Type": "string"},
      {"Field": "request_info",      "Type": "string"},
      {"Field": "user_id",           "Type": "string"},
      {"Field": "item_id",           "Type": "string"},
      {"Field": "raw_features",      "Type": "string"},
      {"Field": "generate_features", "Type": "string"},
      {"Field": "context_features",  "Type": "string"}
    ]
  }
}

Replace ${ProjectName} with your DataHub project name.

Feature loading configuration

Feature loading for the callback flow follows the same structure as standard feature configuration. The only difference is the scene alias: use {scene_name}_callback as the scene name in FeatureConfs. For example, for the scene home_feed, use home_feed_callback.

"FeatureConfs": {
  "home_feed_callback": {
    "AsynLoadFeature": true,
    "FeatureLoadConfs": [
      {
        "FeatureDaoConf": {
          "AdapterType": "hologres",
          "HologresName": "pairec-holo",
          "FeatureKey": "user:uid",
          "UserFeatureKeyName": "client_str",
          "HologresTableName": "dwd_ali_user_all_feature_v2_holo",
          "UserSelectFields": "*",
          "FeatureStore": "user"
        },
        "Features": [
          {
            "FeatureType": "new_feature",
            "FeatureName": "day_h",
            "Normalizer": "hour_in_day",
            "FeatureStore": "user"
          },
          {
            "FeatureType": "new_feature",
            "FeatureName": "week_day",
            "Normalizer": "weekday",
            "FeatureStore": "user"
          },
          {
            "FeatureType": "new_feature",
            "FeatureName": "rand_int_v",
            "Normalizer": "random",
            "FeatureStore": "user"
          }
        ]
      },
      {
        "FeatureDaoConf": {
          "AdapterType": "hologres",
          "HologresName": "pairec-holo",
          "FeatureKey": "user:uid",
          "UserFeatureKeyName": "client_str",
          "HologresTableName": "dwd_ali_user_table_v3_expo_static_feature_v2_holo",
          "UserSelectFields": "*",
          "FeatureStore": "user"
        }
      }
    ]
  }
}

Log format

Each callback request contains one user ID and a list of items. Because user features can be large, user and item features are written as separate log records to the same DataHub topic, distinguished by the module field.

User feature log

Field	Type	Description
`module`	string	Always `"user"` — identifies this as a user feature record
`request_id`	string	The request ID from the callback request
`scene`	string	The scene name from the callback request
`request_time`	integer	The Unix timestamp of the request
`user_id`	string	The user ID
`user_features`	string	User features as a JSON string
`request_info`	string	Additional request information from the callback request

Item feature log

Field	Type	Description
`module`	string	Always `"item"` — identifies this as an item feature record
`request_id`	string	The request ID from the callback request
`scene`	string	The scene name from the callback request
`request_time`	integer	The Unix timestamp of the request
`user_id`	string	The user ID
`item_id`	string	The item ID
`item_features`	string	Item features as a JSON string
`raw_features`	string	Raw features returned by the EasyRec model
`generate_features`	string	Feature Generation (FG) output from the EasyRec model
`context_features`	string	Context features returned by the EasyRec model

Enable automatic scene callback

By default, you must call the callback manually after each recommendation request. The DPI engine also supports automatic invocation: after a recommendation request completes, the system runs the callback flow automatically.

To enable automatic callbacks, set AutoInvokeCallBack to true in SceneConfs:

"SceneConfs": {
  "${scene_name}": {
    "default": {
      "RecallNames": [
        "collaborative_filter"
      ],
      "AutoInvokeCallBack": true,
      "AutoInvokeCallBackRate": 100
    }
  }
}

Replace ${scene_name} with the actual scene name, such as home_feed.

Parameter	Description	Default	Constraints
`AutoInvokeCallBack`	Whether to automatically invoke the callback flow after each recommendation request	—	Set to `true` to enable
`AutoInvokeCallBackRate`	Percentage of recommendation requests that trigger an automatic callback	—	Integer from `1` to `100`. If set to `0` or omitted, all traffic triggers a callback.

What's next

Feature configuration