:Recall configuration

Hologres

Collaborative filtering requires two tables with fixed schemas: a u2i table to retrieve an item list based on a user_id, and an i2i table to retrieve similar items.

In addition to retrieving similar items directly from an i2i table (the u2i2i method), you can also retrieve them indirectly by using an i2x table and an x2i table (the u2i2x2i method). In this approach, 'x' represents an item attribute, such as a category, brand, or city. The process first retrieves the 'x' attribute of an item (for example, the category field) and then uses the x2i table to fetch popular items that share this attribute value for recommendation.

u2i table

i2i table (required only for u2i2i)

i2x table (required only for u2i2x2i)

x2i table (required only for u2i2x2i)

u2i2i configuration example:

{
    "RecallConfs": [
        {
            "Name": "collaborative_filter",
            "RecallType": "UserCollaborativeFilterRecall",
            "RecallCount": 1000,
            "UserCollaborativeDaoConf": {
                "AdapterType": "hologres",
                "HologresName": "holo_info",
                "User2ItemTable": "u2i_table",
                "Item2ItemTable": "i2i_table",
                "Normalization": "on"
            }
        }
    ]
}

u2i2x2i configuration example:

{
    "RecallConfs": [
        {
            "Name": "collaborative_filter",
            "RecallType": "UserCollaborativeFilterRecall",
            "RecallCount": 1000,
            "UserCollaborativeDaoConf": {
                "AdapterType": "hologres",
                "HologresName": "holo_info",
                "User2ItemTable": "u2i_table",
                "Item2XTable": "i2x_table",
                "X2ItemTable": "x2i_table",
                "XKey": "category",
                "XDelimiter": ",",
                "Normalization": "on"
            }
        }
    ]
}

UserCollaborativeDaoConfig

PAI-FeatureStore

Collaborative filtering requires two tables with fixed schemas: a u2i table to retrieve an item list based on a user_id, and an i2i table to retrieve similar items.

The data for both tables is generated in MaxCompute and must be registered with PAI-FeatureStore as offline views. The required MaxCompute table schema is described below.

u2i table

The following example shows how to register data with PAI-FeatureStore:

i2i table

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "collaborative_filter",
            "RecallType": "UserCollaborativeFilterRecall",
            "RecallCount": 1000,
            "UserCollaborativeDaoConf": {
                "AdapterType": "featurestore",
                "FeatureStoreName": "fs_pairec",
                "User2ItemFeatureViewName": "u2i_recall",
                "Item2ItemFeatureViewName": "i2i_collaborative",
                "Normalization": "on"
            }
        }
    ]
}

UserCollaborativeDaoConfig

Tablestore

Collaborative filtering requires two tables with fixed schemas: a u2i table to retrieve an item list based on a user_id, and an i2i table to retrieve similar items.

u2i table

i2i table

{
    "RecallConfs": [
        {
            "Name": "collaborative_filter",
            "RecallType": "UserCollaborativeFilterRecall",
            "RecallCount": 1000,
            "UserCollaborativeDaoConf": {
                "AdapterType": "tablestore",
                "TableStoreName": "tablestore_info",
                "User2ItemTable": "u2i_table",
                "Item2ItemTable": "i2i_table",
                "Normalization": "on"
            }
        }
    ]
}

Redis

The collaborative filtering process for Redis is unique, involving two steps:

• Construct a key by concatenating the RedisPrefix value with the user ID (uid) to query the u2i list. The result is a string in one of the following formats: item_id1,item_id2,item_id3... or item_id1:score1,item_id2:score2,item_id3:score3....

• Query the i2i list. Use the MGET command with the item IDs obtained in the previous step. The i2i data is also a string, with the following format: item_id1:score1,item_id2:score2,item_id3:score3....

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "collaborative_filter",
            "RecallType": "UserCollaborativeFilterRecall",
            "RecallCount": 1000,
            "UserCollaborativeDaoConf": {
                "AdapterType": "redis",
                "RedisName": "redis_info",
                "RedisPrefix": "cr_",
                "Normalization": "on"
            }
        }
    ]
}

Hologres

This approach retrieves data using the same logic as collaborative filtering, but computes U2I data in real time from a user historical behavior table.

The system updates the user historical behavior table in real time based on logs, enabling real-time recall.

Like collaborative filtering, this method also supports u2i2x2i. First, it retrieves an item's x attribute, and then recalls other items that have the same attribute.

User historical behavior table

i2i table (required only for u2i2i)

i2x table (required only for u2i2x2i)

x2i table (required only for u2i2x2i)

Statement for creating the behavior table:

BEGIN;
CREATE TABLE "sv_rec"."user_behavior_seq" (
  "user_id" text NOT NULL,
  "item_id" text NOT NULL,
  "event" text NOT NULL,
  "play_time" float8  NULL,
  "timestamp" int8 NOT NULL
);
CALL SET_TABLE_PROPERTY('"sv_rec"."user_behavior_seq"', 'orientation', 'column');
call set_table_property('table_name', 'distribution_key', '"user_id"');
CALL SET_TABLE_PROPERTY('"sv_rec"."user_behavior_seq"', 'clustering_key', '"user_id:asc","timestamp:desc"');
CALL SET_TABLE_PROPERTY('"sv_rec"."user_behavior_seq"', 'bitmap_columns', '"user_id","event"');
CALL SET_TABLE_PROPERTY('"sv_rec"."user_behavior_seq"', 'dictionary_encoding_columns', '"user_id:auto","item_id:auto","event"');
CALL SET_TABLE_PROPERTY('"sv_rec"."user_behavior_seq"', 'time_to_live_in_seconds', '2592000');
comment on table "sv_rec"."user_behavior_seq" is 'User real-time behavior sequence';
comment on column "sv_rec"."user_behavior_seq"."user_id" is 'User ID';
comment on column "sv_rec"."user_behavior_seq"."item_id" is 'Item ID';
comment on column "sv_rec"."user_behavior_seq"."event" is 'Event type';
comment on column "sv_rec"."user_behavior_seq"."play_time" is 'Reading or playback duration';
comment on column "sv_rec"."user_behavior_seq"."timestamp" is 'Timestamp, in seconds';
COMMIT;

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "RealTimeEtrecRecall",
            "RecallType": "RealTimeU2IRecall",
            "RecallCount": 200,
            "RealTimeUser2ItemDaoConf": {
                "UserTriggerDaoConf": {
                    "AdapterType": "hologres",
                    "HologresName": "holo_info",
                    "HologresTableName": "user_behavior_table",
                    "WhereClause": "event='xxx'",
                    "Limit": 200,
                    "EventPlayTime": "playback:5000;playvslide:5000",
                    "EventWeight": "playback:1;playvslide:2",
                    "WeightExpression": "exp((-0.2)*((currentTime-eventTime)/3600/24))",
                    "WeightMode": "sum",
                    "NoUsePlayTimeField": false
                },
                "Item2ItemTable": "i2i_table",
                "SimilarItemIdField": "similar_item_ids"
            }
        }
    ]
}

u2i2x2i configuration example:

{
    "RecallConfs": [
        {
            "Name": "RealTimeU2I2X2IRecall",
            "RecallType": "RealTimeU2IRecall",
            "RecallCount": 200,
            "RealTimeUser2ItemDaoConf": {
                "UserTriggerDaoConf": {
                    "AdapterType": "hologres",
                    "HologresName": "holo_info",
                    "HologresTableName": "user_behavior_table",
                    "WhereClause": "event='xxx'",
                    "Limit": 200,
                    "EventPlayTime": "playback:5000;playvslide:5000",
                    "EventWeight": "playback:1;playvslide:2",
                    "WeightExpression": "exp((-0.2)*((currentTime-eventTime)/3600/24))",
                    "WeightMode": "sum",
                    "NoUsePlayTimeField": false
                },
                "Item2XTable": "i2x_table",
                "X2ItemTable": "x2i_table",
                "XKey": "category",
                "XDelimiter": ","
            }
        }
    ]
}

RealTimeUser2ItemDaoConf:

PAI-FeatureStore

This approach retrieves data using the same logic as collaborative filtering, but computes U2I data in real time from a user historical behavior table.

The system updates the user historical behavior table in real time based on logs, enabling real-time recall.

On the PAI-FeatureStore platform, you can use a behavior sequence feature view to store user historical behavior data.

User historical behavior table

i2i table

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "realtimeu2i",
            "RecallType": "RealTimeU2IRecall",
            "RecallCount": 200,
            "RealTimeUser2ItemDaoConf": {
                "UserTriggerDaoConf": {
                    "AdapterType": "featurestore",
                    "FeatureStoreName": "fs_pairec",
                    "FeatureStoreViewName": "user_behavior_table",
                    "TriggerCount": 100,
                    "EventWeight": "click:1;order:1",
                    "WeightExpression": "exp((-0.2)*((currentTime-eventTime)/3600/24))",
                    "WeightMode": "sum",
                    "NoUsePlayTimeField": false,
                    "ItemIdFieldName": "item_id",
                    "EventFieldName": "event",
                    "PlayTimeFieldName": "playtime",
                    "TimestampFieldName": "event_unix_time"
                },
                "Item2ItemFeatureViewName": "i2i_collaborative",
                "SimilarItemIdField": "similar_item_ids"
            }
        }
    ]
}

RealTimeUser2ItemDaoConf:

Hologres

Retrieves a list of items based on a user ID. The table uses a predefined schema.

U2I table

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "user2item_recall",
            "RecallType": "UserCustomRecall",
            "RecallCount": 500,
            "DaoConf": {
                "AdapterType": "hologres",
                "HologresName": "holo_info",
                "HologresTableName": "user_item_table"
            }
        }
    ]
}

PAI-FeatureStore

Retrieves a list of items based on a user ID. The table uses a predefined schema.

The data for this table is generated in MaxCompute and must be registered as an offline feature view in PAI-FeatureStore. The MaxCompute table must use the following schema.

U2I table

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "user2item_recall",
            "RecallType": "UserCustomRecall",
            "RecallCount": 500,
            "DaoConf": {
                "AdapterType": "featurestore",
                "FeatureStoreName": "fs_pairec",
                "FeatureStoreViewName": "u2i_recall"
            }
        }
    ]
}

Tablestore

Retrieves a list of items based on a user ID. The table uses a predefined schema.

U2I table

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "user2item_recall",
            "RecallType": "UserCustomRecall",
            "RecallCount": 500,
            "DaoConf": {
                "AdapterType": "tablestore",
                "TableStoreName": "ots_info",
                "TableStoreTableName": "user_item_table"
            }
        }
    ]
}

Redis

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "user2item_recall",
            "RecallType": "UserCustomRecall",
            "RecallCount": 500,
            "DaoConf": {
                "AdapterType": "redis",
                "RedisName": "redis_info",
                "RedisPrefix": ""
            }
        }
    ]
}

Hologres

The user group hot recall table has a predefined schema.

group_hot_table

The trigger_id combines user features and contextual information, such as region and OS.

• Concatenate the feature values in order with an underscore (_) to form the trigger_id.

• If a feature value is empty, use the string "NULL".

• Features that include a Boundaries parameter must be discretized into left-open, right-closed intervals. For example, for age boundaries of [20, 30, 40, 50], the resulting trigger values are <=20, 20-30, 30-40, 40-50, and >50.

A user age of 23 corresponds to "20-30".

An empty user age corresponds to "NULL".

A user age of 60 corresponds to ">50".

A user age of 19 corresponds to "<=20".

Hologres table example:

This example uses three features: gender, age, and OS.

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "user_group_hot_recall",
            "RecallType": "UserGroupHotRecall",
            "RecallCount": 500,
            "Triggers": [
                {
                    "TriggerKey": "gender"
                },
                {
                    "TriggerKey": "age",
                    "Boundaries": [20,30,40,50]
                },
                {
                    "TriggerKey": "os"
                }
            ],
            "DaoConf": {
                "AdapterType": "hologres",
                "HologresName": "holo_info",
                "HologresTableName": "group_hotness_table"
            }
        }
    ]
}

PAI-FeatureStore

The user group hot recall table has a predefined schema.

You prepare the data for this table in MaxCompute. You must register the table with PAI-FeatureStore as an offline feature view. The MaxCompute table requires the following schema.

group_hot_table

The trigger_id combines user features and contextual information, such as region and OS.

• Concatenate the feature values in order with an underscore (_) to form the trigger_id.

• If a feature value is empty, use the string "NULL".

• Features that include a Boundaries parameter must be discretized into left-open, right-closed intervals. For example, for age boundaries of [20, 30, 40, 50], the resulting trigger values are <=20, 20-30, 30-40, 40-50, and >50.

A user age of 23 corresponds to "20-30".

An empty user age corresponds to "NULL".

A user age of 60 corresponds to ">50".

A user age of 19 corresponds to "<=20".

MaxCompute table example:

This example uses three features: gender, age, and OS.

This configuration uses a trigger feature entity.

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "user_group_hot_recall",
            "RecallType": "UserGroupHotRecall",
            "RecallCount": 500,
            "Triggers": [
                {
                    "TriggerKey": "gender"
                },
                {
                    "TriggerKey": "age",
                    "Boundaries": [20,30,40,50]
                },
                {
                    "TriggerKey": "os"
                }
            ],
            "DaoConf": {
                "AdapterType": "featurestore",
                "FeatureStoreName": "fs_pairec",
                "FeatureStoreViewName": "group_hot_recall"
            }
        }
    ]
}

Hologres

The table schema for global hot recall is the same as that for group recall, except that the global hot recall table contains only one record, and its trigger_id is -1.

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "UserGlobalHotRecall",
            "RecallType": "UserGlobalHotRecall",
            "RecallCount": 500,
            "DaoConf": {
                "AdapterType": "hologres",
                "HologresName": "holo_info",
                "HologresTableName": "global_hotness_table"
            }
        }
    ]
}

PAI-FeatureStore

The table schema for global hot recall is the same as that for group recall, except that the global hot recall table contains only one record, and its trigger_id is -1.

The table data is generated in MaxCompute. You must register the table in Feature Store by using an offline FeatureView. The schema for the MaxCompute table is as follows.

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "UserGlobalHotRecall",
            "RecallType": "UserGlobalHotRecall",
            "RecallCount": 500,
            "DaoConf": {
                "AdapterType": "featurestore",
                "FeatureStoreName": "fs_pairec",
                "FeatureStoreViewName": "global_hot_recall"
            }
        }
    ]
}

Tablestore

The table schema for global hot recall is the same as that for group recall, except that the global hot recall table contains only one record, and its trigger_id is -1.

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "UserGlobalHotRecall",
            "RecallType": "UserGlobalHotRecall",
            "RecallCount": 500,
            "DaoConf": {
                "AdapterType": "tablestore",
                "TableStoreName": "ots_info",
                "TableStoreTableName": "global_hotness_recall"
            }
        }
    ]
}

Hologres

{
    "RecallConfs": [
        {
            "Name": "AllLiveItemRecall",
            "RecallType": "ColdStartRecall",
            "RecallCount": 3000,
            "ColdStartDaoConf": {
                "AdapterType": "hologres",
                "HologresName": "holo_info",
                "HologresTableName": "item_status_table",
                "WhereClause": "islist_status=1",
                "PrimaryKey": "\"item_id\"",
                "TimeInterval": 0
            }
        }
    ]
}

ColdStartDaoConf:

You can use WhereClause to filter the candidate set based on features from the API request or user features. For example, to filter the candidate set by language and the user's city, you can set WhereClause to:

"language=${context.features.language} AND city=${user.city}"

In this example, language comes from the features field of the API request, and city is a user feature.

Parameter sources follow these formats:

• user.xxx: Retrieves the value of the user feature named xxx.

• context.features.xxx: Retrieves the value of the feature named xxx from the features field of the API request.

PAI-FeatureStore

This feature randomly recalls a batch of item IDs from a FeatureView that uses the item ID as its primary key.

{
    "RecallConfs": [
        {
            "Name": "item_cold_start_recall",
            "RecallType": "ColdStartRecall",
            "RecallCount": 200,
            "ColdStartDaoConf": {
                "AdapterType": "featurestore",
                "FeatureStoreName": "fs_pairec",
                "FeatureStoreViewName": "rec_sln_demo_item_table_preprocess_all_feature_v1"
            },
            "FilterParams": [
            ]
        }
    ]
}

The FeatureView specified by FeatureStoreViewName can be an offline or a real-time FeatureView. If you use a real-time FeatureView, this configuration also enables real-time recall of newly added item IDs.

If FilterParams is empty, no filter conditions are applied, and item IDs are recalled randomly. Use FilterParams to filter for specific item IDs.

{
            "Name": "item_cold_start_recall",
            "RecallType": "ColdStartRecall",
            "RecallCount": 200,
            "ColdStartDaoConf": {
                "AdapterType": "featurestore",
                "FeatureStoreName": "fs_pairec",
                "FeatureStoreViewName": "rec_sln_demo_item_table_preprocess_all_feature_v1"
            },
            "FilterParams": [
                {
                    "Name": "click_count",
                    "Type": "int",
                    "Operator": "greater",
                    "Value": "15"
                },
                {
                    "Name": "category",
                    "Type": "string",
                    "Operator": "in",
                    "Value": "user.category"
                }
            ]
        }

In the preceding example, the user. prefix in the Value field specifies that the value is retrieved from a user feature. For instance, user.category retrieves the value from the user's category feature.

For more information about how to use the Operator parameter, see the appendix.

Hologres

I2I recall is used for similarity recommendation. To retrieve similar items, you must pass the item_id of a source item. For the API definition, see API testing.

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "I2IRecall",
            "RecallType": "ItemCollaborativeFilterRecall",
            "RecallCount": 500,
            "ItemCollaborativeDaoConf": {
                "AdapterType": "hologres",
                "HologresName": "holo_info",
                "Item2ItemTable": "item_collaborative_list"
            }
        }
    ]
}

The following table describes the configuration parameters.

The Item2ItemTable table is defined as follows:

PAI-FeatureStore

I2I recall is used for similarity recommendation. To retrieve similar items, you must pass the item_id of a source item. For the API definition, see API testing.

The Item2ItemTable table is defined as follows:

Configuration example:

{
    "RecallConfs": [
        {
            "Name": "I2IRecall",
            "RecallType": "ItemCollaborativeFilterRecall",
            "RecallCount": 200,
            "ItemCollaborativeDaoConf": {
                "AdapterType": "featurestore",
                "FeatureStoreName": "fs_pairec",
                "FeatureStoreViewName": "i2i_recall"
            }
        }
    ]
}

The following table describes the configuration parameters.

If the item_id passed to the API is in a multi-value format and is delimited by ,, you can refer to the following configuration:

{
    "RecallConfs": [
        {
            "Name": "I2IRecall",
            "RecallType": "ItemCollaborativeFilterRecall",
            "RecallCount": 200,
            "ItemCollaborativeDaoConf": {
                "AdapterType": "featurestore",
                "FeatureStoreName": "fs_pairec",
                "FeatureStoreViewName": "i2i_recall"
            },
            "EnableMultipleItemId": true,
            "MultipleItemIdDelimiter": ","
        }
    ]
}