Platform For AI:Recall configuration

2.1 How it works

DSSM is a two-tower deep learning model that encodes users and items into fixed-dimension vectors and retrieves candidates by vector similarity.

    User Features         Item Features
         |                     |
   ┌─────┴──────┐        ┌─────┴─────┐
   │ User Tower │        │ Item Tower│
   │  (DNN)     │        │  (DNN)    │
   └─────┬──────┘        └─────┬─────┘
    user_embedding         item_embedding
         └──── dot product ────┘
               Match Score

2.2 Core advantages

2.3 Key parameters

2.4 Embedding dimension policy

The embed_dim_policy parameter determines embedding dimensions for discrete features. In the following formulas, N is the feature cardinality.

Calculation Example (Assume feature cardinality N = 10,000):

• EMB_SQRT4_STEP4: round(10000^0.25 / 4) × 4 = round(10 / 4) × 4 = 8

• EMB_LN_STEP4: round(ln(10000) / 4) × 4 = round(9.21 / 4) × 4 = 8

2.5 DSSM features in TorchEasyRec

TorchEasyRec DSSM supports the following capabilities:

Negative Sampling Configuration:

{
    "negative_sampler": {
        "sampler_type": "negative_sampler",
        "num_sample": 1024
    }
}

Negative sampling improves discriminative capability by dynamically sampling negative examples during training.

2.6 TorchEasyRec service architecture

TorchEasyRec service components:

• FG encoding: Ensures offline-online feature consistency.

• User Tower: A Torch ScriptModule for real-time user vector inference.

• FAISS index: Efficient CPU-based retrieval, deployable on PAI-EAS CPU instances.

• TorchEasyRec Processor: The easyrec-torch-{version} processor. Available versions are listed in the version list.

Deployment Configuration Example:

{
  "processor": "easyrec-torch-1.12",
  "model_config": {
    "fg_mode": "normal",
    "faiss_neigh_num": 500,
    "faiss_nprobe": 1000,
    "steady_mode": true,
    "period": 2880
  }
}

3.1 Global hot-item recall

Principle: Returns the most popular items across the entire site based on user behavior statistics.

Formula:

$$hot\_score=\Sigma (bhv\_weight\times decay(time))$$

Use cases:

• Cold start for new users with no behavior history.

• A fallback recall strategy.

• Distributing popular content.

Key Configurations:

3.2 User-grouped hot-item recall

Principle: Groups users by attributes such as gender, age group, and location, then identifies the most popular items within each group.

Use cases:

• On the Guess You Like homepage.

• Scenarios where different user groups have distinct preferences.

Advantages:

• More personalized than global hot-item recall.

• Simple to implement and does not require user behavior history.

3.3 Item-grouped hot-item recall

Principle: Groups items by attributes such as category and tag, then identifies the most popular items within each group.

Use cases:

• In the Related Recommendation scenario on a detail page, this supplements results when other algorithms such as ETrec I2I return insufficient related items.

• Recommending popular items from the same category as the current item.

4.1 ETrec U2I recall

Reference: Collaborative Filtering (etrec)

This item-based collaborative filtering (Item-based CF) algorithm calculates item co-occurrence similarity from user-item interactions and recommends items similar to those in a user's history.

Key Features:

• Supports real-time behavior triggers (U2I Trigger).

• Applies time decay to prioritize recent behaviors.

• Combines multiple behavior types by weighting clicks, favorites, and cart additions.

Key Configurations:

4.2 ETrec I2I recall

Principle: An item-based collaborative filtering algorithm that calculates similarity between items based on co-occurrence or embeddings.

Use cases:

• Related Recommendation (e.g., "Frequently Bought Together").

• Recommending similar items on a detail page.

Similarity Calculation:

• Co-occurrence-based: Based on how many times the same user interacts with two items.

• Vector-based: Based on the cosine similarity of item embeddings.

4.3 Swing U2I recall

Reference: Swing Algorithm Tool

Principle: An improved collaborative filtering algorithm that leverages the Swing structure in a user-item bipartite graph to measure similarity.

Core Idea:

• Two items are considered similar if many common users interact with them.

• Penalizes highly active users by reducing their weight.

Advantages:

• Reduces the over-recommendation of popular items.

• Helps discover associations for long-tail items.

4.4 SimRank U2I recall

Reference: SimRank++ Similarity Calculation Algorithm

⚠️ Note: SimRank is resource-intensive due to multiple iterations and full-graph random walks. Use MaxCompute subscription resources to reduce costs.

Principle: A graph-based similarity algorithm that measures node similarity through random walks on a user-item bipartite graph.

Core Idea:

• Two users are similar if they interact with many similar items.

• Two items are similar if similar users interact with them.

Use cases:

• Sparse user behavior data.

• Indirect relationship discovery.

Characteristics:

• Iterative with potentially slow convergence.

• Captures multi-hop relationships.

5.1 Why differentiate scenarios?

Different scenarios call for different recall strategies because user intents and goals vary:

5.2 Guess you like

Applicable algorithms:

• DSSM vector recall: The core algorithm for deep personalization.

• ETrec U2I: Collaborative filtering based on user behavior history.

• Swing U2I: For discovering long-tail interests.

• SimRank U2I: For discovering indirect relationships.

• User-grouped hot items: A fallback for cold-start users.

Configuration Characteristics:

• Emphasis on user personalization

• Fusion of multiple recall channels

• Controlling recommendation diversity

5.3 Related recommendation

Applicable algorithms:

• ETrec I2I: Collaborative filtering based on item similarity.

• Item-grouped hot items: For recommending popular items from the same category.

• DSSM vector recall: For matching items based on semantic similarity.

Configuration Characteristics:

• Triggered by the currently viewed item

• Emphasizes content relevance

• Prioritizes items in the same category or topic

5.4 Algorithm-scenario mapping

6.1 Selection by business stage

6.2 Multi-channel recall fusion

In production, you typically combine multiple recall channels for better coverage.

Fusion strategies:

• Weighted fusion: Assigns different weights to each recall channel.

• Deduplication fusion: Deduplicates by item ID and retains the highest score.

• Quota fusion: Assigns a fixed slot count to each recall channel.