DeepFM algorithm - PolarDB - Alibaba Cloud Documentation Center

Overview

DeepFM combines a deep neural network (DNN) with a factorization machine (FM) to capture both low-order explicit feature interactions and high-order implicit feature interactions. DeepFM requires no manual feature engineering and is commonly used in recommendation systems and advertising systems.

Input features typically fall into two categories:
- Categorical features: string values, such as gender (male or female) or product category (clothing, toys, electronics).
- Numerical features: integer or floating-point values, such as user activity level or product price.
The output is typically a floating-point number between 0 and 1, representing the probability of the target value being 1. The output can be used for ranking or binary classification.

Use cases

DeepFM is commonly used in classification or ranking scenarios. It is especially effective when manually constructed features cannot directly reflect outcomes. In various recommendation scenarios, both low-order and high-order feature interactions affect user behavior. Because these interactions are often unclear to humans, DeepFM automatically learns them.

For example, in a personalized product recommendation scenario, a click-through rate (CTR) prediction model is required. Historical user behavior data (clicks, impressions without clicks, purchases) is used as training data to predict the probability of a user clicking or purchasing. When user behavior data is extensive and does not directly indicate future click or purchase patterns, DeepFM combines user behavior features and converts sparse features into high-dimensional dense representations.

Parameters

The following table lists the parameters that you can specify in the CREATE MODEL syntax by using the model_parameter clause. Select the parameters that suit your use case.

Parameter	Description
metrics	The evaluation metric for the model. Valid values: accuracy (default): the accuracy metric for classification models. binary_crossentropy: the cross-entropy metric for binary classification problems. mse: the mean squared error metric for regression models.
loss	The loss function for the learning task. Valid values: binary_crossentropy (default): the cross-entropy loss for binary classification problems. mean_squared_error: the mean squared error loss for regression models.
optimizer	The optimizer. Valid values: adam (default): combines the advantages of AdaGrad (adaptive learning rate gradient descent) and momentum gradient descent. It handles sparse gradients well (suitable for natural language processing and computer vision) and mitigates gradient oscillation. sgd: stochastic gradient descent. rmsprop: an improvement over AdaGrad that introduces a weight parameter and uses a weighted sum for the step size accumulation.
validation_split	The fraction of data used for cross-validation. Default value: 0.2.
epochs	The number of training iterations. Default value: 6.
batch_size	The number of training samples per batch. Default value: 64. A smaller batch size is more prone to overfitting, but a larger batch_size increases memory consumption per iteration. For large datasets, consider increasing this value (for example, to 128 or 256) to improve training efficiency. For small datasets or memory-constrained environments, consider decreasing it.
learning_rate	The learning rate that controls the magnitude of parameter updates at each iteration. This parameter should be tuned together with epochs: when increasing epochs, you typically need to decrease learning_rate to prevent the model from diverging or failing to converge. Conversely, when decreasing epochs, you can increase the learning rate.
dnn_hidden_units	The architecture of the DNN hidden layers. Example: `[64,32]` specifies two hidden layers with 64 and 32 neurons respectively. If the default network structure is too complex for your dataset, training may be slow and prone to failure. We recommend simplifying the network structure based on the number of input features and dataset size. Use smaller hidden layers when you have fewer features.
task	The task type. Valid values: binary (default): classification model. regression: regression model.

Note

Best practices for parameter configuration:

We recommend explicitly specifying key parameters such as loss, metrics, and batch_size to avoid unexpected training behavior caused by relying on default values.
batch_size and epochs should be tuned together: when increasing batch_size, consider decreasing epochs; when decreasing batch_size, increase epochs to ensure sufficient training.
learning_rate and epochs should be tuned together: when increasing epochs, decrease the learning rate to prevent divergence.
dnn_hidden_units should be set based on the number of features. Avoid using overly deep network structures when you have fewer features.

Examples

Train a model

/*polar4ai*/CREATE MODEL airline_deepfm WITH
(model_class = 'deepfm',
x_cols = 'Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',
y_cols='Delay',model_parameter=(epochs=6))
AS (SELECT * FROM db4ai.airlines);

Evaluate the model

/*polar4ai*/SELECT Airline FROM EVALUATE(MODEL airline_deepfm, 
SELECT * FROM db4ai.airlines LIMIT 20) WITH 
(x_cols = 'Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay',metrics='acc');

Run predictions

/*polar4ai*/SELECT Airline FROM PREDICT(MODEL airline_deepfm,
SELECT * FROM db4ai.airlines limit 20) WITH
(x_cols = 'Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length');

FAQ

Q: How do I troubleshoot slow or failed DeepFM model training?

A: If DeepFM model training takes an unusually long time or eventually fails, troubleshoot the issue from the following aspects:

Possible cause	Troubleshooting and solution
`batch_size` is too small: the default value is 64, which may result in slow processing for large datasets.	Increase `batch_size` to 128 or 256 to improve per-batch processing efficiency.
`learning_rate` and `epochs` are mismatched: an excessively high learning rate causes model divergence, while an excessively low learning rate prevents convergence.	Adjust `learning_rate`: when increasing `epochs`, decrease the learning rate; when decreasing `epochs`, increase the learning rate.
`dnn_hidden_units` defines an overly complex model structure: when the dataset is insufficient to support a complex network, training is slow and prone to failure.	Simplify the network structure. For example, change `[128,64,32]` to `[64,32]` and adjust the hidden layer size based on the number of input features.
Key parameters are not explicitly specified: relying on default values may result in unexpected training behavior.	Explicitly configure key parameters such as `batch_size`, `loss`, and `metrics` to ensure predictable training behavior.
Data quality issues: the training data contains a large number of missing values, outliers, or unevenly distributed features.	Check the training data quality. Fill in missing values or remove outlier samples before retraining the model.