This topic describes how to use the ML_PREDICT function for AI inference.
Limits
Only Ververica Runtime (VVR) 11.1 or later is supported.
The throughput of ML_PREDICT operators is subject to the rate limits of Alibaba Cloud Model Studio. When the rate limits for a model are reached, the Flink job will be backpressured with ML_PREDICT operators as the bottleneck. In some cases, timeout errors and job restarts may be triggered.
Syntax
ML_PREDICT(TABLE <TABLE NAME>, MODEL <MODEL NAME>, DESCRIPTOR(<INPUT COLUMN NAMES>))Input parameters
Parameter | Data type | Description |
TABLE <TABLE NAME> | TABLE | The input stream for model inference. It can be a physical table or a view. |
MODEL <MODEL NAME> | MODEL | The registered model's name. |
DESCRIPTOR(<INPUT COLUMN NAMES>) | N/A | The columns in the input data used for model inference. |
Example
Test data
id | movie_name | user_comment | actual_label |
1 | Her Story | My favourite part was when the kid guessed the sounds. It was romantic, heartwarming, and loving. | POSITIVE |
2 | The Dumpling Queen | Nothing special. | NEGATIVE |
Test statement
The following SQL code registers the Qwen-Turbo model and uses it to predict the sentiment categories for movie reviews through the ML_PREDICT function.
CREATE TEMPORARY MODEL ai_analyze_sentiment
INPUT (`input` STRING)
OUTPUT (`content` STRING)
WITH (
'provider'='bailian',
'endpoint'='<YOUR ENDPOINT>',
'apiKey' = '<YOUR KEY>',
'model'='qwen-turbo',
'systemPrompt' = 'Classify the text below into one of the following labels: [positive, negative, neutral, mixed]. Output only the label.'
);
CREATE TEMPORARY VIEW movie_comment(id, movie_name, user_comment, actual_label)
AS VALUES (1, 'Her Story', 'My favourite part was when the kid guessed the sounds. It was romantic, heartwarming, and loving.', 'positive'), (2, 'The Dumpling Queen', 'Nothing special.', 'negative');
SELECT id, movie_name, content as predict_label, actual_label
FROM ML_PREDICT(TABLE movie_comment, MODEL ai_analyze_sentiment, DESCRIPTOR(user_comment));Output results
The prediction results in the predict_label column match the actual results in the actual_label column.
id | movie_name | predict_label | actual_label |
1 | Her Story | POSITIVE | POSITIVE |
2 | The Dumpling Queen | NEGATIVE | NEGATIVE |