AI_TRANSLATE - Realtime Compute for Apache Flink - Alibaba Cloud Documentation Center

This topic describes how to use the AI_TRANSLATE function to translate text.

Limitations

This function requires Ververica Runtime (VVR) 11.4+.
The throughput of AI_TRANSLATE operators is subject to the rate limits of Alibaba Cloud Model Studio. When the rate limits for a model are reached, the Flink job will be backpressured with AI_TRANSLATE operators as the bottleneck. In some cases, timeout errors and job restarts may be triggered.

Syntax

AI_TRANSLATE(
  MODEL => MODEL <MODEL NAME>, 
  INPUT => <INPUT COLUMN NAME>,
  SOURCE_LANG => <SOURCE LANGUAGE>,
  TARGET_LANG => <TARGET LANGUAGE>
)

Input parameters

Parameter	Data type	Description
MODEL <MODEL NAME>	MODEL	The name of the registered model. For more information, see Model settings. Note: The output type of the model must be VARIANT.
<INPUT COLUMN NAME>	STRING	The source text for the model to analyze.
<SOURCE LANGUAGE>	STRING	The abbreviation for the source language. For example, enter `en` for English. If you enter `auto`, the model automatically detects the language. The following source languages are supported: Chinese (`zh`), English (`en`), Japanese (`ja`), Korean (`ko`), French (`fr`), German (`de`), Spanish (`es`), Russian (`ru`), Arabic (`ar`), and Portuguese (`pt`). Note: The supported language types are determined by the model. This input parameter must be a constant.
<TARGET LANGUAGE>	STRING	The abbreviation for the target language. Fo example, `en`. The following target languages are supported: Chinese (`zh`), English (`en`), Japanese (`ja`), Korean (`ko`), French (`fr`), German (`de`), Spanish (`es`), Russian (`ru`), Arabic (`ar`), and Portuguese (`pt`). Note: The supported language types are determined by the model. This input parameter must be a constant.

Return values

Parameter	Data type	Description
translated_text	STRING	The translation.
detected_language	STRING	The source language detected.

Examples

Sample data

id	content
1	Apache Flink is an open-source, distributed framework over unbounded and bounded data streams.

Sample statements

The sample SQL statements use the Qwen-Plus model and the AI_TRANSLATE function to translate the input text.

CREATE TEMPORARY MODEL general_model
INPUT (`input` STRING)
OUTPUT (`content` VARIANT)
WITH (
    'provider' = 'openai-compat',
    'endpoint'='<YOUR ENDPOINT>',
    'apiKey' = '<YOUR KEY>',
    'model' = 'qwen-plus'
);

CREATE TEMPORARY VIEW infos(id, content)
AS VALUES (1, 'Apache Flink is an open-source, distributed framework over unbounded and bounded data streams.');

-- Use positional argument to call AI_TRANSLATE
SELECT id, translated_text, detected_language
FROM infos,
LATERAL TABLE(
  AI_TRANSLATE(
    MODEL general_model, 
    content, 
    'auto',
    'zh'
    ));
   
-- Use named argument to call AI_TRANSLATE
SELECT id, translated_text, detected_language
FROM infos,
LATERAL TABLE(
  AI_TRANSLATE(
    MODEL => MODEL general_model, 
    INPUT => content, 
    SOURCE_LANG =>'auto',
    TARGET_LANG => 'zh'
    ));

Output

id	translated_text	detected_language
1	Apache Flink 是一个开源的、分布式的框架，支持无界和有界数据流的处理。	en