Model settings - Realtime Compute for Apache Flink - Alibaba Cloud Documentation Center

Notes

Supports large model services from Flink AI service (built-in models), Alibaba Cloud Model Studio, Platform for AI (PAI), and other services with OpenAI-compatible APIs.
Large model services deployed on Platform for AI (PAI) must be in the same region as your Realtime Compute for Apache Flink service.
Requires Ververica Runtime (VVR) version 11.1 or later.

Create a model

Register a model

In the Data Query editor, enter the following command.

CREATE [TEMPORARY] MODEL [catalog_name.][db_name.]model_name
INPUT ( { <physical_column_definition> [, ...n] )
OUTPUT ( { <physical_column_definition> [, ...n] )
WITH (key1=val1, key2=val2, ...)

<physical_column_definition>:
  column_name column_type [COMMENT column_comment]

Clause	Description	Key parameters	Schema constraints	Example
INPUT	Defines the columns, data types, and order of the model's input data.	`column_name` (column name) `column_type` (data type) `COMMENT` (comment)	Requires exactly one column of type STRING.	INPUT (`input_text` STRING COMMENT 'User comment')
OUTPUT	Defines the columns, data types, and order of the model's output data.	`column_name` (column name) `column_type` (data type) `COMMENT` (comment)	Constraints vary by model task type: chat/completions: Requires exactly one output column. The type must be `STRING` for `ML_PREDICT` calls, or `VARIANT` for specialized AI functions (e.g., `AI_CLASSIFY`, `AI_SUMMARIZE`). embeddings: Requires exactly one column of type ARRAY<FLOAT>.	OUTPUT (`sentiment_label` STRING COMMENT 'Sentiment label')
WITH	See WITH parameters.	`provider` (service provider) `endpoint` (endpoint) `api-key` (API key) `model` (model identifier)	None.	`WITH ('provider'='openai-compat', 'endpoint'='${ENDPOINT}', 'model'='qwen-turbo', 'api-key'='${KEY}')`

Examples

Built-in model

If you have activated the Flink AI service for your Alibaba Cloud account in the target region, you can create a model using the built-in model mode.

CREATE TEMPORARY MODEL my_llm
INPUT (prompt String COMMENT 'Input prompt')
OUTPUT (response String COMMENT 'Text generated by the model')
WITH (
  'provider' = 'openai-compat',
  'task' = 'chat/completions',
  'model' = 'qwen3.5-flash'
);

You do not need to specify the endpoint parameter: the system automatically selects the corresponding server endpoint.
You do not need to specify the api-key parameter because the system uses the Flink-managed API-Key for authentication.
Specify the task parameter to declare the task type of the model.

Platform for AI (PAI)

CREATE MODEL model_pai
INPUT (`input` String)
OUTPUT (`embedding` ARRAY<FLOAT>) 
WITH (
  'provider'='openai-compat',
  'endpoint'='<VPC endpoint>',
  'api-key'='<Token>',
  'model'='qwen3-235b-a22b'
);

To get the endpoint and api-key, first deploy a model service. For more information, see Deploy DeepSeek-V4 and DeepSeek-R1 models with one click and Quick start for Elastic Algorithm Service (EAS).

Model Gallery

Log on to the Platform for AI (PAI) console.
In the left navigation pane, choose Model Gallery > Task Management > Deployment Task, and click the name of the target service.
Click View Invocation Information.
- The VPC endpoint uses HTTP. You must change it to HTTPS and append /v1/<task> to the URL. The task parameter specifies the model task type. Valid values are:
  - chat/completions
  - embeddings
  For example: https://************.vpc.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/quickstart_deploy_20250722_7b22/v1/chat/completions.
- Use the Token for the api-key parameter.

Elastic Algorithm Service (EAS)

In the left navigation pane, choose Elastic Algorithm Service (EAS) > Inference Services, and click the name of the target service to go to the Overview page.
In the Basic Information section, click View Invocation Information.
In the Invocation Information panel, copy the endpoint and token.
- The VPC endpoint uses HTTP. You must change it to HTTPS and append /v1/<task> to the URL. The task parameter specifies the model task type. Valid values are:
  - chat/completions
  - embeddings
  For example: https://************.vpc.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/quickstart_deploy_20250722_7b22/v1/chat/completions.
- Use the Token for the api-key parameter.

For more information, see Invoke a service over the internet or an internal network by using a gateway.

Alibaba Cloud Model Studio

Note

For the most stable and convenient access to Alibaba Cloud Model Studio models, use the built-in model mode. This mode supports multiple regions in and outside the Chinese mainland and automatically handles cross-region access.

CREATE MODEL model_bailian
INPUT (`input` String)
OUTPUT (`content` String)
WITH (
  'provider'='openai-compat',
  'endpoint'='<Endpoint>',
  'api-key'='<bailian-key>',
  'model'='qwen3-235b-a22b'
);

The service endpoint for models on the Bailian platform is in the following format: <base-url>/compatible-mode/v1/<task>. For example, https://dashscope-intl.aliyuncs.com/compatible-mode/v1/chat/completions.

base-url:

Access over the internet: https://dashscope-intl.aliyuncs.com. To access the service over the internet, enable internet access for your Flink workspace. For more information, see Select a network connection type.

Access over an internal network: Access Alibaba Cloud Model Studio model or application APIs over an internal network.

Access is supported from the same region and across regions. To access the service from a different region, for example, a Flink instance in the China (Shanghai) region accessing an Alibaba Cloud Model Studio service in the China (Beijing) region, you must configure Cloud Enterprise Network (CEN). For more information, see Cross-region access to Alibaba Cloud Model Studio APIs over an internal network.

Important
Alibaba Cloud Model Studio processes data in the selected region. If you have data residency requirements, select an appropriate region based on your compliance needs. For more information, see Select a region and service scope.
task: The model task type. The following values are supported:
- chat/completions
- embeddings

WITH parameters

General

Parameter	Description	Type	Required	Default	Notes
provider	The model service type.	String	Yes	None	VVR 11.1 to 11.2: The fixed value is `bailian`. VVR 11.3 ~ 11.7: Valid values are `openai-compat` or `bailian`. We recommend that you use `openai-compat`. Note For Platform for AI (PAI) or other OpenAI-compatible model services, the value must be `openai-compat`.
task	The model task type.	String	No	None	Valid values: chat/completions embeddings Important If you use the built-in model mode, this parameter is required to declare the task type. Supported in VVR 11.7 and later.
endpoint	The endpoint for the OpenAI-compatible model service, such as an embeddings or chat/completions service.	String	No	None	For Alibaba Cloud Model Studio or PAI endpoints, see Examples. For other OpenAI-compatible model services, refer to the API documentation of the service. Note This parameter is not required when you use the built-in model mode.
api-key	The API key used to access the model service.	String	No	None	For more information, see Obtain an API Key. Previous key name: apiKey (VVR 11.1). Note This parameter is not required when you use the built-in model mode.
max-context-size	The maximum context size for a single request, in tokens.	Integer	No	None	If the maximum size is exceeded, the action defined in `context-overflow-action` is triggered. Note Supported in VVR 11.2 and later.
context-overflow-action	The action to take when the context of a single request exceeds the maximum size.	String	No	`truncated-tail`	Valid values: `truncated-tail`: Automatically truncates tokens from the tail when the capacity is exceeded, retaining the most recent `max-context-size` tokens. No logs are recorded. `truncated-tail-log`: Automatically truncates excess tokens from the tail, retaining the most recent `max-context-size` tokens. Logs the truncation event. `truncated-head`: Trims the earliest tokens from the beginning, retaining the most recent `max-context-size` tokens. `truncated-head-log`: Truncates the earliest tokens from the beginning, keeping the latest `max-context-size` tokens. A truncation log is recorded. `skipped`: The data record is discarded. No logs are recorded. `skipped-log`: Discards the record and creates a log entry. Note Supported in VVR 11.2 and later.
max-context-size	Maximum context length (number of tokens)	int	No	None	Note Only VVR 11.2 and later versions are supported.
context-overflow-action	Behavior when the context length exceeds the limit	String	No	truncated-tail	Valid values `truncated-tail`: Automatically truncates excess data from the tail, retaining the most recent `max-context-size` items. `truncated-head`: Trims the earliest data from the head and retains the latest `max-context-size` items. `skipped`: Discards new data that exceeds the capacity and does not update the context. `truncated-tail-log`: Extends truncated-tail by logging the action of truncating the context. `truncated-head-log`: Extends `truncated-head` to log the context truncation behavior. `skipped-log`: Has the same behavior as skipped, but also logs when the context is truncated. Note Supported only in VVR 11.2 or later.
error-handling-strategy	The strategy for handling model request errors.	String	No	retry	Valid values: `retry`: Resends the request. `failover`: Throws an exception. `ignore`: Ignores the exception and skips the data record. Note Supported in VVR 11.4 and later.
retry-num	The number of retry attempts.	int	No	100	Takes effect only when `error-handling-strategy = retry`. Note Supported in VVR 11.4 and later.
retry-fallback-strategy	The fallback strategy to apply after all retry attempts fail.	String	No	failover	Valid values: `failover`, `ignore`. This takes effect only when `error-handling-strategy` is set to a value other than `retry`. Note Supported in VVR 11.4 and later.
retry-backoff-strategy	The retry backoff strategy. This defines how the time interval between retries is calculated.	String	No	fixed	Valid values: `fixed`: Uses a fixed interval between retries. `exponential`: Uses an exponentially increasing interval between retries. Note Supported in VVR 11.4 and later.
retry-backoff-base-interval	The base time interval for the retry backoff strategy.	Duration	No	1 s	Note Supported in VVR 11.4 and later.

chat/completions

The following parameters apply to the chat/completions model task type:

Parameter	Description	Type	Required	Default	Notes
model	The specific model to invoke on the service endpoint.	String	Yes	None	Supports Select model category models. Note You are charged based on the selected model and the number of tokens in the input and output.
system-prompt	The system prompt for the request.	String	Yes	"You are a helpful assistant."	Previous key name: systemPrompt (VVR 11.1). Note In VVR 11.6 and later, you can set this parameter to an empty value.
temperature	Controls the smoothness of the probability distribution for each candidate token during text generation.	Float	No	None	Valid range: [0, 2). A value of 0 is not recommended. A higher temperature value smooths the probability distribution, allowing less likely tokens to be selected and making the output more diverse. A lower value sharpens the distribution, making more likely tokens easier to select and making the output more deterministic.
top-p	The probability threshold for nucleus sampling during generation.	Float	No	None	A higher value increases randomness, while a lower value increases determinism. Previous key name: topP (VVR 11.1).
stop	A stop sequence.	String	No	None	The model stops generation when it encounters this sequence.
max-tokens	The maximum number of tokens that the model can generate.	Integer	No	None	Previous key name: maxTokens (VVR 11.1).
content-type	The type of the input data.	String	No	text	Content type of the single input column. The value can be `text` (default) or `image_url`. Mutually exclusive with `content-types`. Note Supported in VVR 11.6 and later.
content-types	Content types of the input columns for multimodal models.	String	No	N/A	A semicolon-separated list, one type per input column (e.g., `text;image_url`). Supported types: `text`, `image_url`, `multi_image_urls`. Mutually exclusive with `content-type`. Note Supported in VVR 11.8 and later. For more information, see General invocation.
presence-penalty	Controls token repetition.	Double	No	None	Valid range: -2.0 to 2.0. Positive values penalize new tokens based on whether they have appeared in the text so far, which encourages the model to discuss new topics. Note Supported in VVR 11.3 and later.
n	The number of output choices to generate for each input.	int	No	None	Note Supported in VVR 11.3 and later.
seed	A random number seed for the model's response.	Long	No	None	If specified, the model provider makes a best effort to sample deterministically, so that repeated requests with the same seed and parameters return the same result. Note Supported in VVR 11.3 and later.
response-format	The format of the return value.	String	No	text	Valid values: text json_object Note Supported in VVR 11.3 and later.
extra-header	Extra HTTP headers for the request.	String	No	None	Must be a JSON-formatted string where values are strings or lists of strings. Note Supported in VVR 11.3 and later.
extra-body	Extra HTTP body for the request.	String	No	None	Must be a JSON-formatted string. Note Supported in VVR 11.3 and later.
user-prompt	The user prompt for the request.	String	No	None	Similar to `system-prompt`, but submitted as a user role. Note Supported in VVR 11.7 and later.

embeddings

The following parameters apply to the embeddings model task type:

Parameter	Description	Type	Required	Default	Note
model	The specific model to invoke on the service endpoint.	String	Yes	None	Supports Select model models. Note You are charged based on the selected model and the number of tokens in the input and output.
dimension	The vector dimension of the output embeddings.	Integer	No	None	The value depends on the supported vector dimensions of the specific model. Common values include 1024, 768, and 512.

Query models

In the Data Query editor, run one of the following commands.

List the names of registered models.

SHOW MODELS [ ( FROM | IN ) [catalog_name.]database_name ];

Show a model's creation statement.

SHOW CREATE MODEL [catalog_name.][db_name.]model_name;

Show a model's input and output schema.

DESCRIBE MODEL [catalog_name.][db_name.]model_name;

Example

SHOW MODELS;

-- RESULT
--+------------+
--| model name |
--+------------+
--|          m |
--+------------+

DESCRIBE MODEL m;

-- RESULT
-- +---------+--------+------+----------+
-- |    name |   type | null | is input |
-- +---------+--------+------+----------+
-- | content | String | TRUE |     TRUE |
-- |   label | BIGINT | TRUE |    FALSE |
-- +---------+--------+------+----------+

Modify a model

Run the following command in the Data Query editor.

ALTER MODEL [IF EXISTS] [catalog_name.][db_name.]model_name {
  RENAME TO new_table_name
  SET (key1=val1, ...)
  RESET (key1, ...)
}

Examples

Rename a registered model:

ALTER MODEL m RENAME TO m1; -- Renames the model to m1.

Modify a model parameter:

ALTER MODEL m SET ('endpoint' = '<Your_Endpoint>'); -- Updates the endpoint path.

Reset a model parameter to its default:

ALTER MODEL m RESET ('endpoint'); -- Resets the endpoint path.

Delete a model

Run the following command in the Data Query editor:

DROP [TEMPORARY] MODEL [IF EXISTS] [catalog_name.][db_name.]model_name

Example

DROP MODEL m;

Built-in models

Name	Task	Description	Input type	Output type
qwen3.6-plus	chat/completions	Visual understanding and text generation. Latest flagship Qwen model.	String (text, image, video)	String
qwen3.6-flash	chat/completions	Visual understanding and text generation. Cost-effective model.	String (text, image, video)	String
qwen3.5-plus	chat/completions	Visual understanding and text generation. High-performance model.	String (text, image, video)	String
qwen3.5-flash	chat/completions	Visual understanding and text generation. Optimized for speed and low cost.	String (text, image, video)	String
qwen-vl-ocr-latest	chat/completions	Image text extraction (a vertical model for OCR).	String (image)	String
qwen-mt-flash	chat/completions	Text translation.	String (text)	String
text-embedding-v4	embeddings	Text embedding.	String (text)	ARRAY<FLOAT>
qwen3-vl-embedding	embeddings	Multimodal embedding.	String (image, text, video)	ARRAY<FLOAT>

Note

If the specified model is not supported, the system reports an error and lists the supported models.