The AI Function in EMR Serverless Spark provides built-in SQL and PySpark functions for tasks like sentiment analysis, translation, and information extraction. It calls large models through a unified interface to seamlessly integrate data and AI. - E-MapReduce

AI Function integrates large language model (LLM) capabilities into Serverless Spark, enabling data engineers to call large models from SQL or PySpark for tasks such as sentiment analysis, translation, and information extraction, without SDKs, APIs, or model management.

Prerequisites

Activate AI Center.
For jobs submitted through sessions or as batch jobs, use engine version esr-4.7.0 or later.
For jobs submitted through Livy Gateway or Kyuubi Gateway, use engine version esr-4.9.0 or later.
Add the spark.emr.serverless.ai.function.enable true Spark configuration when submitting a job or creating a session.

Function overview

This table lists the built-in AI Functions in Serverless Spark.

Function	Description	Default model	Use cases
`ai_query()`	A general-purpose interface to invoke models, supporting custom prompts, model selection, and generation parameters.	qwen3.6-plus	Batch summarization, information extraction, and calling external or self-hosted model endpoints.
`ai_sentiment()`	Performs sentiment analysis (positive, negative, or neutral).	qwen3.6-plus	Public opinion monitoring, customer service comment analysis, and user feedback categorization.
`ai_fix_grammar()`	Corrects grammar and word choice.	qwen3.6-plus	Automatically polishes text before generating external-facing content.
`ai_classify()`	Performs multi-class or multi-label classification.	qwen3.6-plus	Ticket categorization (complaint, inquiry, or suggestion), news topic identification, and log type annotation.
`ai_extract()`	Extracts fields from text into a predefined STRUCT.	qwen3.6-plus	Key information extraction (such as client, amount, and date) and comment attribute parsing (such as product, issue, and satisfaction).
`ai_similarity()`	Calculates the semantic similarity score (0.0–1.0) between two text segments.	qwen3.6-plus	Deduplication, retrieval matching, question-answer pair similarity scoring, and retrieving similar documents for Retrieval-Augmented Generation (RAG).
`ai_translate()`	Translates text into a specified language.	qwen3.6-plus	End-to-end multilingual reporting and cross-border use cases.
`ai_sql_optimize()`	Optimizes SQL queries with rewrite and annotation suggestions.	qwen3.6-plus	Provides automatic suggestions for predicate pushdown, bucketing/partitioning, and join order.
`ai_sql_hive_to_spark()`	Converts HiveQL to Spark SQL-compatible syntax.	qwen3.6-plus	Hive job migration, UDF syntax adaptation, and window function standardization.
`ai_embedding()`	Generates semantic embeddings for text.	text-embedding-v4	RAG vector database construction, semantic clustering, similarity-based retrieval, and feature engineering.
`ai_embedding_multimodal()`	Converts images into vectors. Note This AI Function is only available in the Indonesia (Jakarta) region.	tongyi-embedding-vision-plus	Cross-modal image-text retrieval, product visual search, semantic image clustering and deduplication, and multimodal RAG knowledge base construction.

Function: `ai_query()`

The ai_query() function is a general-purpose interface provided by AI Function to invoke a large language model with a custom prompt. It supports flexible integration with built-in models and external services, making it ideal for advanced scenarios that task-specific functions cannot handle, such as complex reasoning, multi-turn instructions, and structured output.

Syntax

ai_query(prompt [, service_name] [, options][, data][,data_type])

Parameters

Parameter	Type	Required	Description
`prompt`	STRING	Yes	The prompt for the model. It should include context, instructions, and input data. Explicitly specifying the output format improves stability.
`service_name`	STRING	No	The name of the Model Service to invoke. • If omitted, the function uses the default `qwen3.6-plus` model. • For service names registered with Model Service, see Model Service.
`options`	STRING	No	A JSON string containing additional parameters that control generation behavior. For a list of supported parameters, see Options parameters.
`data`	BINARY/STRING	No	The image data column to process. Required only when processing image data. The value you pass depends on the `data_type` setting: If `data_type` is `binary` (default): The image's binary data (BINARY type) or its HTTP or HTTPS URL. Examples include the `content` column read by the `read_files` function, or a publicly accessible URL of the image. If `data_type` is `uri`: Pass a file path (STRING type). Supported formats include an OSS path (for example, `oss://bucket/path/to/image.jpg`) or a local file path (for example, `file:///mnt/data/path/to/image.jpg`). Using a local file path requires the Spark-managed mount feature.
`data_type`	STRING	No	Specifies how image data is passed. This parameter applies only when processing image data. Valid values are: `binary` (default): The `data` parameter contains the image's binary data (BINARY type) or its HTTP or HTTPS URL. `uri`: The `data` parameter contains a file path (STRING type); the function reads the file content internally.

Note

Optional parameters support named parameter notation.

Return type: STRING.

Examples

Use the default model

-- Call ai_query() without specifying a model.
select
  ai_query ('Determine if the following comment is positive or negative:\n This is quite bad');

Result:

This comment is negative. Reason: The phrase "quite bad" is a clear negative expression with a strong tone, indicating strong dissatisfaction or a negative evaluation. Therefore, the overall sentiment is negative.

Specify a model

-- Call ai_query() and specify the 'my_qwen_service' model.
select
  ai_query ('Determine if the following comment is positive or negative:\n This is quite bad.', 'my_qwen_service')
;

Result:

<think> The user wants me to determine if this comment is positive or negative. The comment is "This is quite bad." First, I need to understand its meaning. The user is saying "this thing" is not good, so it should be classified as a negative evaluation. Next, I will consider the common characteristics of negative comments. They typically contain explicit negative words like "bad," "not good," or "terrible." The word "quite" is used here to emphasize the degree, but the primary message is one of dissatisfaction. Then, I need to consider the user's potential need. They are likely performing sentiment analysis and require a clear classification. I also need to be aware of any deeper intentions. The user might be seeking guidance on how to handle such comments or whether improvements are needed. Therefore, my response must clearly state the negative nature of the comment and explain the reasoning, ensuring the user understands it completely. Finally, I will make sure the answer is concise, directly states the comment is negative, and maintains a polite and professional tone. This ensures the user receives accurate information without ambiguity. </think> This comment is **negative**.

Specify a model and parameters

-- Call ai_query() with the 'my_qwen_service' model and a parameter to disable the chain-of-thought process.
select
  ai_query (
    'Determine if the following comment is positive or negative:\n This is quite bad.',
    'my_qwen_service',
    '{"chat_template_kwargs": {"enable_thinking": false}}'
  );

Result:

This comment is **negative**.

Process image data

SELECT
  path,
  ai_query(
    'Return the license plate number in the image. IMPORTANT: Reply only with the license plate information. If the image is too blurry to be identified, reply with ''unrecognizable''.',
    service_name => 'qwen3.5-plus',
    data => path,
    data_type => 'uri'
  ) AS plate_number
FROM (
  SELECT /*+ REPARTITION(10) */ path
  FROM ccpd_1m_1000
) t;

Task-specific functions

Sentiment analysis: `ai_sentiment()`

The ai_sentiment() function analyzes text to determine its sentiment, returning "positive", "negative", or "neutral". It is ideal for applications like user comment analysis and public opinion monitoring.

Syntax

ai_sentiment(text[, service_name] [, options])

Parameters

Parameter	Type	Required	Description
`text`	STRING	Yes	The text to analyze. For best results, use a complete sentence or paragraph. Short or ambiguous text reduces accuracy.
`service_name`	STRING	No	The name of the Model Service to use. • If omitted, the function uses the default `qwen3.6-plus` model. • For model service names registered with Model Service, see Model Service.
`options`	STRING	No	A JSON string that contains additional parameters to control the model's behavior. For details on supported parameters, see Options Parameters.

Return type: STRING.

Example

SELECT
  ai_sentiment ('I absolutely love this  phone! It’s amazing!');

The query returns:

positive

Grammar correction: `ai_fix_grammar()`

The ai_fix_grammar() function is a specialized AI Function that automatically detects and corrects spelling errors, grammatical issues, and awkward phrasing. It is ideal for cleaning user-generated content, optimizing customer service conversations, and preprocessing documents.

Syntax

ai_fix_grammar(text[, service_name] [, options])

Parameters

Parameter	Type	Required	Description
`text`	STRING	Yes	The original text to correct. For best results, use a complete sentence or paragraph.
`service_name`	STRING	No	Specifies the name of the Model Service to invoke: • If you omit this parameter, the function uses the default model `qwen3.6-plus`. • For a list of registered service names, see Model Service.
`options`	STRING	No	A JSON string containing additional invocation parameters to control generation behavior. For details on supported parameters, see Options parameters.

Return type: STRING.

Example

SELECT
  ai_fix_grammar ('He go to school yesterday and dont like math.');

Return value:

He went to school yesterday and didn't like math.

Text classification: `ai_classify()`

The ai_classify() function is a specialized AI Function for text classification. It automatically categorizes text based on a provided list of labels and is ideal for multi-class classification tasks such as news classification, ticket routing, and comment topic recognition.

Syntax

ai_classify(text, labels[, service_name] [, options])

Parameters

Parameter	Type	Required	Description
`text`	STRING	Yes	The text to classify. For best results, use a complete sentence or phrase.
`labels`	ARRAY	Yes	An array of candidate labels for classification, for example, `array('Technology', 'Sports', 'Entertainment')`. Labels should be distinct to avoid semantic overlap.
`service_name`	STRING	No	The Model Service to use: • If omitted, the system uses the default model, `qwen3.6-plus`. • For service names registered via Model Service, see Model Service.
`options`	STRING	No	Additional parameters in a JSON string to control the generation behavior. For supported parameters, see Options parameters.

Return type: string

Example

SELECT ai_classify(
  'Messi scored a spectacular free kick in the match',
  array('Technology', 'Sports', 'Entertainment', 'Finance')
) AS category;

Result:

Sports

Information extraction: `ai_extract()`

The ai_extract() function is a specialized AI Function for information extraction. It extracts predefined fields from unstructured text. This function is ideal for parsing user comments, extracting key information from contracts, and structuring logs.

Syntax

ai_extract(text, labels[, service_name] [, options])

Parameters

Parameter	Type	Required	Description
`text`	STRING	Yes	The source text to extract information from.
`labels`	ARRAY	Yes	A list of predefined fields to extract, such as `Array('country', 'age', 'name')`.
`service_name`	STRING	No	Specifies the name of the model service to invoke. • If you omit this parameter, the function uses the default model, `qwen3.6-plus`. • For the names of registered model services, see model service.
`options`	STRING	No	A JSON string that contains additional parameters to control generation behavior. For details on the supported parameters, see Options parameter description.

Return type: JSON.

Example

select
  ai_extract (
    'I am Alice,from china . I am 6 years old. Welcome!',
    Array('country', 'age', 'name')
  );

Result:

{"country": "china", "age": "6", "name": "Alice"}

Similarity calculation: `ai_similarity()`

ai_similarity() is an AI function that uses vector cosine similarity to calculate the semantic similarity between two texts. It is ideal for use cases like duplicate content identification, recommendation systems, and question-and-answer matching.

Function syntax

ai_similarity(text1, text2[, service_name] [, options])

Parameters

Parameter	Type	Required	Description
`text1`	STRING	Yes	The first text to compare.
`text2`	STRING	Yes	The second text to compare.
`service_name`	STRING	No	The name of the Model Service to use. • If omitted, the function uses the default model, `qwen3.6-plus`. • For a list of available service names, see Model Service.
`options`	STRING	No	A JSON string containing additional parameters to control the model's generation behavior. For details, see Options parameters.

Return type: DOUBLE.

Example

SELECT
  ai_similarity (
    'I enjoy hiking in the mountains.',
    'I love walking through mountain trails.'
  );

Result:

0.85

Translation: `ai_translate()`

The ai_translate() function is an AI Function that translates text into a target language. It is useful for cross-language data analysis and international content processing.

Syntax

ai_translate(text, target_lang[, service_name] [, options])

Parameters

Parameter	Type	Required	Description
`text`	STRING	Yes	The text to translate.
`target_lang`	STRING	Yes	Target language code. Supported values: • zh: Chinese • en: English • ja: Japanese • ko: Korean • ru: Russian • de: German • es: Spanish • fr: French • it: Italian
`service_name`	STRING	No	The name of the Model Service to invoke. • If omitted, the default model, `qwen3.6-plus`, is used. • For service names registered through Model Service, see Model Service.
`options`	STRING	No	A JSON string with additional parameters to control generation behavior. For details, see Options parameters.

Return type: STRING.

Example

select
  ai_translate ('你来自哪儿？', 'en')
;

Result:

Where are you from?

SQL optimization: `ai_sql_optimize()`

The ai_sql_optimize() AI Function analyzes an input SQL query and suggests optimizations, such as predicate pushdown, bucketing or partitioning, and join ordering. Use this function to accelerate queries, save resources, and improve execution plans.

Syntax

ai_sql_optimize(sql_text[, service_name] [, options])

Parameters

Parameter	Type	Required	Description
`sql_text`	STRING	Yes	The SQL query to optimize. For best results, include the complete table schema.
`service_name`	STRING	No	Specifies the Model Service to use: • Omit this parameter to use the default model, `qwen3.6-plus`. • For a list of registered service names, see Model Service.
`options`	STRING	No	Additional parameters in a JSON string to control generation. For details on supported parameters, see Options parameter details.

Return type: STRING.

Example

SELECT
  ai_sql_optimize (
    'SELECT * FROM logs WHERE date = "2025-01-01" AND user_id IN (SELECT id FROM users WHERE status = "active");'
  );

Result:

SELECT
  l.*
FROM
  logs l
  JOIN users u ON l.user_id = u.id
WHERE
  l.date = '2025-01-01'
  AND u.status = 'active'
;

HiveQL to Spark SQL: `ai_sql_hive_to_spark()`

The ai_sql_hive_to_spark() AI Function automatically converts HiveQL to Spark SQL. It recognizes Hive-specific syntax, such as LATERAL VIEW and SORT BY, and translates it into Spark SQL-compatible equivalents. This function is ideal for migrating Hive jobs to a Spark environment.

Syntax

ai_sql_hive_to_spark(hql_text[, service_name] [, options])

Parameters

Parameter	Type	Required	Description
`hql_text`	STRING	Yes	The input HiveQL statement. Supports complex syntax structures, such as complex queries and window functions.
`service_name`	STRING	No	Specify the name of the model service to call: • If omitted, the default model is used. The default is `qwen3.6-plus`. • For service names registered in Model Service, see Model Service.
`options`	STRING	No	Additional parameters, specified as a JSON string, that control generation behavior. For details on supported parameters, see Options parameters.

Return type: STRING.

Example

SELECT
  ai_sql_hive_to_spark (
    '
  SELECT 
    user_id,
    nvl2(email, "verified", "anonymous") AS user_type,
    current_date() AS load_date
  FROM users
  WHERE login_time >= date_sub(current_date(), 7)
'
  );

Result:

SELECT
  user_id,
  CASE
    WHEN email IS NOT NULL THEN "verified"
    ELSE "anonymous"
  END AS user_type,
  current_date() AS load_date
FROM
  users
WHERE
  login_time >= date_sub(current_date(), 7)

Text vectorization: `ai_embedding()`

The ai_embedding() is an AI function for text vectorization. It converts text into a high-dimensional semantic vector (embedding) for downstream AI tasks such as semantic retrieval, clustering, and similarity computation.

Syntax

ai_embedding(text[, dimension][, service_name] [, options])

Parameters

Parameter	Type	Required	Description
`text`	STRING	Yes	The input text to vectorize. Text that exceeds the model's length limit will be truncated.
`dimension`	INT	No	The dimension of the output vector. Supported values: 2,048, 1,536, 1,024 (default), 768, 512, 256, 128, and 64.
`service_name`	STRING	No	The name of the Model Service to use. • If omitted, this parameter defaults to `text-embedding-v4`. • For a list of model services registered in Model Service, see Model Service.
`options`	STRING	No	Additional parameters, provided as a JSON string, to control the generation behavior. For a list of supported parameters, see Options parameters.

Return type: ARRAY<DOUBLE>.

Example

select ai_embedding ('This is a sentence', 256);

Result:

[-0.08807944506406784,0.041450440883636475,0.027626311406493187,-0.03082999959588051,-0.027538539841771126,0.030325308442115784,-0.0742553174495697,0.12841078639030457,-0.20047178864479065,0.2492731511592865,-0.07697625458240509,-0.0023876791819930077,0.01588677428662777,0.039453621953725815,0.035416096448898315,-0.044588297605514526,-0.025783095508813858,-0.09935817122459412,0.062230516225099564,0.06122113764286041,-0.019803611561655998,0.06359098851680756,0.08360305428504944,0.10120139271020889,-0.08680674433708191,-0.04792364314198494,0.012836690060794353,-0.06468813866376877,-0.08948379755020142,-0.00902956910431385,-0.17264799773693085,0.009084426797926426,-0.026200013235211372,0.009852433577179909,-0.02606835402548313,0.08351528644561768,-0.0015181854832917452,0.06165999919176102,-0.03504306450486183,0.019748752936720848,-0.04077020660042763,-0.047440893948078156,-0.04590488225221634,-0.017225300893187523,-0.03984859585762024,0.09795381873846054,-0.016599925234913826,-0.05077623948454857,-0.04487355798482895,-0.029623130336403847,-0.09058094769716263,0.027209393680095673,0.020637447014451027,-0.05981678143143654,0.07285095751285553,-0.012353942729532719,-0.02896483801305294,-0.12604093551635742,-0.060343414545059204,0.008782709948718548,0.0731581598520279,0.014306874945759773,-0.012321027927100658,0.003225629683583975,0.04382029175758362,-0.054506558924913406,-0.04298645257949829,0.011119645088911057,-0.013692469336092472,-0.06780405342578888,0.027604369446635246,0.11024193465709686,-0.06323989480733871,-0.04594876617193222,-0.037325143814086914,0.05415547266602516,-0.013747327029705048,0.07969719171524048,-0.11858029663562775,0.004797301255166531,-0.012759889476001263,0.006895606406033039,-0.07465028762817383,-0.04066048935055733,0.1428932100534439,0.037390973418951035,-0.012748917564749718,-0.07846838235855103,0.05415547266602516,-0.0560864619910717,-0.045422133058309555,0.04989851638674736,0.07842449843883514,0.0030226565431803465,0.031356632709503174,-0.04254759103059769,-0.13051731884479523,-0.01195896789431572,0.037654291838407516,-0.02290855348110199,-0.013494981452822685,-0.15605904161930084,-0.018443141132593155,-0.0013947556726634502,0.05305831879377365,0.06736519187688828,-0.09172198921442032,-0.05419935658574104,0.004629985429346561,0.09751495718955994,-0.010527183301746845,-0.08500741422176361,-0.06253772228956223,0.029228154569864273,9.778375970199704E-4,-0.057359158992767334,-0.0935652032494545,2.768597041722387E-4,-0.01802622340619564,0.028262659907341003,0.07342147827148438,-0.023676561191678047,-0.09769050031900406,-0.07697625458240509,0.05665697902441025,-0.15061716735363007,0.012310056015849113,-0.004180152900516987,0.087508924305439,-0.02492731623351574,-0.04120906442403793,-0.09865599870681763,0.055472053587436676,-0.03798343613743782,0.013308465480804443,-0.035591643303632736,0.028196832165122032,0.0260244682431221,-1.6988728020805866E-4,-0.02999616228044033,-0.023435188457369804,0.02786768600344658,0.06337155401706696,0.009775632992386818,0.04219650477170944,-0.057315271347761154,0.01100444421172142,-0.07895112782716751,-0.020341215655207634,0.013703440316021442,-0.0821109265089035,0.008376763202250004,-0.01687421277165413,-0.0050935326144099236,-0.025475893169641495,0.013286522589623928,0.05169785022735596,0.04568545147776604,0.08329585194587708,0.03730320185422897,0.06069450452923775,0.020670361816883087,0.04309616982936859,0.026353614404797554,-0.061352793127298355,-0.07236821204423904,-0.012847661040723324,-0.006835263222455978,-0.034428659826517105,0.009084426797926426,-0.011827308684587479,0.034560319036245346,-0.03793954849243164,0.05929014831781387,0.026419444009661674,0.03372648358345032,-0.021229909732937813,-0.015711231157183647,0.017587361857295036,-0.03291459009051323,0.05371661111712456,-0.02712162211537361,0.25787484645843506,-0.04318394139409065,-0.002493280218914151,-0.00854133628308773,-0.07478194683790207,0.024773715063929558,-0.009320314973592758,0.032475728541612625,0.05367272347211838,-0.031488291919231415,0.05275111645460129,0.11199737340211868,-0.05582314357161522,0.00895276851952076,-0.016259808093309402,0.16071097552776337,-0.04412749409675598,-0.04083603248000145,-0.04465412721037865,-0.09856822341680527,-0.08979099988937378,0.02593669667840004,0.11910692602396011,-0.027253279462456703,0.05915848910808563,-0.11910692602396011,0.03693016991019249,-0.07144660502672195,-0.04480772837996483,-0.04586099460721016,-0.04274507984519005,-0.019430579617619514,-0.00844807829707861,0.027604369446635246,-0.022579409182071686,0.019397664815187454,0.035503871738910675,0.07899501174688339,-0.06657524406909943,0.02505897358059883,-0.023786276578903198,0.0518295057117939,0.03671073913574219,0.05678863823413849,0.1004992127418518,0.008546821773052216,0.03872950002551079,0.0085248788818717,0.08641176670789719,-0.006901092361658812,-0.13771463930606842,-0.06767239421606064,-0.07789786159992218,0.06122113764286041,-0.02087882161140442,-0.031246917322278023,0.022623294964432716,-0.006160513963550329,0.09268748760223389,0.08465632051229477,-0.1036590114235878,-0.09900708496570587,0.008299962617456913,0.06367875635623932,-0.004986560437828302,-0.0022011632099747658,0.02088979259133339,-0.016139119863510132,-0.07934610545635223,0.029754789546132088,-0.04880136623978615,0.05595480278134346,-0.01239782851189375,0.02087882161140442]

Image embedding: `ai_embedding_multimodal()`

Note

This AI Function is available only in the Indonesia (Jakarta) region.

The ai_embedding_multimodal() function specializes in multi-modal vectorization. It converts an image into a high-dimensional semantic embedding for downstream AI tasks such as cross-modal retrieval, image search by image, and image clustering.

Syntax

ai_embedding_multimodal(data[, service_name][, options[, data_type]])

Parameters

Parameter	Type	Required	Description
`data`	BINARY/STRING	Yes	The image data column to convert into an embedding. The value you pass depends on the `data_type` setting: When `data_type` is `binary` (default), pass the binary data of an image of the BINARY type, or the http(s) URL of the image. For example, you can pass the data from the `content` column that is read by the `read_files` function, or directly pass a publicly accessible URL of the image. If `data_type` is `uri`: Pass the file path (STRING type). The path can be an OSS path (for example, `oss://bucket/path/to/image.jpg`) or a local file path (for example, `file:///mnt/data/path/to/image.jpg`, which requires using the Spark-managed mount feature).
`service_name`	STRING	No	The name of the model service to invoke. If you omit this parameter, the function uses the default model, `tongyi-embedding-vision-plus`, which produces 1152-dimensional vectors. For available model service names, see model service.
`options`	STRING	No	A JSON string that specifies additional invocation parameters to control model behavior. For a list of supported parameters, see Options parameters.
`data_type`	STRING	No	Specifies how the image data is passed. The supported values are: `binary` (default): The `data` parameter accepts the image's binary data or URL. `uri`: The `data` parameter accepts a file path (STRING type). The function then reads the file's content.

Return type: ARRAY<DOUBLE>.

Example

ai_embedding_multimodal(
  image_col,
  service_name => 'multimodal-embedding-model',
  data_type => 'binary'
)

Options parameters

This topic describes the parameters supported by the built-in model service in Serverless Spark. For external model services, refer to their official documentation for parameter support.

Parameter	Description	qwen3.6-plus	text-embedding-v4
`temperature`	Controls the diversity of the generated text. A higher value produces more diverse output, while a lower value produces more deterministic text. Valid range: [0, 2.0).	✓	✗
`top_p`	Sets the probability threshold for nucleus sampling to control text diversity. A higher value results in more diverse output. Valid range: (0, 1.0].	✓	✗
`top_k`	Specifies the number of highest-probability tokens to consider for sampling. A larger value increases randomness, while a smaller value increases determinism. The value must be greater than or equal to 0.	✓	✗
`presence_penalty`	Controls the repetition of content in the generated text. A positive value reduces repetition, while a negative value increases it. Valid range: [-2.0, 2.0].	✓	✗
`seed`	A random number seed that ensures reproducible results given the same input and parameters. The value must be an integer in the range [0, 2³¹-1].	✓	✗
`stop`	Specifies one or more stop words. Generation stops immediately when the model outputs a string or `token_id` listed in this parameter.	✓	✗
`response_format`	Specifies the format of the response. Valid values: • `{"type": "text"}`: Outputs a text response. • `{"type": "json_object"}`: Outputs a standard JSON string. • `{"type": "json_schema","json_schema": {...} }`: Outputs a JSON string that conforms to the provided JSON schema.	✓	✗
`encoding_format`	Specify the vector encoding format, such as `float` or `base64`.	✗	✓
`max_input_tokens`	Specifies the maximum number of tokens allowed in the input. The system automatically truncates inputs that exceed this limit.	✓	✗
`max_tokens`	Specifies the maximum number of tokens to generate. Generation stops if the output reaches this limit, which is useful for controlling output length.	✓	✗
`enable_thinking`	Specifies whether to enable thinking mode when using a model that supports a chain-of-thought process.	✓	✗
`thinking_budget`	Specifies the maximum length of the thinking process in tokens.	✓	✗
`logprobs`	Specifies whether to return the log probabilities of output tokens. Valid values: • `true`: Returns log probabilities. • `false`: Does not return log probabilities.	✓	✗
`top_logprobs`	Specifies the number of candidate tokens with the highest probability to return at each generation step. The value range is [0, 5]. This parameter takes effect only when `logprobs` is `true`.	✓	✗
`enable_search`	Specifies whether the model uses web search results to inform its generation. Valid values: • `true`: Enables web search. • `false`: Disables web search.	✓	✗
`search_options`	The internet search policy. This parameter is effective only when `enable_search` is `true`.	✓	✗
`chat_template_kwargs`	Specifies extra arguments for the chat template to control its rendering behavior, such as enabling specific capability tags.	✓	✗
`parameters`	A generic parameter wrapper object from the native DashScope API. When using the native API, pass the parameters described above in this field.	✓	✓

FAQ

Q: How do I disable the thinking process (such as chain of thought or reasoning steps) in the ai_query() function?
A: You can control the model's output behavior by passing parameters to the ai_query() function. To disable the thinking process and return only the final result, pass the following configuration when you call the function: {"chat_template_kwargs": {"enable_thinking": false}}.

Note

The enable_thinking parameter applies only to Qwen-series models. For other models, refer to their official documentation to find the specific parameter for disabling the chain of thought.
You can adjust the randomness and diversity of the model's output with parameters such as temperature and top_k. For a complete list of supported parameters, see the official documentation for your model.

Prerequisites

Function overview

Function: ai_query()

Task-specific functions

Sentiment analysis: ai_sentiment()

Grammar correction: ai_fix_grammar()

Text classification: ai_classify()

Information extraction: ai_extract()

Similarity calculation: ai_similarity()

Translation: ai_translate()

SQL optimization: ai_sql_optimize()

HiveQL to Spark SQL: ai_sql_hive_to_spark()

Text vectorization: ai_embedding()

Image embedding: ai_embedding_multimodal()

Options parameters

FAQ

Function: `ai_query()`

Sentiment analysis: `ai_sentiment()`

Grammar correction: `ai_fix_grammar()`

Text classification: `ai_classify()`

Information extraction: `ai_extract()`

Similarity calculation: `ai_similarity()`

Translation: `ai_translate()`

SQL optimization: `ai_sql_optimize()`

HiveQL to Spark SQL: `ai_sql_hive_to_spark()`

Text vectorization: `ai_embedding()`

Image embedding: `ai_embedding_multimodal()`