AI functions let you call AI services directly from your database. Examples include text generation, conditional judgment, sentiment analysis, and classification. This topic describes how to use AI functions in AnalyticDB for MySQL and provides usage examples.
Important
Because AI models are inherently stochastic, identical inputs may produce different outputs when processed by an AI function.
-
Text functions
-
ai_filter: evaluates the factuality or compliance of input text.
-
ai_translate: translates input text into a specified language.
-
ai_classify: performs semantic analysis on input text and classifies it into predefined categories or labels.
-
ai_extract: extracts structured information of specified types (such as names, dates, addresses, or keywords) from unstructured text.
-
ai_sentiment: analyzes the sentiment of input text and returns a judgment such as positive, negative, or neutral.
-
ai_similarity: calculates the semantic similarity between two pieces of text.
-
ai_mask: masks sensitive information (such as ID numbers, phone numbers, or email addresses) in input text.
-
ai_summarize: compresses long text semantically to generate a concise and accurate summary.
-
-
Audio functions
-
ai_audio_transcribe: transcribes audio into corresponding text. You can specify segmentation mode, target language, and other options as needed.
-
ai_audio_embed: converts audio into its corresponding embedding vector and returns an array of embedding vectors.
-
Prerequisites
-
AnalyticDB for MySQL Enterprise Edition, Basic Edition, Data Lakehouse Edition, or Data Warehouse Edition in elastic mode, with ENI access enabled.
Important-
Log on to the AnalyticDB for MySQL console. In the navigation pane on the left, choose . In the Network Information section, turn on the switch for ENI access.
-
Enabling or disabling ENI access interrupts database connections for approximately 2 minutes. During this period, you cannot read or write data. We recommend that you carefully evaluate the potential impact before you perform this operation.
-
-
This feature is in public preview. To use it, submit a ticket to contact technical support for activation.
Preparations
To use AI functions, configure a network connection from AnalyticDB for MySQL to Alibaba Cloud Model Studio and create a model.
Connect over the public network
-
Configure an Internet NAT gateway.
Create an Internet NAT gateway in the same region as your AnalyticDB for MySQL cluster. Associate it with an elastic IP address (EIP), then create a SNAT entry (we recommend creating a SNAT entry at the vSwitch granularity and specifying any vSwitch). For more information, see Internet NAT gateway.
-
Create a model. For syntax details, see CREATE MODEL.
Example:
CREATE MODEL qwen_plus_external OPTIONS ( type='external', provider='bailian', name='qwen-plus', interface='TEXT_TO_TEXT', api_key='sk-xxx' -- Replace with the API key for the model service enabled in Alibaba Cloud Model Studio. )
Connect over PrivateLink
-
Configure an endpoint and obtain a private domain name. For more information, see Access Alibaba Cloud Model Studio models or application APIs over a private network.
ImportantAlibaba Cloud Model Studio is available in the following regions: Singapore, China (Beijing).
The endpoint must be in the same region as the Model Studio service. To use a VPC in another region, see Cross-region private access.
-
Create a model. For syntax details, see CREATE MODEL.
Example:
CREATE MODEL qwen_plus_external OPTIONS ( type='external', provider='bailian', name='qwen-plus', interface='TEXT_TO_TEXT', api_key='sk-xxx', -- Replace with the API key for the model service enabled in Alibaba Cloud Model Studio. endpoint='ep-xxx.com' -- Replace with the private endpoint domain name. )
Text functions
ai_filter
ai_filter(text)
ai_filter(model_name, text)-
Description: Evaluates the factuality or compliance of input text. Returns true if the content is credible, reasonable, or meets predefined rules. Otherwise, returns false.
-
Input types:
-
model_name: Model name, VARCHAR.
-
text: Input text, VARCHAR.
-
-
Return type: BOOLEAN.
Example 1
SELECT ai_filter("Is Zhejiang in China?")Result
+---------------------------------+
| ai_filter("Is Zhejiang in China?") |
+---------------------------------+
| 1 | Example 2
SELECT ai_filter("qwen_plus_external", "Is Zhejiang in China?")Result
+----------------------------------------------+
| ai_filter("qwen_plus_external", "Is Zhejiang in China?") |
+----------------------------------------------+
| 1 | ai_translate
ai_translate(text, targetLang)
ai_translate(model_name, text, targetLang)-
Description: Translates input text into a specified language.
-
Input types:
-
model_name: Model name, VARCHAR.
-
text: Input text, VARCHAR.
-
targetLang: Target language for translation, VARCHAR.
ImportantThe target language must be supported by the model.
-
-
Return type: VARCHAR.
Example
SELECT ai_translate("AnalyticDB for MySQL is a data analytics platform based on a lakehouse architecture.", "cn") as translate_textResult
+-------------------------------------------------------------------------------------------+
| translate_text |
+-------------------------------------------------------------------------------------------+
| AnalyticDB for MySQL is a data analytics platform based on a data lakehouse architecture. | ai_classify
ai_classify(text, labels)
ai_classify(model_name, text, labels)-
Description: Performs semantic analysis on input text and classifies it into predefined categories or labels.
-
Input types:
-
model_name: Model name, VARCHAR.
-
text: Input text, VARCHAR.
-
labels: Predefined list of category labels. Supports VARCHAR or Array<VARCHAR>.
-
-
Return type: VARCHAR.
Example 1
SELECT ai_classify('Last night I tried a new pasta recipe in the kitchen. The sauce was rich, and my family loved it!', "['travel', 'cooking', 'reading', 'driving']")
AS predicted_label;Result
+----------------------------------------------+
| predicted_label |
+----------------------------------------------+
| cooking | Example 2
SELECT ai_classify('Last night I tried a new pasta recipe in the kitchen. The sauce was rich, and my family loved it!', ARRAY['travel', 'cooking', 'reading', 'driving'])
AS predicted_label;Result
+----------------------------------------------+
| predicted_label |
+----------------------------------------------+
| cooking | ai_extract
ai_extract(text, labels)
ai_extract(model_name, text, labels)-
Description: Extracts structured information of specified types (such as names, dates, addresses, or keywords) from unstructured text.
-
Input types:
-
model_name: Model name, VARCHAR.
-
text: Input text, VARCHAR.
-
labels: Predefined list of category labels. Supports VARCHAR or Array<VARCHAR>.
-
-
Return type: VARCHAR.
Example
select ai_extract('Yesterday (June 15, 2024), I bought an iPhone 15 Pro on Taobao. Its titanium body and A17 chip are truly impressive!', "['product_name', 'date', 'key_feature']") as resultResult
+----------------------------------------------------------------------------------------------------+
| result |
+----------------------------------------------------------------------------------------------------+
| product_name=iPhone 15 Pro, date=June 15, 2024, key_feature=Titanium body and A17 chip | ai_generate
ai_generate(text)
ai_generate(model_name, text)-
Extracts structured information, such as names, dates, addresses, and keywords, from unstructured text.
-
Input types:
-
model_name: Model name, VARCHAR.
-
text: Input text, VARCHAR.
-
-
Return type: VARCHAR.
Example
select ai_generate('Briefly introduce the TPC-H test set') as resultResult
+----------------------------------------------------------------------------------------------------+
| result |
+----------------------------------------------------------------------------------------------------+
| TPC-H is a standard benchmark test set used to evaluate the decision support capabilities of a database. | ai_sentiment
ai_sentiment(text)
ai_sentiment(model_name, text)-
Description: Analyzes the sentiment of input text and returns one of four fixed labels:
positive,negative,neutral, ormixed. -
Input types:
-
model_name: Model name, VARCHAR.
-
text: Input text, VARCHAR.
-
-
Return type: VARCHAR.
Example
SELECT ai_sentiment('These headphones have excellent sound quality and are comfortable to wear. Highly recommended!') AS sentiment;Result
+----------------------------------------------+
| sentiment |
+----------------------------------------------+
| positive | ai_similarity
ai_similarity(text1, text2)
ai_similarity(model_name, text1, text2)-
Description: Calculates the semantic similarity between two pieces of text. Returns a similarity score from 0 to 10. A higher score indicates greater similarity.
-
Input types:
-
model_name: Model name, VARCHAR.
-
text1 and text2: Input text, VARCHAR.
-
-
Return type: VARCHAR.
Example
SELECT ai_similarity(
'How do I reset my account password?',
'I forgot my logon password. How can I recover it?'
) AS result;Result
+----------------------------------------------+
| result |
+----------------------------------------------+
| 9.3 | ai_mask
ai_mask(text, labels)
ai_mask(model_name, text, labels)-
Description: Masks sensitive information (such as ID numbers, phone numbers, or email addresses) in input text.
-
Input types:
-
model_name: Model name, VARCHAR.
-
text: Input text, VARCHAR.
-
labels: Predefined list of category labels. Supports VARCHAR or Array<VARCHAR>.
-
-
Return type: VARCHAR.
Example
SELECT ai_mask(
'Contact me: 13812345678, email is user@example.com, ID number 110101199003072316',
"['phone', 'email', 'id_card']"
) AS result;Result
+----------------------------------------------+
| result |
+----------------------------------------------+
| Contact me: [MSKED], email is [MSKED], ID number [MSKED] | ai_summarize
ai_summarize(text)
ai_summarize(model_name, text)-
Description: Compresses long text semantically to generate a concise and accurate summary.
-
Input types:
-
model_name: Model name, VARCHAR.
-
text: Input text, VARCHAR.
-
-
Return type: VARCHAR.
Example
SELECT ai_summarize('Above the winding lotus pond, the view is filled with layers of leaves. The leaves rise high out of the water, like the skirts of graceful dancers. Among the layers of leaves, white flowers are scattered. Some are blooming gracefully, others are shyly budding. They are like pearls, like stars in a clear sky, or like a beauty fresh from her bath. A gentle breeze carries a faint fragrance, like a distant song from a high tower. At this moment, the leaves and flowers tremble slightly, like a flash of lightning, instantly crossing to the other side of the pond. The leaves were originally packed tightly side by side, creating a ripple of deep green. Beneath the leaves, the gentle water flows, hidden from view. But this makes the leaves even more charming. The moonlight, like flowing water, quietly pours onto the leaves and flowers. A thin, blue mist rises from the pond. The leaves and flowers look as if they have been washed in milk, or like a dream veiled in light gauze. Although it is a full moon, a thin layer of clouds in the sky prevents it from shining brightly. But I think this is just right—deep sleep is essential, but a light nap has its own charm. The moonlight shines through the trees. The dense shrubs on high ground cast mottled, dark shadows, stark and ghost-like. The sparse, graceful shadows of the weeping willows, however, look as if they are painted on the lotus leaves. The moonlight in the pond is not uniform. But the light and shadow have a harmonious melody, like a famous piece played on a violin.') AS result;Result
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| result -
| The lotus pond under the moonlight is serene and beautiful: layers of lotus leaves are like dancers' skirts, dotted with white flowers resembling pearls, stars, or a beauty fresh from her bath. A gentle breeze carries fragrance, leaves and flowers tremble, creating green ripples. Water flows hidden beneath the leaves, adding to their charm. Moonlight is like water, a light blue mist veils the scene, and the leaves and flowers seem washed, hazy like a dream. Clouds cover the full moon, creating mottled light and shadow. The stark black shadows of shrubs and the graceful silhouettes of willows are like a painting. The light and shadow blend in a harmony like a famous melody. | Audio functions
ai_audio_transcribe
Only built-in models are supported.
ai_audio_transcribe(url)
ai_audio_transcribe(model_name, url)
ai_audio_transcribe(model_name, url, options)-
Description: Transcribes audio into corresponding text. You can specify segmentation mode, target language, and other options as needed. Returns the transcribed text or a URL to the output file.
-
Input types:
-
model_name: Model name, VARCHAR.
-
url: URL of the input audio file, VARCHAR.
-
options: Optional parameters in JSON format. Supported parameters are listed below.
Parameter name
Description
Options
Default
language
Target language for transcription
-
cn (Chinese)
-
en (English)
-
ja (Japanese)
-
yue (Cantonese)
-
fspk (Chinese-English mixed speech)
cn
diarization_mode
Paragraph segmentation mode
-
word (by word)
-
sentence (by sentence)
-
speaker (by speaker)
word (chunked by word dimension)
output_type
Output format
-
json (returns transcribed JSON text directly)
-
url (saves to OSS and returns the URL of the text file)
json
-
-
-
Return type: VARCHAR.
Example 1
SELECT ai_audio_transcribe("https://dashscope.oss-cn-beijing.aliyuncs.com/samples/audio/paraformer/hello_world_female2.wav") as resultResult
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| result -
| {"TaskId": "xxxx", "Transcription": {"AudioInfo": {"Size": 128480, "Duration": 3834, "SampleRate": 16000, "Language": "cn"}, "Paragraphs": [{"ParagraphId": "1768447804076500000", "SpeakerId": "1", "Words": [{"Id": 10, "SentenceId": 1, "Start": 100, "End": 595, "Text": "Hello, "}, {"Id": 20, "SentenceId": 1, "Start": 596, "End": 841, "Text": "world, "}, {"Id": 30, "SentenceId": 1, "Start": 844, "End": 1588, "Text": "this is "}, {"Id": 40, "SentenceId": 1, "Start": 1588, "End": 2580, "Text": "Alibaba "}, {"Id": 50, "SentenceId": 1, "Start": 2580, "End": 3076, "Text": "Speech "}, {"Id": 60, "SentenceId": 1, "Start": 3076, "End": 3820, "Text": "Lab."}]}], "AudioSegments": [[100, 3820]]}} | Example 2
SELECT ai_audio_transcribe("tingwu", "https://dashscope.oss-cn-beijing.aliyuncs.com/samples/audio/paraformer/hello_world_female2.wav", "{'diarization_mode':'sentence'}") as resultResult
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| result -
| {"TaskId": "xxxx", "Transcription": {"AudioInfo": {"Size": 128480, "Duration": 3834, "SampleRate": 16000, "Language": "cn"}, "Speakers": [{"SpeakerId": "1", "Sentences": [{"SentenceId": 1, "Start": 100, "End": 3820, "Text": "Hello, world, this is Alibaba Speech Lab."}]}]}}| ai_audio_embed
Audio vector embedding supports only built-in models.
ai_audio_embed(text)
ai_audio_embed(model_name, text)
ai_audio_embed(model_name, text, options)-
Description: Converts audio into its corresponding embedding vector and returns an array of embedding vectors. You can use parameters to control whether to segment the audio by start and end time before generating embeddings.
-
Input types:
-
model_name: Model name, VARCHAR.
-
text: Input text, VARCHAR.
-
options: Optional parameters in JSON format. Supported parameters are listed below.
Parameter Name
Description
Notes
access_id
OSS AccessKey ID
If you use role_arn, you no longer need to specify access_id, access_secret, or token.
access_secret
OSS AccessKey secret
token
OSS security token
role_arn
Role assumption credential
start_time
Start time of the audio segment
If not specified, starts from the beginning.
end_time
End time of the audio segment
If not specified, ends at the end.
-
-
Return type: Array<FLOAT>.
Example
SELECT ai_audio_embed("https://dashscope.oss-cn-beijing.aliyuncs.com/samples/audio/paraformer/hello_world_female2.wav")Result
[0.4234241,0.3575562,0.022631245,-0.09829298,-0.40605977,-0.72465694,-1.8039973,2.4773192,0.91702485,0.15146817,0.7209677,0.78260005,0.8019302,2.7577147,-0.86063147,-0.8249354,0.28732437,0.9009849,0.8899265,-1.5320694,-2.0866642,0.5051579,1.3553369,2.0348034,-0.74032015,-0.66065377,0.2756823,0.15714298,-2.8250875,0.8162984,0.20878711,0.8285897,1.4863579,1.0645012,-1.2933203,0.6919463,0.87850314,1.2212433,-0.44420764,-1.0146322,-1.8209833,0.16996253,-1.534067,-0.10253409,0.07611574,2.4005992,-1.2903963,0.6613166,-0.2554945,0.97570425,-0.108424075,0.82061934,-0.18102498,-0.27137256,-1.2723781,-0.9188018,-0.027503261,-0.15346082,-0.9564006,-0.4618776,-1.077228,-0.27756396,0.28749034,0.2570488,-0.39753774,0.5270934,0.6970718,-1.2984067,2.4312186,1.485573,1.6031872,-0.6158852,-1.1512791,0.49305362,-1.1939837,0.72933173,0.2659372,2.5559616,-0.41886744,0.6304756,-0.6171451,-0.5282561,0.78426796,-0.57570314,-0.91166306,-0.531768,0.96598166,0.17423655,-0.9133532,-0.09258405,-0.26951456,-0.34667063,-0.21612991,-0.054430377,0.052798696,0.5198285,1.9431844,-0.5638291,-2.1043544,-0.47198293,0.21959576,0.5975169,0.3951007,0.28112686,1.792782,-0.15896009,-0.71152246,1.1540072,0.37563428,0.0016163457,0.6544563,0.043974634,-1.6375796,-0.3698572,1.5310435,-0.20282944,0.73641545,2.1867669,1.4115202,-0.8114418,-0.36265984,0.2191038,-0.8933491,0.6616433,1.545423,0.10970531,-0.109743424,0.44605017,-0.087265715,0.29077715,-2.4435875,-1.1484046,0.078253455,0.02283861,-0.29482847,2.0237377,-0.6009212,-1.2377249,0.05194488,-0.16573223,0.8370868,-0.43602425,-0.4706988,0.17996503,0.88799554,0.6562157,-0.7520023,0.64789253,0.36865348,-1.4820247,0.5299587,1.1884397,1.4351603,0.21073115,-0.1712549,2.9410155,1.7485802,-0.6828801,-0.4833641,-0.4477328,2.3307724,-0.35595444,-0.61682695,-0.5370858,-0.8068234,1.2195143,-1.0834758,0.45275012,1.6243625,-1.3629726,-0.2959109,0.05071621,-0.7280639,-0.1713935,-0.43650395,-0.2131698,0.25380868,-0.16288652,2.0921175,0.3555297,-0.10752705,0.5716714,-0.09943808,-1.9066161,0.12683412,0.8566219,-0.20323975,1.4260024,-0.28789783,0.32412016,0.88897985,0.86555415,0.5940666,0.6645551,0.046334613,-1.5640875,-0.3510718,0.55363727,-0.13503519,0.67008746,-0.9184686,0.025278697,1.0128921,-0.61241907,-0.01107134,-0.8309523,-0.51025873,1.1141272,0.28813183,-0.10634196,-0.9954202,-0.07171043,-0.5856978,-1.2660285,2.1327019,-0.60268104,-0.6227884,-0.42101067,0.059599742,-0.09640202,-1.258711,0.5054924,-1.1847625,-0.044398762,-0.98595804,0.8883682,0.6221085,-2.5484293,0.7249505,0.69930685,0.7739025,-0.8139478,-1.1988907,-1.0416493,2.0153732,-1.7091763,-0.5611238,-1.2147603,0.9113469,1.5113174,0.23810485,-1.702736,-0.3295935,-0.41867778,-1.0378691,-0.45600057,-0.43525052,-0.1078409,-1.2993969,0.12842774,0.026097976,-0.7705405,-1.0907317,0.28274077,1.236289,1.6190177,2.1874366,0.16072829,-0.33150536,2.218483,-1.1703843,0.10300327,1.1994884,0.48275462,-0.40795812,0.5020531,1.1787555,-0.082187966,0.6315653,0.36654752,-0.24940589,-0.8652801,1.6283739,0.41405886,0.6377814,0.08396838,-1.0169003,1.2100558,1.4457762,-0.07999261,-0.012102162,0.85055244,-0.09711141,-1.0452846,0.13768612,-2.0506873,-1.6474499,0.043265514,-1.0009454,-0.0111249415,-1.2523409,-0.080719866,-0.6187693,-1.398226,0.6425289,-0.4808641,-0.06030046,-0.10275636,-0.31625932,-1.5993032,-0.20966552,-1.4618409,0.34925935,-0.5034448,0.100028045,0.25327235,-1.078896,-0.23394233,1.2247928,-2.6050038,-0.71609926,-0.77765155,-1.2089496,0.8526703,-0.1358416,1.1074059,1.1545771,-0.94525933,0.41012967,0.9361201,-0.14788401,-0.29333082,1.5782444,1.1100405,-0.4074414,-0.3862537,-0.5779069,-0.88644946,0.2233385,1.3612705,1.2413827,-1.3625424,-1.3623037,0.3056319,-1.4446377,0.64613384,0.15064861,-0.61473364,-1.3611295,-0.1975697,-1.0701923,0.7591377,-1.2106745,-2.067824,0.45041704,-0.71582735,-1.743847,1.169414,2.0158787,0.4734838,-0.3133036,-1.9916989,1.1441987,0.9155275,-1.3003027,0.82898057,-0.7439868,1.2072865,0.46877453,0.6648313,0.80477613,1.6927507,0.5842916,0.36608973,2.259473,-1.1628797,-0.3311869,0.36989415,-0.25035658,-0.28012496,1.092324,-0.40238732,0.0046352614,-2.0625768,1.161326,0.92277956,-0.20316431,-0.15164377,1.1715052,-0.7067665,-0.8608931,0.079684004,-0.89916384,0.02488108,-0.57668805,-0.879138,-1.7274998,0.824049,1.5638052,0.28415555,-0.17347385,-2.085917,0.4987632,-0.031601395,-1.0190377,-0.7815816,0.69643223,-0.33574122,2.1242745,0.19785832,0.55690974,-1.3932099,-0.44952002,-0.7719798,1.1326215,0.43839702,0.996999,-0.55692834,1.4014084,-2.3939395,-1.0112343,-1.4143248,1.1514114,1.1233637,-0.3385779,-0.23665123,-0.100857966,0.4633971,1.6215613,-0.0692888,-0.031505972,1.0472811,0.57112235,0.6015763,0.07582237,0.52702487,-1.3809607,-1.7482765,0.38386008,0.99316126,0.95603,0.40644804,-1.5072054,-0.34419048,0.63205683,-1.0854999,-0.92245156,-0.2712947,-0.75696105,0.996232,-0.10738732,-0.4674776,-1.2149413,-1.5094053,1.6796608,0.21961057,-0.35295358,-1.2609407,-0.040009048,-0.38785484,0.7788784,-0.65823495,2.0559616,-1.0074826,1.2282485,1.2540467,0.4914942,-0.47057188,-0.47061247,-0.16255763,-0.6718562,-0.53630847,0.4804698,-0.3134068,0.6407026,-0.727981,0.0481851,0.06927338,0.8321921,-0.6639807,0.74932885,0.23291564,0.76362675,-1.2966217,0.8806557,-1.2141875,0.6996881,-0.8293652,0.9085288,1.8878758,0.11363957,0.148718,0.5030497,-1.0422761,0.08673843,-0.80342984,-0.8046266,-0.18026677,0.28900644,0.76534355,-1.3163859,-0.72775376,0.36529016,-0.3660175,0.31056792,0.052575216,-1.11831,0.7895246,1.1172394,-0.31374845,0.17143561,-0.42633826,0.16579832,-0.012790448,-1.4290546,-0.47322562,0.7557427,1.0487514,-0.14331971,0.30455515,0.6938429,0.5565799]Configure default models for AI functions
If you do not specify a model name when calling an AI function, the system uses the default model. You can change the default model using the following configuration items. For instructions on modifying configurations, see Config and Hint parameters.
|
Model type |
Default model |
Configuration item |
|
TEXT_TO_TEXT (text model) |
qwen-plus |
AI_FUNCTION_TEXT_TO_TEXT_MODEL |
|
TEXT_TO_EMBEDDING (text embedding model) |
text-embedding-v2 |
AI_FUNCTION_TEXT_TO_EMBEDDING_MODEL |
|
AUDIO_TO_TEXT (audio-to-text model) |
tingwu |
AI_FUNCTION_AUDIO_TO_TEXT_MODEL |
|
AUDIO_TO_EMBEDDING (audio embedding model) |
qwen2.5-vl-embedding |
AI_FUNCTION_AUDIO_TO_EMBEDDING_MODEL |
Typical scenario examples
Scenario 1: Text retrieval
Applicable scenarios: In RAG, customer service systems, and similar use cases, retrieve the most relevant text segments from a corpus based on the semantics of a user query.
The query text and corpus texts are converted into vectors using the ai_embedd encoding. Then, the ten texts with the highest cosine similarity are identified by sorting.
SELECT cosine_similarity(ai_embedd("query text"), ai_embedd(text)) AS cos
FROM text_table
ORDER BY cos
LIMIT 10For better vector search performance, see Vector search.
Scenario 2: Voiceprint retrieval
Applicable scenarios: In meetings or daily conversations, customers want to transcribe recordings into text and distinguish dialogue by speaker.
To further analyze speaker information, segment the audio by speaker and compare embeddings against a voice dataset. For the complete solution, see Voiceprint retrieval.
-
Transcription and segmentation: Use the
ai_audio_transcribefunction to transcribe audio into text and segment it by speaker.SELECT AI_AUDIO_TRANSCRIBE("oss://xxx", "{'diarization_mode':'speaker'}") -
Voiceprint retrieval: Use the
ai_audio_embedfunction to compute embedding vectors for both the input audio and the original audio library. Calculate thel2_distanceand return the 10 closest matches.SELECT l2_distance(input_embedding, origin_embedding) AS ld FROM( SELECT ai_audio_embed("oss://xxx") AS input_embedding JOIN SELECT ai_audio_embed(url) AS origin_embedding FROM audio_table) ORDER BY ld LIMIT 10
Scenario 3: Extract dissatisfied customer reviews
Applicable scenarios: Filter out negative feedback from user reviews.
The customer_comment table stores daily customer reviews. Use the sentiment analysis function ai_sentiment to identify customer sentiment and filter all dissatisfied reviews from January 1, 2026.
-- The qwen_plus_external model has been created, corresponding to the qwen-plus model provided by Model Studio.
SELECT name, comment
FROM customer_comment
WHERE ai_sentiment("qwen_plus_external", comment)='negative'
AND dt='20260101'