Retrieves semantically similar documents. The operation is designed for multi-turn conversations and can process message input in historical conversations. The operation returns results that are highly related to the current conversation based on an in-depth understanding of contextual content. It provides consistent and efficient information retrieval in multi-turn conversations.
Operation description
Usage notes
- Before you call this operation, make sure that you are familiar with the billing of Intelligent Media Management (IMM).
- Before you call this operation, make sure that you have indexed file metadata into the dataset automatically by calling the CreateBinding operation or manually by calling the IndexFileMeta or BatchIndexFileMeta operation.
- The response provided in this example is for reference only. The categories and content of metadata vary based on configurations of workflow templates. For any inquiries, join the DingTalk chat group (ID: 21714099) for feedback.
Limitations
- The conversation history can hold up to 100 messages, including user-sent messages and assistant-generated messages.
- Each message cannot exceed 1,000 characters in length.
Debugging
Authorization information
The following table shows the authorization information corresponding to the API. The authorization information can be used in the Action policy element to grant a RAM user or RAM role the permissions to call this API operation. Description:
- Operation: the value that you can use in the Action element to specify the operation on a resource.
- Access level: the access level of each operation. The levels are read, write, and list.
- Resource type: the type of the resource on which you can authorize the RAM user or the RAM role to perform the operation. Take note of the following items:
- For mandatory resource types, indicate with a prefix of * .
- If the permissions cannot be granted at the resource level,
All Resourcesis used in the Resource type column of the operation.
- Condition Key: the condition key that is defined by the cloud service.
- Associated operation: other operations that the RAM user or the RAM role must have permissions to perform to complete the operation. To complete the operation, the RAM user or the RAM role must have the permissions to perform the associated operations.
| Operation | Access level | Resource type | Condition key | Associated operation |
|---|---|---|---|---|
| imm:ContextualRetrieval | none | *All Resources * |
| none |
Request parameters
| Parameter | Type | Required | Description | Example |
|---|---|---|---|---|
| ProjectName | string | Yes | The name of the project. For more information, see CreateProject . | test-project |
| DatasetName | string | Yes | The dataset. | test-dataset |
| Messages | array | Yes | The conversation or tool invocation history. The latest message is at the end of the list (with an index number of n-1), whereas the earliest message is at the beginning of the list (with an index number of 0). Historical messages must be provided in user-assistant pairs. The maximum number of messages that you can specify is 2*n+1. The current question cannot exceed 1,000 characters in length. The maximum number of historical messages allowed is 100. | |
| ContextualMessage | Yes | The message list. | ||
| RecallOnly | boolean | No | Indicates whether to enable recall-only (embedding-based search). If you set this parameter to true, returned results have not been re-ranked and can be ranked in custom order. Default value: false. | false |
| SmartClusterIds | array | No | The IDs of clusters from which results are retrieved. | |
| string | No | The ID of the cluster. | SmartCluster-01686226-be59-4391-a1ac-b7b073455de3 |
Response parameters
Examples
Sample success responses
JSONformat
{
"RequestId": "6E93D6C9-5AC0-49F9-914D-E02678D3****",
"Results": [
{
"OwnerId": "102321002467****",
"ProjectName": "test-project",
"DatasetName": "test-dataset",
"ObjectType": "file",
"ObjectId": "75d5de2c50754e3dadd5c35dbca5f9949369e37eb342a73821f690c94c36c7f7",
"UpdateTime": "2021-06-29T14:50:13.011643661+08:00",
"CreateTime": "2021-06-29T14:50:13.011643661+08:00",
"URI": "oss://examplebucket/sampleobject.jpg",
"OSSURI": "oss://examplebucket/sampleobject.jpg",
"Filename": "sampleobject.jpg",
"MediaType": "image",
"ContentType": "image/jpeg",
"Size": 1000,
"FileHash": "1d9c280a7c4f67f7ef873e28449dbe17",
"FileModifiedTime": "2021-06-29T14:50:13.011643661+08:00",
"FileCreateTime": "2021-06-29T14:50:13.011643661+08:00",
"FileAccessTime": "2021-06-29T14:50:13.011643661+08:00",
"ProduceTime": "2021-06-29T14:50:13.011643661+08:00",
"LatLong": "30.134390,120.074997",
"Timezone": "",
"Addresses": [
{
"Language": "zh-hans",
"AddressLine": "",
"Country": "",
"Province": "",
"City": "",
"District": "",
"Township": ""
}
],
"TravelClusterId": "-",
"Orientation": 0,
"Figures": [
{
"FigureId": "868a9e74-cde5-4c7a-9013-28bb984****",
"FigureConfidence": 1,
"FigureClusterId": "Cluster-dbe72fec-b84c-4ab6-885b-3678e64****",
"FigureClusterConfidence": 1,
"FigureType": "face",
"Age": 29,
"AgeSD": 5,
"Gender": "female",
"GenderConfidence": 1,
"Emotion": "happiness",
"EmotionConfidence": 0.70784568786621,
"FaceQuality": 0.960875928401947,
"Boundary": {
"Width": 200,
"Height": 300,
"Left": 10,
"Top": 30,
"Polygon": [
{
"X": 10,
"Y": 10
}
]
},
"Mouth": "close",
"MouthConfidence": 1,
"Beard": "none",
"BeardConfidence": 1,
"Hat": "none",
"HatConfidence": 1,
"Mask": "none",
"MaskConfidence": 1,
"Glasses": "none",
"GlassesConfidence": 0.8,
"Sharpness": 0.7,
"Attractive": 0.96,
"HeadPose": {
"Pitch": 18.385589599609375,
"Roll": 4.204030513763428,
"Yaw": 2.4945924282073975
}
}
],
"FigureCount": 10,
"Labels": [
{
"Language": "zh-Hans",
"LabelName": "",
"LabelLevel": 1,
"LabelConfidence": 0.9891784601980591,
"ParentLabelName": "",
"CentricScore": 0.7319999933242798
}
],
"Title": "test",
"ImageWidth": 270,
"ImageHeight": 500,
"EXIF": {
"Compression": {
"value": 6
},
"DateTime": {
"value": "2020:08:19 17:11:11"
}
},
"ImageScore": {
"OverallQualityScore": 0.736
},
"CroppingSuggestions": [
{
"AspectRatio": "2:3",
"Confidence": 0.7079545259475708,
"Boundary": {
"Width": 200,
"Height": 300,
"Left": 10,
"Top": 30,
"Polygon": [
{
"X": 10,
"Y": 10
}
]
}
}
],
"OCRContents": [
{
"Language": "zh-hans",
"Contents": "",
"Confidence": 0.8254936695098877,
"Boundary": {
"Width": 200,
"Height": 300,
"Left": 10,
"Top": 30,
"Polygon": [
{
"X": 10,
"Y": 10
}
]
}
}
],
"VideoWidth": 1080,
"VideoHeight": 1920,
"VideoStreams": [
{
"Index": 0,
"Language": "en",
"CodecName": "h264",
"CodecLongName": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
"Profile": "High",
"CodecTimeBase": "1373627/83160000",
"CodecTagString": "avc1",
"CodecTag": "0x31637661",
"Width": 1280,
"Height": 720,
"HasBFrames": 2,
"SampleAspectRatio": "1:1",
"DisplayAspectRatio": "16:9",
"PixelFormat": "yuv420p",
"Level": 40,
"FrameRate": "25/1",
"AverageFrameRate": "25/1",
"TimeBase": "1/12800",
"StartTime": 0,
"Duration": 22.88,
"Bitrate": 5407765,
"FrameCount": 572,
"Rotate": 90,
"BitDepth": 8,
"ColorSpace": "bt709",
"ColorRange": "tv",
"ColorTransfer": "bt709",
"ColorPrimaries": "bt709"
}
],
"Subtitles": [
{
"Index": 2,
"Language": "en",
"CodecName": "mov_text",
"CodecLongName": "MOV text",
"CodecTagString": "tx3g",
"CodecTag": "0x67337874",
"StartTime": 0,
"Duration": 71.378,
"Bitrate": 74,
"Content": "",
"Width": 600,
"Height": 30
}
],
"AudioStreams": [
{
"Index": 1,
"Language": "en",
"CodecName": "aac",
"CodecLongName": "AAC (Advanced Audio Coding)",
"CodecTimeBase": "1/44100",
"CodecTagString": "mp4a",
"CodecTag": "0x6134706d",
"TimeBase": "1/48000",
"StartTime": 0.0235,
"Duration": 3.690667,
"Bitrate": 320087,
"FrameCount": 173,
"Lyric": "test",
"SampleFormat": "fltp",
"SampleRate": 48000,
"Channels": 2,
"ChannelLayout": "stereo"
}
],
"Artist": "Jane",
"AlbumArtist": "Jane",
"AudioCovers": [
{
"ImageWidth": 500,
"ImageHeight": 820,
"EXIF": {
"FileSize": {
"value": 29304
},
"Format": {
"value": "jpg"
}
},
"ImageScore": {
"OverallQualityScore": 0.736
},
"CroppingSuggestions": [
{
"AspectRatio": "2:3",
"Confidence": 0.7079545259475708
}
],
"OCRContents": [
{
"Language": "zh-hans",
"Contents": "",
"Confidence": 0.8254936695098877
}
]
}
],
"Composer": "Jane",
"Performer": "Jane",
"Language": "eng",
"Album": "FirstAlbum",
"PageCount": 5,
"ETag": "1D9C280A7C4F67F7EF873E28449****",
"CacheControl": "no-cache",
"ContentDisposition": "attachment; filename =test.jpg",
"ContentEncoding": "UTF-8",
"ContentLanguage": "zh-CN",
"AccessControlAllowOrigin": "https://aliyundoc.com",
"AccessControlRequestMethod": "PUT",
"ServerSideEncryptionCustomerAlgorithm": "SM4",
"ServerSideEncryption": "AES256",
"ServerSideDataEncryption": "SM4",
"ServerSideEncryptionKeyId": "9468da86-3509-4f8d-a61e-6eab1eac****",
"OSSStorageClass": "Standard",
"OSSCRC64": 559890638950338000,
"ObjectACL": "default",
"ContentMd5": "HZwoCnxPZ/fvhz4oRJ2+Fw==",
"OSSUserMeta": {
"key": "val"
},
"OSSTaggingCount": 2,
"OSSTagging": {
"key": "val"
},
"OSSExpiration": "2120-01-01T12:00:00.000Z",
"OSSVersionId": "CAEQNhiBgMDJgZCA0BYiIDc4MGZjZGI2OTBjOTRmNTE5NmU5NmFhZjhjYmY0****",
"OSSDeleteMarker": "CAEQMhiBgIDXiaaB0BYiIGQzYmRkZGUxMTM1ZDRjOTZhNjk4YjRjMTAyZjhl****",
"OSSObjectType": "Normal",
"CustomId": "\t\nmember-image-id-0001",
"CustomLabels": {
"MemberName": "Tim",
"Enabled": "True",
"ItemCount": 10
},
"StreamCount": 1,
"ProgramCount": 1,
"FormatName": "mov",
"FormatLongName": "QuickTime / MOV",
"StartTime": 0,
"Bitrate": 13091201,
"Duration": 15.263,
"SemanticSimilarity": 0.855123,
"SemanticTypes": [
"content"
],
"Elements": [
{
"ElementContents": [
{
"Type": "text",
"Content": "",
"URL": "http://aliyun.com",
"TimeRange": [
0
]
}
],
"ObjectId": "id1",
"ElementType": "",
"SemanticSimilarity": 0.8,
"ElementRelations": [
{
"Type": "child，parent",
"ObjectId": "id1"
}
]
}
],
"SceneElements": [
{
"TimeRange": [
100
],
"FrameTimes": [
30
]
}
],
"OCRTexts": "Alibaba Cloud IMM\n",
"FileStatus": "",
"Reason": "[InternalError] The request has been failed due to some unknown error. status: 500, requestId: CC5ACFBD-BB7A-496D-A9D6-****",
"SmartClusters": [
{
"SmartClusterId": "SmartCluster-12cd1645-deae-4b5e-9434-613747b75f6d",
"Similarity": 0.9
}
],
"ObjectStatus": "Indexed",
"SequenceNumber": 3,
"ClusterStatus": {
"key": {
"key": [
"smart-cluster-modified-0001"
]
}
},
"ClusterModifications": {
"key": {
"key": [
"exlude"
]
}
},
"Insights": {
"Video": {
"Description": "",
"Caption": ""
},
"Image": {
"Caption": "",
"Description": ""
}
},
"MetaLockVersion": 0
}
]
}Error codes
For a list of error codes, visit the Service error codes.
Change history
| Change time | Summary of changes | Operation |
|---|
No change history