AI Search Open Platform image content extraction - OpenSearch

OpenSearch allows you to call the image content extraction service via an API, integrate the service into your business workflows, and use the parsed text for image retrieval and question answering.

Service list

Service name	Service ID	Description	API QPS limit
Image Content Understanding Service 001	ops-image-analyze-vlm-001	Analyzes image content and recognizes text using a multimodal large model, enabling applications like image retrieval and question answering.	10 Note To request a higher API QPS limit, submit a ticket to technical support.
Image OCR Service 001	ops-image-analyze-ocr-001	Extracts text from images using OCR, enabling applications like image retrieval and question answering.

Get authentication credentials

The AI Search open platform requires an API key for authentication. For instructions, see Get an API key.
Get the service endpoint

You can call the service via the public network or a VPC. For details, see Get the service endpoint.

Create an asynchronous extraction task

Request method

POST

URL

{host}/v3/openapi/workspaces/{workspace_name}/image-analyze/{service_id}/async

host: The endpoint for the API service. You can access the API service over the public network or through a VPC. For more information, see Obtain an endpoint.

In the AI Search Open Platform console, choose API Keys from the left navigation pane. You can then find the Public API Domain Name and Private API Domain Name in the Access Domain Name section at the top of the page.

workspace_name: The workspace name. For example, default.
service_id: The built-in system service ID. For example, ops-image-analyze-vlm-001.

Request parameters

Header parameters

API key authentication

Parameter	Type	Required	Description	Example
Content-Type	String	Required	Specifies the media type of the request body. Must be application/json.	application/json
Authorization	String	Required	The API key for authentication, prefixed with Bearer .	Bearer OS-d1**2a

Body parameter

Parameter	Type	Required	Description	Example
service_id	String	Yes	Built-in service ID: ops-image-analyze-vlm-001 ops-image-analyze-ocr-001	ops-image-analyze-vlm-001
document.url	String	No	The URL of the file. Either this parameter or `document.content` must be specified. Supports `http` and `https` protocols.	http://path/to/***.jpg
document.content	String	No	The Base64-encoded content of the file. Either this parameter or `document.url` must be specified.	"aGVsbG8gd29ybGQ="
document.file_name	String	No	The file name. If this parameter is omitted, the name is inferred from `document.url`. Required if `document.url` is also omitted.	test.jpg
document.file_type	String	No	The file type (e.g., `jpg`, `jpeg`, `png`, `bmp`, `tiff`). If this parameter is omitted, the type is inferred from the extension of `document.file_name`. Required if the type cannot be automatically inferred.	jpg

Return parameters

Parameter	Type	Description	Value
result.task_id	String	The ID of the asynchronous image analysis task.	6177bf71-f87f-4d86-ab0c-e2b64dfe****

cURL request

curl -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  "http://***-hangzhou.opensearch.aliyuncs.com/v3/openapi/workspaces/default/image-analyze/ops-image-analyze-vlm-001/async"
  --data '{
    "document": {
      "url": "https://img01.yzcdn.cn/****/2017/05/11/FoTMgBa0SvUaAeFruY7i7O_EUMhf.jpg%21middle.jpg",
      "file_type": "jpg"
    }
  }' \

Response example

Normal response

{
	"request_id":"CD4E26F0-23FF-449C-83DC-20CC8FF1****",
        "latency":8.0,
        "http_code":200,
        "result":{
                  "task_id":"cd4e26f0-23ff-449c-83dc-20cc8ff1****"
        }
}

Error response example

If an access request fails, the output contains a code and a message explaining the error.

{
      "request_id":"0CCAC03B-D83F-432F-B6BA-C3049576****",
      "latency":0.0,
      "code":"InvalidParameter",
      "http_code":400,
      "message":"document.content or document.url required, and both cannot be present at the same time"
}

Asynchronous extraction task status

Request method

GET

URL

{host}/v3/openapi/workspaces/{workspace_name}/image-analyze/{service_id}/async/task-status?task_id=${task_id}

host: The API service endpoint. You can call the API service over the public network or through a VPC. For details, see Get a service endpoint.
workspace_name: The name of the workspace. For example, default.
service_id: The built-in service ID. For example, ops-image-analyze-vlm-001.
task_id: The task ID returned in the response when you create an image analysis task. For example, cd4e26f0-23ff-449c-83dc-20cc8ff1****.

Request parameters

Header parameters

API key authentication

Parameter	Type	Required	Description	Example
Content-Type	string	Yes	The request type. Must be `application/json`.	application/json
Authorization	string	Yes	The API key for authentication, prefixed with `Bearer` .	Bearer OS-d1**2a

Response parameters

Parameter	Type	Description	Example
request_id	String	The unique identifier for the API call.	3C09570D-12DB-46B4-BF0F-A100D79B****
latency	Float/Int	The request latency, in milliseconds.	3.0
result.task_id	String	The asynchronous task ID. This parameter is not returned for synchronous calls.	a7e4c0f6-874c-47e3-b05b-02278a96e****
result.status	String	The task status. Possible values: `PENDING`: The task is pending. `SUCCESS`: The task succeeded. `FAILED`: The task failed.	SUCCESS
result.data	Object	The image analysis results.	{"content":"The image shows XXXX", "content_type":"plain"}
result.data.content	String	The extracted image content.	"XXX"
result.data.content_type	String	The content type of the output. The value is always `plain`.	plain
usage.token_count	int	The token count for the output. This parameter applies to the `ops-image-analyze-vlm-001` service.	1234
usage.pv_count	int	The number of calls, fixed at 1. Applies to the `ops-image-analyze-ocr-001` service.	1

cURL request example

curl -X GET \
-H"Content-Type: application/json" \
-H "Authorization: Bearer Your API-KEY" \
"http://***-hangzhou.opensearch.aliyuncs.com/v3/openapi/workspaces/default/image-analyze/ops-image-analyze-vlm-001/async/task-status?task_id=d9781786-20b8-4fb4-bbb5-38f82e69****"

Response example

Successful response

{
      "request_id":"3C09570D-12DB-46B4-BF0F-A100D79B****",
      "latency":3.0,
      "http_code":200,
      "result":{
           "status":"SUCCESS",
           "data":{
                "content":"The image shows a WMF blender surrounded by fruits and vegetables. Next to it is a glass of red juice with a straw. On the table are lemon slices, strawberries, and kiwifruit. A cut pineapple and an orange sit in one corner. Chopped carrots are in the blender, ready for juicing. The scene looks healthy and appetizing.",
                "content_type":"plain"
            },
            "task_id":"d9781786-20b8-4fb4-bbb5-38f82e69****"
       },
            "usage":{
                "token_count":95
            }
}

Sample error response

When an access request fails, the output includes a code and message explaining the error.

{
  "request_id":"153FC253-468D-4C46-873E-2AEB918C****",
  "latency":2.0,
  "code":"BadRequest.TaskNotExist",
  "http_code":404,
  "message":"task[d9781786-20b8-4fb4-bbb5-38f82e690b****] does not exist"
}

Creating a synchronous extraction task

Request method

POST

URL

{host}/v3/openapi/workspaces/{workspace_name}/image-analyze/{service_id}/sync

Parameters

host: The API service endpoint. You can call the API service over the public network or through a VPC. For more information, see Get a service endpoint.
workspace_name: The name of your workspace. For example, default.
service_id: The ID of the built-in service. For example, ops-image-analyze-vlm-001.

Parameters

Header parameters

API key authentication

Parameter	Type	Required	Description	Example value
Content-Type	String	Yes	The media type of the request.	application/json
Authorization	String	Yes	API key	Bearer OS-d1**2a

Body parameters

Parameter	Type	Required	Description	Example
service_id	String	Yes	The built-in service ID. Valid values: ops-image-analyze-vlm-001 ops-image-analyze-ocr-001	ops-image-analyze-vlm-001
document.url	String	No	The URL of the file. Supports HTTP and HTTPS. You must specify either `document.url` or `document.content`.	http://path/to/***.jpg
document.content	String	No	The Base64-encoded content of the file. You must specify either `document.url` or `document.content`.	"aGVsbG8gd29ybGQ="
document.file_name	String	No	The file name. If this parameter is left blank, the name is inferred from `document.url`. If `document.url` is also blank, you must specify this parameter.	test.jpg
document.file_type	String	No	The file type. If this parameter is blank, the type is inferred from the file name extension. If the type cannot be inferred, you must explicitly specify the file type. Examples: `jpg`, `jpeg`, `png`, `bmp`, and `tiff`.	jpg

Response parameters

Parameter	Type	Description	Example value
result.status	String	The task status. Possible values: PENDING: The task is in progress. SUCCESS: The task succeeded. FAIL: The task failed.	SUCCESS
result.error	String	The error message when `result.status` is `FAIL`. This field is empty otherwise.	Failed to decrypt the document.
result.data	Object	The results of the image analysis.	{"content":"The image shows XXXX", "content_type":"plain"}
result.data.content	String	The extracted content from the image.	"XXX"
result.data.content_type	String	The type of the output content. The value is always `plain`.	plain
request_id	String	A unique identifier for the API call.	B4AB89C8-B135-xxxx-A6F8-2BAB801A2CE4
latency	number	The request latency, in ms.	10
usage	Object	Usage for this API call.	"usage": { "token_count": 1234 }
usage.token_count	integer	The output token count. This applies to the `ops-image-analyze-vlm-001` service.	1234
usage.pv_count	integer	The invocation count. The value is fixed at 1. This applies to the `ops-image-analyze-ocr-001` service.	1

cURL

This section provides a sample API call and a glossary of relevant technical terms.
API call example

The following example shows how to make a synchronous call to the image analysis API endpoint.
curl -X POST \
-H"Content-Type: application/json" \
-H "Authorization: Bearer " \
 "http://***-hangzhou.opensearch.aliyuncs.com/v3/openapi/workspaces/default/image-analyze/ops-image-analyze-vlm-001/sync" \
\ -d "{    
      \"document\":{    
            \"url\":\"https://img01.yzcdn.cn/****/2017/05/11/FoTMgBa0SvUaAeFruY7i7O_EUMhf.jpg%21middle.jpg\",   
            \"file_type\":\"jpg\"
      }
}"
Terminology

The following table lists the technical terms used in this API call, their standard formatting, and related conventions.



Term
Standard form
Type
Notes




API key
API key
Industry Standard
Standard industry term. Use lowercase in prose.


China (Hangzhou)
China (Hangzhou)
Alibaba Cloud Region
The official region name. The identifier hangzhou in the URL is for system use.


Open Search
Open Search
Alibaba Cloud Product
The official product name. The identifier opensearch in the URL is for system use.


workspace
workspace
Industry Standard
A common cloud resource concept. Use lowercase.


image analysis
image analysis
Industry Standard
A functional description. The identifier image-analyze in the URL is for system use.


sync / async
sync / async
Industry Standard
Standard terms for synchronous and asynchronous operations.


document
document
JSON Field
A standard JSON field name. Use lowercase.


url
url
JSON Field
A standard JSON field name. Use lowercase.


file_type
file_type
JSON Field
A standard JSON field name. Use lowercase.



UI elements

This API call example does not reference any user interface (UI) elements.

Response example

Successful response

{
    "request_id":"BB5CD4C3-C8B6-40E7-A037-4ADAE88A****", 
    "latency":12525.0,
    "http_code":200,
    "result":{
        "status":"SUCCESS",
        "data":{
              "content":"The image shows a WMF blender surrounded by fruit and vegetables. Next to the blender is a cup of red juice with a straw in it. Scattered on the table are a few lemon slices, several strawberries, and some kiwifruit. In one corner of the table, there is a cut pineapple and an orange. Additionally, chopped carrots are in the blender, ready to be juiced. Everything looks healthy and delicious.",
              "content_type":"plain"
        }
      },
      "usage":{
          "token_count":95
      }
}

Error response

A failed access request returns a code and a message explaining the error.

{
    "request_id": "6F33AFB6-A35C-4DA7-AFD2-9EA16CCF****",
    "latency": 2.0,
    "code": "InvalidParameter",
    "http_code": 400,
    "message": "JSON parse error: Cannot deserialize value of type `ImageStorage` from String \\"xxx\\"
}

Status codes

HTTP status code	Error code	Description
200	-	A successful request does not guarantee task success; check the task status in `result.status`.
404	BadRequest.TaskNotExist	The task does not exist.
400	InvalidParameter	The request is invalid.
500	InternalServerError	An internal error occurred.

For more information about status codes, see status codes.