OpenSearch:Deploy services

Available models

Prerequisites

Before you begin, make sure you have:

• An AI Search Open Platform account

• (RAM users only) The Model Service-Service Deployment permission granted to your Resource Access Management (RAM) user

Deploy a model service

1. In the AI Search Open Platform console, go to Model Service > Service Deployment, then click Deploy Service.

2. On the Deploy Service page, configure the following settings.

   Setting	Description
   Service name	A name to identify this deployment
   Deployment region	The region where the service runs. Currently, only Germany (Frankfurt) is supported.
   Resource type	The compute resource type for model inference
   Estimated price	The estimated cost for the selected configuration

   

3. Click Deploy.

After you click Deploy, the console shows the service status in the service list.

View service credentials

To call a deployed service, you need its endpoint and credentials. In the service list, click Manage to open the service details page.

Test the service

Use curl to verify your deployment. Each request requires two credentials:

• Authorization header: your API key, retrieved from the service details page under API key

• Token header: the service Token (public or private network), retrieved from the service details page under Token

The following example calls a text vectorization model to generate embeddings for two input strings:

curl -X POST \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer <your-api-key>" \
     -H "Token: <your-service-token>" \
     "http://default-0fm.platform-cn-hangzhou.opensearch.aliyuncs.com/v3/openapi/deployments/******_1zj19x_1yc/predict" \
     -d '{
           "input": [
             "Science and technology is the primary productive force",
             "opensearch product documentation"
           ],
           "input_type": "query",
           "dimension": 567
         }'

Replace the following placeholders:

A successful response returns a JSON object with an embeddings array:

{
  "embeddings": [
    {
      "index": 0,
      "embedding": [
        -0.028656005859375,
        0.0218963623046875,
        -0.04168701171875,
        -0.0440673828125,
        0.02142333984375,
        0.012345678901234568,
        ...
        0.0009876543210987654
      ]
    }
  ]
}

Call the service through SDK

After testing, use the Python SDK to integrate inference calls into your application. All examples use get_prediction_with_options() to submit inference requests.

import json

from alibabacloud_tea_openapi.models import Config
from alibabacloud_searchplat20240529.client import Client
from alibabacloud_searchplat20240529.models import GetPredictionRequest
from alibabacloud_searchplat20240529.models import GetPredictionHeaders
from alibabacloud_tea_util import models as util_models

if __name__ == '__main__':
    config = Config(
        bearer_token="<your-api-key>",
        # Endpoint: remove the http:// or https:// prefix
        endpoint="default-xxx.platform-cn-shanghai.opensearch.aliyuncs.com",
        # Supports HTTP and HTTPS
        protocol="http"
    )
    client = Client(config=config)

    # Request body
    request = GetPredictionRequest().from_map({
        "body": {
            "input_type": "document",
            "input": ["search", "test"]
        }
    })

    # Pass the service Token in the request headers
    headers = GetPredictionHeaders(
        token="<your-service-token>"
    )

    runtime = util_models.RuntimeOptions()

    # deploymentId: the Service ID from the service details page
    response = client.get_prediction_with_options(
        "<your-service-id>",
        request,
        headers,
        runtime
    )
    print(response)

Replace the following placeholders:

What's next

• Text embedding API — Full API reference for text vectorization models

• Ranking service API — Full API reference for reranking models

• API key management — Create and manage API keys