How to deploy DeepSeek or Qwen models on the AnalyticDB for MySQL console - AnalyticDB

The AnalyticDB Ray platform provides AI applications as a standardized framework for deploying AI solutions. The inference service lets you deploy trained models such as DeepSeek and Qwen as high-performance online API services with low latency, high concurrency, and auto scaling, making them ideal for AI inference scenarios such as natural language processing and computer vision. This topic describes how to create and use an inference service.

Prerequisites

An AnalyticDB for MySQL Enterprise Edition, Basic Edition, or Data Lakehouse Edition cluster is created.

Billing

A Ray Cluster resource group incurs the following charges:

Head node disk space and Worker Disk Storage are billed based on configured storage capacity.
If Head Node Resource Type and Worker Resource Type are set to CPU, you are billed for ACU-based elastic resource groups.
If Head Node Resource Type and Worker Resource Type are set to GPU, you are billed based on GPU specifications and quantity.

Procedure

Step 1: Create an inference service

Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. Find the cluster that you want to manage and click the cluster ID.
In the left-side navigation pane, click AI Applications.
Click Create Inference Service.

In the Create Inference Service panel, configure the following parameters:

Basic Settings:

Parameter	Description
Application name	A custom name for the AI application.
Open-source LLM	Supported options: Deepseek-R1 Qwen32B
Inference framework	Supported options: SGLang SGLang is a fast serving framework for large language models and vision-language models. vLLM vLLM is a popular library for accelerating LLM inference.

Resource Configuration:

To use an existing Ray Cluster resource group, click Use Existing Resource Group and select the target resource group from the Resource Group Name drop-down list.

To create a new Ray Cluster resource group, click Create Resource Group.

Configure worker parameters here. Adjust other settings on the Resource Group Management page after the resource group is created.

Parameter	Description
Resource specifications	Worker resource specifications. Default: ADB.MLLarge.24. GPU model availability varies. For sizing assistance, submit a ticket.
Number of resources	Number of workers in the worker group. Default: 1.
Allocation unit	Number of GPUs per worker node. Default: 1.

Storage Configuration (Optional):

Parameter	Description
Storage type	Supported types: OSS NAS
Storage name	Select an existing storage volume. If none are available, click Create Now to create a storage volume.
Mount path	The mount path for the AI application data.

Note

To mount multiple storage volumes, click Add Storage and configure each volume.

Network Settings (Optional):

Parameter	Description
Mount elastic network interface (ENI)	Enable this switch to ensure access to mounted storage volumes.

After you configure the parameters, click OK.

Step 2: Access the inference service

Obtain the invocation information for the AI application.
1. On the AI Applications page, click the Application Management tab.
2. In the Service Call column for the target application, click Invocation Information.
  
  The panel displays invocation details such as the base-url (base URL), api-key, and model-name (in this example, deepseek-r1).

Invoke the inference service using an OpenAI-compatible method.

Replace the placeholders in the following example with the information you obtained.

import os
from openai import OpenAI
client = OpenAI(
    # Your api-key, obtained from the invocation information.
    # For security, use an environment variable and do not hard-code the key in production code.
    api_key="<api-key>", 
    # Your base URL, from the invocation information.
    base_url="https://<base-url>",
)
completion = client.chat.completions.create(
    model="<model-name>",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello"},
    ]
)
print(completion.model_dump_json())