How to create a custom LLM in OpenSearch LLM-Based Conversational Search Edition - OpenSearch

To enhance the search and conversational experience, OpenSearch LLM-Based Conversational Search Edition enables you to fine-tune a pre-trained, general-purpose LLM for your specific business scenarios. This article explains how to create and train a custom LLM.

Create a custom LLM

Note

The custom LLM feature is only available for Professional Edition instances of OpenSearch LLM-Based Conversational Search Edition.

Go to Model Management > LLM and click Create Custom LLM to open the configuration page.

Enter the required information for Model Name, Base Model, and Existing Q&A Pair.

Model Name: Must be 1 to 30 characters long, start with a letter, and contain only letters, numbers, and underscores. The name must be unique.
Base Model: Currently, only opensearch-llama2-13b is supported.

Existing Q&A Pair: Upload an XLSX file. The file size cannot exceed 128 MB. The filename can only contain letters, numbers, underscores, and hyphens. You can download the sample data to see the required format.

Example format:

Query	Answer
What is OpenSearch?	OpenSearch is a one-stop intelligent search service development platform built on a large-scale distributed search engine developed by Alibaba.
What are the use cases for the LLM-Based Conversational Search Edition?	OpenSearch LLM-Based Conversational Search Edition is suitable for various scenarios, such as internal enterprise knowledge bases, intelligent customer service, e-commerce, and content communities.

Billing Method: For details, see Billing overview of LLM-Based Conversational Search Edition Professional Edition.

Note

Limit: The maximum number of custom LLMs you can create is the same as the number of GPUs in your instance. You can view the GPU count on the instance details page. A single instance can have a maximum of 10 GPUs.

Click Submit to create the custom LLM. After the model is created, you can either click Train to start training immediately or click Done to return to the Model Management page.

Train a custom LLM

After you create a custom LLM, go to the Model Management page and click Details to view the model's Basic Information, Configuration Information, and Training History. At this point, the Model Status is Unavailable, and the Latest Version Status is Pending Training. You can click Configure to update the Q&A data.
On the Model Management page, click Train, and then click OK in the confirmation dialog to start training the model.
Click Details to check the model's status. The Model Status is still Unavailable, and the Latest Version Status changes to Training.
After the model is trained, click Test Effect to evaluate its performance.