Update data - OpenSearch - Alibaba Cloud Documentation Center

This topic describes how to use OpenSearch Vector Search Edition SDKs for Java and Python to update table data. You can upload, update, and delete documents.

Dependencies

Java

<dependency>
    <groupId>com.aliyun</groupId>
    <artifactId>aliyun-sdk-ha3engine-vector</artifactId>
    <version>1.1.8</version>
</dependency>

Python

# Requires: Python >=3.6
pip install alibabacloud_ha3engine_vector

Go

go get github.com/aliyun/alibabacloud-ha3-go-sdk@v1.1.8-vector

Java in asynchronous mode

<dependency>
  <groupId>com.aliyun</groupId>
  <artifactId>aliyun-sdk-ha3engine-async</artifactId>
  <version>1.1.4</version>
</dependency>

Parameter description

You must specify the following parameters in the SDKs for Java and Python: endpoint, instance_id, access_user_name, access_pass_word, and data_source_name.

endpoint: the internal or public endpoint.

You can view the endpoints in the Network Information and API Endpoint sections of the Instance Details page.

Important

After you turn on Public Access, you can access the OpenSearch Vector Search Edition instance by using the public endpoint of the instance on your on-premises machine. A public endpoint contains the word "public". You can configure an IP address whitelist for access to the instance. For more information, see Network Information section in the "View instance details" topic. If you do not configure such an IP address whitelist, an access prohibition error will be returned. We recommend that you ping the instance before you use it to ensure that the instance is accessible.

If you use an Elastic Compute Service (ECS) instance to access the OpenSearch Vector Search Edition instance, you can specify the same vSwitch as that of the ECS instance to access the OpenSearch Vector Search Edition instance by using the API endpoint.

instance_id: the ID of the OpenSearch Vector Search Edition instance.

access_user_name: the username.
access_pass_word: the password.

You can view the username and password in the API Endpoint section of the Instance Details page. The password is specified when you purchase the instance and can be modified.

data_source_name: the name of the API data source. The default value is in the format of Instance ID_Table name.

Example: ha-cn-zpr3dgzxg04_test_image_vector.

Data update demo

Upload a document

Java

import com.aliyun.ha3engine.vector.Client;
import com.aliyun.ha3engine.vector.models.Config;
import com.aliyun.ha3engine.vector.models.PushDocumentsRequest;
import com.aliyun.ha3engine.vector.models.PushDocumentsResponse;
import com.aliyun.tea.TeaException;
import org.junit.Before;
import org.junit.Test;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;

/**
 * @author alibaba
 */
public class PushDoc {

    /**
     * The engine client of the OpenSearch Vector Search Edition instance.
     */
    private Client client;

    @Before
    public void clientInit() throws Exception {
        /*
          Initialize the engine client.
         */
        Config config = new Config();

        // The name of the instance. You can view the instance name in the upper-left corner of the Instance Details page. Example: ha-cn-i7*****605.
        config.setInstanceId("ha-cn-i7*****605");
        // The username. You can view the username in the API Endpoint section of the Instance Details page.
        config.setAccessUserName("username");
        // The password. You can modify the password in the API Endpoint section of the Instance Details page.
        config.setAccessPassWord("password");
        // The API endpoint of the instance. You can view the API endpoint in the API Endpoint section of the Instance Details page.
        config.setEndpoint("ha-cn-i7*****605.public.ha.aliyuncs.com");

        client = new Client(config);
    }

    @Test
    public void add() throws Exception {
        // The name of the table to which the document is pushed. Format: <Instance ID>_<Table name>.
        String tableName = "<instance_id>_<table_name>";

        // The primary key field of the document whose data is to be pushed.
        String pkField = "<field_pk>";

        try {
            // The outer structure that is used to push documents. You can specify one or more document operations in the structure.
            ArrayList<Map<String, ?>> documents = new ArrayList<>();

            // The document to be uploaded.
            Map<String, Object> add2Document = new HashMap<>();
            Map<String, Object> add2DocumentFields = new HashMap<>();

            // The content of the document. Keys and values must be matched in pairs.
            // The value of the field_pk field must be the same as the value of the pkField field.
            add2DocumentFields.put("<field_pk>", "<field_pk_value>");
            add2DocumentFields.put("<field_map_key_1>", "<field_map_value_1>");
            add2DocumentFields.put("<field_map_key_2>", "<field_map_value_2>");

            // The content can be of multi-value attribute types supported by OpenSearch Vector Search Edition. Set the multi_value parameter to true when you configure an index table.
            ArrayList<Object> addDocumentMultiFields = new ArrayList<>();
            addDocumentMultiFields.add("multi_value_1");
            addDocumentMultiFields.add("multi_value_2");
            add2DocumentFields.put("<multi_value_key>", addDocumentMultiFields);

            // Add the document content to an add2Document structure.
            add2Document.put("fields", add2DocumentFields);
            // Run the add command to upload the document.
            add2Document.put("cmd", "add");
            documents.add(add2Document);

            // Push data.
            PushDocumentsRequest request = new PushDocumentsRequest();
            request.setBody(documents);
            PushDocumentsResponse response = client.pushDocuments(tableName, pkField, request);
            String responseBody = response.getBody();

            System.out.println("result:" + responseBody);

        } catch (TeaException e) {
            System.out.println(e.getCode());
            System.out.println(e.getMessage());
            Map<String, Object> exceptionData = e.getData();
            System.out.println(com.aliyun.teautil.Common.toJSONString(exceptionData));
        }
    }
}

Python

from alibabacloud_ha3engine_vector.client import Client
from alibabacloud_ha3engine_vector.models import Config
from alibabacloud_ha3engine_vector import models
from Tea.exceptions import TeaException, RetryError

config = Config(
    # The API endpoint of the instance. You can view the API endpoint in the API Endpoint section of the Instance Details page.
    endpoint="http://ha-cn-i7*****605.public.ha.aliyuncs.com",
    # The username. You can view the username in the API Endpoint section of the Instance Details page.
    access_user_name="username",
    # The password. You can modify the password in the API Endpoint section of the Instance Details page.
    access_pass_word="password")

# Initialize the engine client.
client = Client(config)

def push():
    # The name of the table to which the document is pushed. Format: <Instance ID>_<Table name>.
    tableName = "<instance_id>_<table_name>";

    try:
        # The document to be uploaded.
        # If the document already exists, the existing document is deleted, and the specified document is uploaded. 
        # =====================================================
        # The content of the document.
        add2DocumentFields = {
            "id": 1,                          # The ID of the primary key field. The value is of the INT type.
            "name": "Search",                    # The value of this single-value field is of the STRING type.
            "str_arr": "a\x1Db\x1Dc\x1Dd"     # The value of this multi-value field is of the STRING type.
        }

        # Add the document content to an add2Document structure.
        add2Document = {
            "fields": add2DocumentFields,
            "cmd": "add"                      # Run the add command to upload the document.
        }

        optionsHeaders = {}
        # The outer structure that is used to push documents. You can specify one or more document operations in the structure.
        documentArrayList = []
        documentArrayList.append(add2Document)
        pushDocumentsRequest = models.PushDocumentsRequest(optionsHeaders, documentArrayList)

        # The primary key field of the document whose data is to be pushed.
        pkField = "id"
        # Use the default runtime parameters for the request.
        response = client.push_documents(tableName, pkField, pushDocumentsRequest)
        print(response.body)

    except TeaException as e:
        print(f"send request with TeaException : {e}")
    except RetryError as e:
        print(f"send request with Connection Exception  : {e}")

if __name__ == "__main__":
    push()

Go

The demo dynamically encapsulates document data into Map objects and calls the add() method to add these Map objects to the cache. Then, the demo calls the pushDocuments() method to submit the document data in these Map objects at a time.

package main

import (
	"fmt"
	"github.com/alibabacloud-go/tea/tea"
	ha3engine "github.com/aliyun/alibabacloud-ha3-go-sdk/client"
)

func main() {
	// Create a Config instance.
	config := &ha3engine.Config{
		// The API endpoint of the instance. You can view the API endpoint in the API Endpoint section of the Instance Details page.
		Endpoint: tea.String("ha-cn-i7*****605.public.ha.aliyuncs.com"),
		// The username. You can view the username in the API Endpoint section of the Instance Details page.
		AccessUserName: tea.String("username"),
		// The password. You can modify the password in the API Endpoint section of the Instance Details page.
		AccessPassWord: tea.String("password"),
	}

	// Initialize a client for sending requests.
	client, _clientErr := ha3engine.NewClient(config)

	// If an error occurs when the system creates the client, _clientErr and an error message are returned.
	if _clientErr != nil {
		fmt.Println(_clientErr)
		return
	}
	docPush(client)
}

func docPush(client *ha3engine.Client) {
	pushDocumentsRequestModel := &ha3engine.PushDocumentsRequest{}
  // The name of the table to which the document is pushed. Format: <Instance ID>_<Table name>.
  tableName := "<instance_id>_<table_name>"
  // The primary key field of the document whose data is to be pushed.
  keyField := "<field_pk>"

	a := [20]int{}
	array := []map[string]interface{}{}
	for x := range a {
		filed := map[string]interface{}{
			"fields": map[string]interface{}{
				"id":          tea.ToString(x),
				"fb_boolean":  tea.BoolValue(nil),
				"fb_datetime": "2167747200000",
				"fb_string":   "409a6b18-a10b-409e-af91-07121c45d899",
			},
			"cmd": tea.String("add"),
		}
		array = append(array, filed)

		pushDocumentsRequestModel.SetBody(array)

		// Call the method for sending a request.
		response, _requestErr := client.PushDocuments(tea.String(dataSourceName), tea.String(keyField), pushDocumentsRequestModel)

		// If an error occurs when the system sends the request, _requestErr and an error message are returned.
		if _requestErr != nil {
			fmt.Println(_requestErr)
			return
		}

		// Display the response if no error occurs.
		fmt.Println(response)
	}
}

Java in asynchronous mode

import com.aliyun.ha3engine.async.AsyncClient;
import com.aliyun.ha3engine.async.models.PushDocumentsRequest;
import com.aliyun.ha3engine.async.models.PushDocumentsResponse;
import com.aliyun.sdk.ha3engine.async.core.AsyncConfigInfoProvider;
import com.aliyun.tea.TeaException;
import darabonba.core.client.ClientOverrideConfiguration;
import org.junit.Before;
import org.junit.Test;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;


/**
 * @author alibaba
 */
public class AddDoc {

    /**
     * The engine client of the OpenSearch Vector Search Edition instance.
     */
    private AsyncClient client;

    @Before
    public void clientInit() {
        // The username and password that are used to access the instance. You can view the username and password in the API Endpoint section of the Instance Details page.
        AsyncConfigInfoProvider provider = AsyncConfigInfoProvider.create("username", "password");
        // Initialize the asynchronous client.
        client = AsyncClient.builder()
                .credentialsProvider(provider)
                .overrideConfiguration(
                        ClientOverrideConfiguration.create()
                                .setEndpointOverride("ha-cn-i7*****605.public.ha.aliyuncs.com")
                                .setProtocol("http")
                ).build();
    }

    @Test
    public void add() {
        try {
            // The name of the table to which the document is pushed. Format: <Instance ID>_<Table name>.
            String tableName = "<instance_id>_<table_name>";

            // The primary key field of the document whose data is to be pushed.
            String pkField = "<field_pk>";

            // The outer structure that is used to push documents. You can specify one or more document operations in the structure.
            ArrayList<Map<String, ?>> documents = new ArrayList<>();

            // The document to be uploaded.
            Map<String, Object> add2Document = new HashMap<>();
            Map<String, Object> add2DocumentFields = new HashMap<>();

            // The content of the document. Keys and values must be matched in pairs.
            // The value of the field_pk field must be the same as the value of the pkField field.
            add2DocumentFields.put("<field_pk>", "<field_pk_value>");
            add2DocumentFields.put("<field_map_key_1>", "<field_map_value_1>");
            add2DocumentFields.put("<field_map_key_2>", "<field_map_value_2>");

            // The content can be of multi-value attribute types supported by OpenSearch Vector Search Edition. Set the multi_value parameter to true when you configure an index table.
            ArrayList<Object> addDocumentMultiFields = new ArrayList<>();
            addDocumentMultiFields.add("multi_value_1");
            addDocumentMultiFields.add("multi_value_2");
            add2DocumentFields.put("<multi_value_key>", addDocumentMultiFields);

            // Add the document content to an add2Document structure.
            add2Document.put("fields", add2DocumentFields);
            // Run the add command to upload the document.
            add2Document.put("cmd", "add");
            documents.add(add2Document);

            // Push data.
            PushDocumentsRequest request = PushDocumentsRequest.builder().body(documents).build();
            CompletableFuture<PushDocumentsResponse> responseCompletableFuture = client.pushDocuments(tableName, pkField, request);
            String responseBody = responseCompletableFuture.get().getBody();

            System.out.println("result:" + responseBody);
        } catch (ExecutionException | InterruptedException e) {
            System.out.println(e.getMessage());
        } catch (TeaException e) {
            System.out.println(e.getCode());
            System.out.println(e.getMessage());
            Map<String, Object> abc = e.getData();
            System.out.println(com.aliyun.teautil.Common.toJSONString(abc));
        }
        
    }
}

Note

If you call the add() method to push data, new data that uses the same primary key as the old data overwrites the old data.

Delete a document

Java

import com.aliyun.ha3engine.vector.Client;
import com.aliyun.ha3engine.vector.models.*;
import com.aliyun.tea.TeaException;
import org.junit.Before;
import org.junit.Test;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;

/**
 * @author alibaba
 */
public class PushDoc {

    /**
     * The engine client of the OpenSearch Vector Search Edition instance.
     */
    private Client client;

    @Before
    public void clientInit() throws Exception {
        /*
          Initialize the engine client.
         */
        Config config = new Config();

        // The name of the instance. You can view the instance name in the upper-left corner of the Instance Details page. Example: ha-cn-i7*****605.
        config.setInstanceId("ha-cn-i7*****605");
        // The username. You can view the username in the API Endpoint section of the Instance Details page.
        config.setAccessUserName("username");
        // The password. You can modify the password in the API Endpoint section of the Instance Details page.
        config.setAccessPassWord("password");
        // The API endpoint of the instance. You can view the API endpoint in the API Endpoint section of the Instance Details page.
        config.setEndpoint("ha-cn-i7*****605.public.ha.aliyuncs.com");

        client = new Client(config);
    }


    @Test
    public void delete() throws Exception {
        // The name of the table to which the document is pushed. Format: <Instance ID>_<Table name>.
        String tableName = "<instance_id>_<table_name>";

        // The primary key field of the document whose data is to be pushed.
        String pkField = "<field_pk>";

        try {
            // The outer structure that is used to push documents. You can specify one or more document operations in the structure.
            ArrayList<Map<String, ?>> documents = new ArrayList<>();

            // The document to be deleted.
            Map<String, Object> delete2Document = new HashMap<>();
            Map<String, Object> delete2DocumentFields = new HashMap<>();

            // The content of the document. Keys and values must be matched in pairs.
            // The value of the field_pk field must be the same as the value of the pkField field.
            delete2DocumentFields.put("<field_pk>", "<field_pk_value>");

            // Add the document content to a delete2Document structure.
            delete2Document.put("fields", delete2DocumentFields);
            // Run the delete command to delete the document.
            delete2Document.put("cmd", "delete");
            documents.add(delete2Document);

            // Push data.
            PushDocumentsRequest request = new PushDocumentsRequest();
            request.setBody(documents);
            PushDocumentsResponse response = client.pushDocuments(tableName, pkField, request);
            String responseBody = response.getBody();

            System.out.println("result:" + responseBody);

        } catch (TeaException e) {
            System.out.println(e.getCode());
            System.out.println(e.getMessage());
            Map<String, Object> exceptionData = e.getData();
            System.out.println(com.aliyun.teautil.Common.toJSONString(exceptionData));
        }
    }
}

Python

from alibabacloud_ha3engine_vector.client import Client
from alibabacloud_ha3engine_vector.models import Config
from alibabacloud_ha3engine_vector import models
from Tea.exceptions import TeaException, RetryError

config = Config(
    # The API endpoint of the instance. You can view the API endpoint in the API Endpoint section of the Instance Details page.
    endpoint="http://ha-cn-i7*****605.public.ha.aliyuncs.com",
    # The username. You can view the username in the API Endpoint section of the Instance Details page.
    access_user_name="username",
    # The password. You can modify the password in the API Endpoint section of the Instance Details page.
    access_pass_word="password")

# Initialize the engine client.
ha3EngineClient = Client(Config)


def pushDoc():
    # The name of the table to which the document is pushed. Format: <Instance ID>_<Table name>.
    tableName = "<instance_id>_<table_name>";

    try:
        # The outer structure that is used to push documents. You can specify one or more document operations in the structure.
        documentArrayList = []

        # The document to be deleted.
        # If you want to delete a document, you must specify the primary key field of the document.
        # If you build an index by using the multi-level hashing structure, you must specify the primary key field of each level of hash. 
        delete2DocumentFields = {
            "id": 1  # The ID of the primary key field. The value is of the INT type.
        }
        delete2Document = {
            "fields": delete2DocumentFields,  # Add the document content to a delete2Document structure.
            "cmd": "delete"  # Run the delete command to delete the document.
        }

        optionsHeaders = {}
        documentArrayList.append(delete2Document)
        pushDocumentsRequest = models.PushDocumentsRequest(
            optionsHeaders, documentArrayList
        )

        # The primary key field of the document whose data is to be pushed.
        pkField = "id"
        # Use the default runtime parameters for the request.
        response = ha3EngineClient.push_documents(
            tableName, pkField, pushDocumentsRequest
        )
        print(response)

    except TeaException as e:
        print(f"send request with TeaException : {e}")
    except RetryError as e:
        print(f"send request with Connection Exception  : {e}")


if __name__ == "__main__":
    pushDoc()

Go

package main

import (
    "fmt"
    "github.com/alibabacloud-go/tea/tea"
    ha3engine "github.com/aliyun/alibabacloud-ha3-go-sdk/client"
)

func main() {
    // Create a Config instance.
    config := &ha3engine.Config{
        // The API endpoint of the instance. You can view the API endpoint in the API Endpoint section of the Instance Details page.
        Endpoint: tea.String("ha-cn-i7*****605.public.ha.aliyuncs.com"),
        // The username. You can view the username in the API Endpoint section of the Instance Details page.
        AccessUserName: tea.String("username"),
        // The password. You can modify the password in the API Endpoint section of the Instance Details page.
        AccessPassWord: tea.String("password"),
    }

    // Initialize a client for sending requests.
    client, _clientErr := ha3engine.NewClient(config)

    // If an error occurs when the system creates the client, _clientErr and an error message are returned.
    if _clientErr != nil {
        fmt.Println(_clientErr)
        return
    }
    deleteDoc(client)
}

func deleteDoc(client *ha3engine.Client) {
    pushDocumentsRequestModel := &ha3engine.PushDocumentsRequest{}
    // The name of the table to which the document is pushed. Format: <Instance ID>_<Table name>.
    tableName := "<instance_id>_<table_name>"
    // The primary key field of the document whose data is to be pushed.
    keyField := "<field_pk>"

    var array []map[string]interface{}
    filed := map[string]interface{}{
        "fields": map[string]interface{}{
            "id": 2,
        },
        "cmd": tea.String("delete"),
    }
    array = append(array, filed)

    pushDocumentsRequestModel.SetBody(array)

    // Call the method for sending a request.
    response, _requestErr := client.PushDocuments(tea.String(tableName), tea.String(keyField), pushDocumentsRequestModel)

    // If an error occurs when the system sends the request, _requestErr and an error message are returned.
    if _requestErr != nil {
        fmt.Println(_requestErr)
        return
    }

    // Display the response if no error occurs.
    fmt.Println(response)
}

Java in asynchronous mode

import com.aliyun.ha3engine.async.AsyncClient;
import com.aliyun.ha3engine.async.models.PushDocumentsRequest;
import com.aliyun.ha3engine.async.models.PushDocumentsResponse;
import com.aliyun.sdk.ha3engine.async.core.AsyncConfigInfoProvider;
import com.aliyun.tea.TeaException;
import darabonba.core.client.ClientOverrideConfiguration;
import org.junit.Before;
import org.junit.Test;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;


/**
 * @author alibaba
 */
public class DeleteDoc {

    /**
     * The engine client of the OpenSearch Vector Search Edition instance.
     */
    private AsyncClient client;

    @Before
    public void clientInit() {
        // The username and password that are used to access the instance. You can view the username and password in the API Endpoint section of the Instance Details page.
        AsyncConfigInfoProvider provider = AsyncConfigInfoProvider.create("username", "password");
        // Initialize the asynchronous client.
        client = AsyncClient.builder()
                .credentialsProvider(provider)
                .overrideConfiguration(
                        ClientOverrideConfiguration.create()
                                .setEndpointOverride("ha-cn-i7*****605.public.ha.aliyuncs.com")
                                .setProtocol("http")
                ).build();
    }

    @Test
    public void delete() throws Exception {
        try {
            // The name of the table to which the document is pushed. Format: <Instance ID>_<Table name>.
            String tableName = "<instance_id>_<table_name>";

            // The primary key field of the document whose data is to be pushed.
            String pkField = "<field_pk>";

            // The outer structure that is used to push documents. You can specify one or more document operations in the structure.
            ArrayList<Map<String, ?>> documents = new ArrayList<>();

            // The document to be deleted.
            Map<String, Object> deleteDocument = new HashMap<>();
            Map<String, Object> deleteDocumentFields = new HashMap<>();

            // The content of the document. Keys and values must be matched in pairs.
            // The value of the field_pk field must be the same as the value of the pkField field.
            deleteDocumentFields.put("<field_pk>", "<field_pk_value>");

            // Add the document content to a deleteDocument structure.
            deleteDocument.put("fields", deleteDocumentFields);
            // Run the delete command to delete the document.
            deleteDocument.put("cmd", "delete");
            documents.add(deleteDocument);

            // Push data.
            PushDocumentsRequest request = PushDocumentsRequest.builder().body(documents).build();
            CompletableFuture<PushDocumentsResponse> responseCompletableFuture = client.pushDocuments(tableName, pkField, request);
            String responseBody = responseCompletableFuture.get().getBody();

            System.out.println("result:" + responseBody);
        } catch (ExecutionException | InterruptedException e) {
            System.out.println(e.getMessage());
        } catch (TeaException e) {
            System.out.println(e.getCode());
            System.out.println(e.getMessage());
            Map<String, Object> exceptionData = e.getData();
            System.out.println(com.aliyun.teautil.Common.toJSONString(exceptionData));
        }
    }
}

Additional information

For information about the response to a request, see Update data.
Do not run the go get github.com/aliyun/alibabacloud-ha3-go-sdk command to pull dependencies. The SDK dependencies for OpenSearch Vector Search Edition and OpenSearch Retrieval Engine Edition are classified into the same tag in GitHub. You must specify the corresponding edition based on the instance edition when you pull dependencies.