All Products
Search
Document Center

SDKs for data collection 2.0

Last Updated: Sep 09, 2021

SDKs for data collection

Overview of SDK features

SDKs for data collection provide the class that is used to collect data and push documents.

You can use the SDKs to manage data collection and document pushes for OpenSearch applications. You can push one or more documents at a time.

Download the SDK for Java 3.4.0

Download the SDK for PHP 3.2.0

  • Class name: DataCollectionClient

  • Namespace: OpenSearch\Client

Method description

A constructor.

Method definition

voidOpenSearch\Client\DataCollectionClient::__construct(\OpenSearch\Client\OpenSearchClient $openSearchClient)

Parameter description

Parameter

Type

Description

$openSearchClient

OpenSearch\Client\OpenSearchClient

The basic class that is used to calculate signatures, interact with the server, and return the results.


add

Method description

Adds a document.

Note

Note: After you call this method to add a document, the document is added to the SDK client buffer but is not committed to the server. The document is committed to the server only when you call the commit() method. You can call the add() method multiple times to add multiple documents, and then call the commit() method to commit the documents to the server at a time.

Method definition

\OpenSearch\Generated\Common\OpenSearchResult OpenSearch\Client\DataCollectionClient::add(array $fields)

Parameter description

Parameter

Type

Description

$fields

ARRAY

All fields in a document of behavioral data, user data, or goods data. Example: array("user_id" => "1021468", "bhv_type" => "click");


commit

Method description

Commits documents in the SDK client buffer to the server.

Note

Note: Before the documents in the buffer are committed to the server, the documents are cleared from the buffer. Therefore, if the server returns an error and you want to commit the documents again, you must regenerate the documents before you commit them. This prevents the loss of data.

Method definition

\OpenSearch\Generated\Common\OpenSearchResult OpenSearch\Client\DataCollectionClient::commit(string $searchAppName,string $dataCollectionName,string $dataCollectionType)

Parameter description

Parameter

Type

Description

$searchAppName

STRING

The name of the associated OpenSearch application.

$dataCollectionName

STRING

The name of data collection, which is returned by the OpenSearch console when the feature of collecting behavioral data is enabled.

$dataCollectionType

STRING

Data collection type: BEHAVIOR


push

Method description

Pushes multiple documents at a time.

Note

Note: This operation synchronously sends documents to the server.

Method definition

\OpenSearch\Generated\Common\OpenSearchResult OpenSearch\Client\DataCollectionClient::push(string $docJson,string $searchAppName,string $dataCollectionName,string $dataCollectionType)

Parameter description

Parameter

Type

Description

$docJson

STRING

The list of documents in JSON format.

$searchAppName

STRING

The name of the associated OpenSearch application.

$dataCollectionName

STRING

The name of data collection, which is returned by the OpenSearch console when the feature of collecting behavioral data is enabled.

$dataCollectionType

STRING

Data collection type: BEHAVIOR


Demo for using the push() method of the SDK for PHP 3.2.0 to collect data

<?php
require_once("Config.inc.php");
use OpenSearch\Client\DataCollectionClient;
use OpenSearch\Generated\DataCollection\Command;
$searchAppName = "opensearch_app_name";
$dataCollectionName = "opened_data_collection_name";
$dataCollectionType = "BEHAVIOR";
$docs = json_encode(array(
    [
        "cmd" => Command::$__names[Command::ADD],
        "fields" => [
            // The unique user ID.
            "user_id" => "1120021255", 
            // The numeric ID used to distinguish between different search services. A numeric ID corresponds to an OpenSearch application.
            "biz_id" => 1365378,
            // The value of this field is the value of the request_id parameter that OpenSearch returns in the search results. Pass in the value of the request_id parameter as it is.
            "rn" => "156516585419723283227314",
            // If the document is searched and collected from OpenSearch, set this field to Alibaba.
            "trace_id" => "Alibaba",
            // The value of this field is the value of the ops_request_misc parameter that OpenSearch returns in the search results. Pass in the value of the ops_request_misc parameter as it is.
            "trace_info" => "%7B%22request%5Fid%22%3A%22156516585419723283227314%22%2C%22scm%22%3A%2220140713.120006678..%22%7D",
            // The value of this field is the primary key value of the primary table in the OpenSearch application.
            "item_id" => "2223",
            // The type of the item is goods or commodity.
            "item_type" => "goods",
            // The behavioral data that is collected on clicks.
            "bhv_type" => "click",
            // The time at which the behavior occurs. The value is a UNIX timestamp that is accurate to the second.
            "bhv_time" => "1566475047"
        ]
    ]
));
// Create the DataCollectionClient object and use the OpenSearchClient object as the construction parameter.
$dataCollectionClient = new DataCollectionClient($client);
$ret = $dataCollectionClient->push($docs, $searchAppName, $dataCollectionName, $dataCollectionType);
print_r(json_decode($ret->result, true));

Demo for using the commit() method of the SDK for PHP 3.2.0 to collect data

<?php
require_once("Config.inc.php");
use OpenSearch\Client\DataCollectionClient;
use OpenSearch\Generated\DataCollection\Command;
$searchAppName = "opensearch_app_name";
$dataCollectionName = "opened_data_collection_name";
$dataCollectionType = "BEHAVIOR";
// Create the DataCollectionClient object and use the OpenSearchClient object as the construction parameter.
$dataCollectionClient = new DataCollectionClient($client);
// Add a document.
// The document is added to the SDK client buffer but is not committed to the server. The document is committed to the server only when you call the commit() method. 
// You can call the add() method multiple times to add multiple documents, and then call the commit() method to commit the documents to the server at a time. 
$dataCollectionClient->add([
    // The unique user ID.
    "user_id" => "1120021255", 
    // The numeric ID used to distinguish between different search services. A numeric ID corresponds to an OpenSearch application.
    "biz_id" => 1365378,
    // The value of this field is the value of the request_id parameter that OpenSearch returns in the search results. Pass in the value of the request_id parameter as it is.
    "rn" => "156516585419723283227314",
    // If the document is searched and collected from OpenSearch, set this field to Alibaba.
    "trace_id" => "Alibaba",
    // The value of this field is the value of the ops_request_misc parameter that OpenSearch returns in the search results. Pass in the value of the ops_request_misc parameter as it is.
    "trace_info" => "%7B%22request%5Fid%22%3A%22156516585419723283227314%22%2C%22scm%22%3A%2220140713.120006678..%22%7D",
    // The value of this field is the primary key value of the primary table in the OpenSearch application.
    "item_id" => "2223",
    // The type of the item is goods or commodity.
    "item_type" => "goods",
    // The behavioral data that is collected on clicks.
    "bhv_type" => "click",
    // The time at which the behavior occurs. The value is a UNIX timestamp that is accurate to the second.
    "bhv_time" => "1566475047"
]);
$ret = $dataCollectionClient->commit($searchAppName, $dataCollectionName, $dataCollectionType);
print_r(json_decode($ret->result, true));

Demo for using the push() method of the SDK for Java 3.4.0 to collect data

package com.aliyun.opensearch.demo;
import com.aliyun.opensearch.DataCollectionClient;
import com.aliyun.opensearch.OpenSearchClient;
import com.aliyun.opensearch.sdk.generated.OpenSearch;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchResult;
public class PushDataCollectionDoc {
    private static String accesskey = "your ak";
    private static String secret = "your secret";
    private static String host = "your host";
    private static String searchAppName = "opensearch_app_name";
    private static String dataCollectionName = "opened_data_collection_name";
    private static String dataCollectionType = "BEHAVIOR";
    public static void main(String[] args) {
        // Create and construct the OpenSearch object.
        OpenSearch opensearch = new OpenSearch(accesskey, secret, host);
        // Create the OpenSearchClient object and use the OpenSearch object as the construction parameter.
        OpenSearchClient client = new OpenSearchClient(opensearch);
        // Create the DataCollectionClient object and use the OpenSearchClient object as the construction parameter.
        DataCollectionClient dataCollectionClient = new DataCollectionClient(client);
        // Push documents.
        String docJson = "[{\"cmd\":\"ADD\",\"fields\":{\"user_id\":\"1120021255\","+
                         "\"biz_id\":1365378,\"rn\":\"156516585419723283227314\","+
                         "\"trace_id\":\"Alibaba\","+
                         "\"trace_info\":\"%7B%22request%5Fid%22%3A%22156516585419723283227314%22%2C%22scm%22%3A%2220140713.120006678..%22%7D\","+
                         "\"item_id\":\"id\",\"item_type\":\"goods\","+
                         "\"bhv_type\":\"click\",\"bhv_time\":\"1566475047\"}}]";
        try {
            OpenSearchResult openSearchResult = dataCollectionClient.push(docJson, 
                                                searchAppName, dataCollectionName, 
                                                dataCollectionType);
            System.out.println(openSearchResult);
        } catch (Exception e) {
            e.printStackTrace();
            assertTrue(false);
            return;
        }
    }
}

Demo for using the commit() method of the SDK for Java 3.4.0 to collect data

package com.aliyun.opensearch.demo;
import com.aliyun.opensearch.DataCollectionClient;
import com.aliyun.opensearch.OpenSearchClient;
import com.aliyun.opensearch.sdk.generated.OpenSearch;
import com.aliyun.opensearch.sdk.generated.commons.OpenSearchResult;
import java.util.HashMap;
import java.util.Map;
public class PushDataCollectionDoc {
    private static String accesskey = "your ak";
    private static String secret = "your secret";
    private static String host = "your host";
    private static String searchAppName = "opensearch_app_name";
    private static String dataCollectionName = "opened_data_collection_name";
    private static String dataCollectionType = "BEHAVIOR";
    public static void main(String[] args) {
        // Create and construct the OpenSearch object.
        OpenSearch opensearch = new OpenSearch(accesskey, secret, host);
        // Create the OpenSearchClient object and use the OpenSearch object as the construction parameter.
        OpenSearchClient client = new OpenSearchClient(opensearch);
        // Create the DataCollectionClient object and use the OpenSearchClient object as the construction parameter.
        DataCollectionClient dataCollectionClient = new DataCollectionClient(client);
        Map<String, Object> fields = new HashMap<String, Object>();
        // The unique user ID.
        fields.put("user_id", "1120021255");
        // The numeric ID used to distinguish between different search services. A numeric ID corresponds to an OpenSearch application.
        fields.put("biz_id", 1365378);
        // The value of this field is the value of the request_id parameter that OpenSearch returns in the search results. Pass in the value of the request_id parameter as it is.
        fields.put("rn", "1564455556323223680397827");
        // If the document is searched and collected from OpenSearch, set this field to Alibaba.
        fields.put("trace_id", "Alibaba");
        // The value of this field is the value of the ops_request_misc parameter that OpenSearch returns in the search results. Pass in the value of the ops_request_misc parameter as it is.
        fields.put("trace_info", "%7B%22request%5Fid%22%3A%22156516585419723283227314%22%2C%22scm%22%3A%2220140713.120006678..%22%7D");
        // The value of this field is the primary key value of the primary table in the OpenSearch application.
        fields.put("item_id", "2223");
        // The type of the item is goods or commodity.
        fields.put("item_type", "goods");
        // The behavioral data that is collected on clicks.
        fields.put("bhv_type", "click");
        // The time at which the behavior occurs. The value is a UNIX timestamp that is accurate to the second.
        fields.put("bhv_time", "1566475047");
        // Add a document.
        // The document is added to the SDK client buffer but is not committed to the server. The document is committed to the server only when you call the commit() method. 
        // You can call the add() method multiple times to add multiple documents, and then call the commit() method to commit the documents to the server at a time. 
        dataCollectionClient.add(fields);
        try {
            OpenSearchResult openSearchResult = dataCollectionClient.commit(searchAppName, dataCollectionName, dataCollectionType);
            System.out.println(openSearchResult);
        } catch (Exception e) {
            e.printStackTrace();
            return;
        }
    }
}