Alibaba Cloud OpenSearch is a full-text search service that is built on the Apsara distributed system by Alibaba Cloud. OpenSearch enables developers to search, sort, filter, and aggregate various types of data, such as text, number, and time data. This tutorial walks you through the process and procedure of using OpenSearch and provides a few usage notes.

Process

1. Create an Alibaba Cloud account and activate OpenSearch.
2. Create a data table and design the schema of the table.
3. Upload data.
4. Create a search application.
5. Configure indexes and analyzers.
6. Start a search.

Procedure

1. Log on to the OpenSearch console and go to the Instance Management page.
2. Create a data table, specify the fields and data types for the table, and then configure a unique identifier. Then, upload data to the table.
3. Create a search application and specify the name, description, and type of the application.
4. Configure indexes and analyzers. Specify the field weight, search scope, and search method.
5. Start a search. You can use the OpenSearch API or SDK to search data.

Usage notes

1. When you design the schema for a data table, make sure that the schema that includes the data type, field, and unique identifier meets your search requirements.
2. When you upload data, make sure that the data is in the valid format and the amount of the data does not exceed the upper limit. If you want to upload large amounts of data, you must upload the data in batches.
3. When you create a search application, make sure that you specify a suitable type and description for the application.
4. When you configure indexes and analyzers, make sure that the configurations such as the field weight, search method, and search scope meet your business requirements.
5. When you call the OpenSearch API or SDK, make sure that your call, including the request and response of the call, follows the syntax of OpenSearch and does not exceed any limit.

Demo: Use OpenSearch SDK to search data

    // Add the following dependencies to the pom.xml file of your project to import OpenSearch SDK. 
    <dependency>
        <groupId>com.aliyun</groupId>
        <artifactId>aliyun-java-sdk-core</artifactId>
        <version>4.5.3</version>
    </dependency>
    <dependency>
         <groupId>com.aliyun</groupId>
         <artifactId>aliyun-java-sdk-opensearch</artifactId>
         <version>2020-09-28</version>
    </dependency>
    
    //Create an OpenSearch object.
    //Use your AccessKey ID and AccessKey secret to create an OpenSearch object.
    
    import com.aliyuncs.DefaultAcsClient;
    import com.aliyuncs.exceptions.ClientException;
    import com.aliyuncs.exceptions.ServerException;
    import com.aliyuncs.opensearch.model.v20171225.SearchRequest;
    import com.aliyuncs.opensearch.model.v20171225.SearchResponse;
    import com.aliyuncs.profile.DefaultProfile;
    import com.aliyuncs.profile.IClientProfile;
    import com.aliyuncs.opensearch.OpenSearchClient;
    import com.aliyuncs.opensearch.model.v20171225.SearchResponse.ResultItem;
    
    public class OpenSearchDemo {
    
        public static void main(String[] args) {
            String accessKeyId = "<your-access-key-id>";
            String accessKeySecret = "<your-access-key-secret>";
            String endpoint = "<your-endpoint>";
            String appName = "<your-app-name>";
    
            IClientProfile profile = DefaultProfile.getProfile("cn-hangzhou", accessKeyId, accessKeySecret);
            DefaultAcsClient client = new DefaultAcsClient(profile);
    
            OpenSearchClient searchClient = new OpenSearchClient(endpoint, appName, accessKeyId, accessKeySecret, client);
        }
    }
    
    //Create a query request.
    //Create a SearchRequest object and specify the search parameters such as the keyword, the number of search results, and the start and end positions of search results.
    
    SearchRequest searchRequest = new SearchRequest();
    searchRequest.setAppName(appName);
    searchRequest.setQuery("query");
    searchRequest.setStart(0);
    searchRequest.setHits(10);
    
    //Send the query request and obtain the response to the request.
    
    try {
        SearchResponse searchResponse = searchClient.search(searchRequest);
        for (ResultItem item : searchResponse.getResult().getItems()) {
            //Handle the search results.
        }
    } catch (ServerException e) {
        e.printStackTrace();
    } catch (ClientException e) {
        e.printStackTrace();
    }

Use temporary credentials provided by STS to access OSS

Prerequisites

The current account is an Alibaba Cloud account or a Resource Access Management (RAM) user that is granted the AliyunRAMFullAccess permission. For more information about how to grant permissions to a RAM user, see Grant permissions to the RAM user.

Scenarios

Assume that a mobile app developer plans to store the user data of an app in Alibaba Cloud Object Storage Service (OSS). The data of each user must be isolated to prevent users from obtaining the data of other users. In this case, you can use Security Token Service (STS) to authorize users to access their own data in your OSS buckets.

The following list describes how to use STS to authorize a user to access OSS:

1. The app user logs on to the app server. The username and password used by the app user for logon are independent from the AccessKey pair of the Alibaba Cloud account used to access OSS. The app server must be able to define the minimum access permissions for each valid app user.
2. The app server sends a request to STS to obtain a security token. Before the app server sends the request to STS, the app server must determine the minimum permissions required by the app user to access OSS and the validity period of the credentials. You can configure a RAM policy to customize the minimum permissions. Then, the app server calls AssumeRole to obtain a security token that indicates a role from STS.
3. STS returns temporary access credentials to the app server. These credentials consist of a security token and a temporary AccessKey pair that can be used to access OSS within a validity period. A temporary AccessKey pair consists of an AccessKey ID and an AccessKey secret.
4. The app server returns the temporary access credentials to the app client. The app client can cache the credentials. After the credentials expire, the app client must apply for new temporary access credentials from the app server. For example, if the temporary access credentials are valid for 1 hour, the app client can send a request to the app server to update the credentials every 30 minutes.
5. The app client uses the locally cached temporary access credentials to initiate a request to call an OSS operation. After OSS receives the request, OSS uses STS to verify the access credentials of the request and then responds to the request.

Step 1: Create a RAM user

1. Log on to the RAM console.
2. In the left-side navigation pane, choose Identities > Users.
3. On the Users page, click Create User.
4. Specify the Logon Name and Display Name parameters.
5. In the Access Mode section, select OpenAPI Access and click OK.
6. Click Copy to save the AccessKey pair of the RAM user.

Step 2: Grant the RAM user the AssumeRole permission

1. On the Users page, click Add Permissions in the Actions column corresponding to the created RAM user.
2. In the Add Permissions panel, select the AliyunSTSAssumeRoleAccess policy from the policy list for System Policy.
3. Click OK.

Step 3: Create a role used to obtain temporary access credentials from STS

1. In the left-side navigation pane, choose Identities > Roles.
2. Click Create Role. In the Create Role panel, set Select Trusted Entity to Alibaba Cloud Account and click Next.
3. Set RAM Role Name to RamOssTest and Select Trusted Alibaba Cloud Account to Current Alibaba Cloud Account.
4. Click OK. After the role is created, click Close.
5. On the Roles page, enter RamOssTest in the search box and click RamOssTest.
6. Click Copy on the right of the RamOssTest page to save the Alibaba Cloud Resource Name (ARN) of the role.

Step 4: Grant the role permissions to upload objects to OSS

1. Create a custom policy to grant the role permissions to upload objects.
2. In the left-side navigation pane, choose Permissions > Policies.
3. On the Policies page, click Create Policy.
4. On the Create Policy page, click JSON. Edit the script in the policy editor to grant the role permissions to upload objects to the exampledir directory in the examplebucket bucket. The following code provides an example on how to grant the role the permissions.

    {
        "Version": "1",
        "Statement": [
         {
               "Effect": "Allow",
               "Action": [
                 "oss:PutObject"
               ],
               "Resource": [
                 "acs:oss:*:*:examplebucket/exampledir",
                 "acs:oss:*:*:examplebucket/exampledir/*"
               ]
         }
        ]
    }

4. Click Next Step.
5. In the Basic Information section, set Policy Name to RamTestPolicy and click OK.
6. Attach the custom policy to the RamOssTest role.
7. In the left-side navigation pane, choose Identities > Roles.
8. On the Roles page, find the RamOssTest role.
9. Click Add Permissions in the Actions column of the RamOssTest role.
10. In the Add Permissions panel, click the Custom Policy tab and select the RamTestPolicy policy.
11. Click OK.

Step 5: Obtain temporary access credentials

You can use STS SDKs to obtain temporary access credentials.

The following code provides an example on how to use STS SDK for Java to obtain temporary access credentials that have the simple upload (oss:PutObject) permission.

    import com.aliyuncs.DefaultAcsClient;
    import com.aliyuncs.exceptions.ClientException;
    import com.aliyuncs.http.MethodType;
    import com.aliyuncs.profile.DefaultProfile;
    import com.aliyuncs.profile.IClientProfile;
    import com.aliyuncs.auth.sts.AssumeRoleRequest;
    import com.aliyuncs.auth.sts.AssumeRoleResponse;
    public class StsServiceSample {
        public static void main(String[] args) { 
            // Specify the endpoint of STS. Example: sts.cn-hangzhou.aliyuncs.com.               
            String endpoint = "<sts-endpoint>";
            // Specify the AccessKey pair that is generated for the RAM user in Step 1. 
            String accessKeyId = "<yourAccessKeyId>";
            String accessKeySecret = "<yourAccessKeySecret>";
            // Specify the role ARN that is obtained in Step 3. 
            String roleArn = "<yourRoleArn>";
            // Specify a custom role session name to distinguish different tokens. Example: SessionTest.
            String roleSessionName = "<yourRoleSessionName>";
            // The following policy allows users to use temporary access credentials to upload objects only to the bucket named examplebucket. 
            // The permissions that are granted to the temporary access credentials are the intersection of the role permissions that are configured in Step 4 and the permissions that are defined in the RAM policy. Users can use the temporary access credentials to upload objects only to the exampledir directory in the examplebucket bucket. 
            // If the policy is empty, the user is granted all permissions of the role. 
            String policy = "{\n" +
                    "    \"Version\": \"1\", \n" +
                    "    \"Statement\": [\n" +
                    "        {\n" +
                    "            \"Action\": [\n" +
                    "                \"oss:PutObject\"\n" +
                    "            ], \n" +
                    "            \"Resource\": [\n" +
                    "                \"acs:oss:*:*:examplebucket/*\" \n" +
                    "            ], \n" +
                    "            \"Effect\": \"Allow\"\n" +
                    "        }\n" +
                    "    ]\n" +
                    "}";
            // Set the validity period of the temporary access credentials to 3600 seconds. 
            Long durationSeconds = 3600L;
            try {
                // regionId specifies the region ID of the RAM user. For example, if the RAM user is located in the China (Hangzhou) region, set regionId to cn-hangzhou. You can also retain the default setting, which is an empty string (""). 
                String regionId = "";
                // Add the endpoint. You can specify this parameter by using STS SDK for Java V3.12.0 or later. 
                DefaultProfile.addEndpoint(regionId, "Sts", endpoint);
                // Add the endpoint. You can specify this parameter by using STS SDK for Java that is earlier than V3.12.0. 
                // DefaultProfile.addEndpoint("",regionId, "Sts", endpoint);
                // Create a default profile.
                IClientProfile profile = DefaultProfile.getProfile(regionId, accessKeyId, accessKeySecret);
                // Use the profile to create a client. 
                DefaultAcsClient client = new DefaultAcsClient(profile);
                final AssumeRoleRequest request = new AssumeRoleRequest();
                // You can specify this parameter by using STS SDK for Java V3.12.0 or later. 
                request.setSysMethod(MethodType.POST);
                // You can specify this parameter by using STS SDK for Java that is earlier than V3.12.0.
                //request.setMethod(MethodType.POST);
                request.setRoleArn(roleArn);
                request.setRoleSessionName(roleSessionName);
                request.setPolicy(policy); 
                request.setDurationSeconds(durationSeconds); 
                final AssumeRoleResponse response = client.getAcsResponse(request);
                System.out.println("Expiration: " + response.getCredentials().getExpiration());
                System.out.println("Access Key Id: " + response.getCredentials().getAccessKeyId());
                System.out.println("Access Key Secret: " + response.getCredentials().getAccessKeySecret());
                System.out.println("Security Token: " + response.getCredentials().getSecurityToken());
                System.out.println("RequestId: " + response.getRequestId());
            } catch (ClientException e) {
                System.out.println("Failed：");
                System.out.println("Error code: " + e.getErrCode());
                System.out.println("Error message: " + e.getErrMsg());
                System.out.println("RequestId: " + e.getRequestId());
            }
        }
    }

Step 6: Use the temporary access credentials to upload objects to OSS

The following code provides an example on how to use OSS SDK for Java V3.12.0 to upload a file named exampletest.txt from D:\localpath to a directory named exampledir in a bucket named examplebucket:

    import com.aliyun.oss.OSSClient;
    import com.aliyun.oss.model.PutObjectRequest;
    
    import java.io.File;
    
    public class Upload {
        public static void main(String[] args) {
    // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
    String endpoint = "oss-cn-hangzhou.aliyuncs.com";
    // Specify the temporary AccessKey ID and AccessKey secret that are included in the temporary access credentials obtained from STS in Step 5. The temporary AccessKey ID and AccessKey secret are not the AccessKey ID and AccessKey secret of an Alibaba Cloud account.
    String accessKeyId = "yourAccessKeyId";
    String accessKeySecret = "yourAccessKeySecret";
    // Specify the security token that is included in the temporary access credentials obtained from STS in Step 5. 
    String securityToken = "yourSecurityToken";
    
    // Create an OSSClient instance. 
    OSS ossClient = new OSSClientBuilder().build(endpoint, accessKeyId, accessKeySecret, securityToken);
    // Upload a local file named exampletest.txt to the directory named exampledir in the bucket named examplebucket. 
    PutObjectRequest putObjectRequest = new PutObjectRequest("examplebucket", "exampledir/exampletest.txt", new File("D:\localpath\exampletest.txt"));
    
    // ObjectMetadata metadata = new ObjectMetadata();
    // Specify the storage class of the object when you upload the file. 
    // metadata.setHeader(OSSHeaders.OSS_STORAGE_CLASS, StorageClass.Standard.toString());
    // Specify the ACL of the object when you upload the file. 
    // metadata.setObjectAcl(CannedAccessControlList.Private);
    // putObjectRequest.setMetadata(metadata);
    
    // Upload the local file. 
    ossClient.putObject(putObjectRequest);
    
    // Shut down the OSSClient instance. 
    ossClient.shutdown();
       }
    }