OpenSearch:Demo code for implementing iterative scroll queries

Prerequisites

Before you begin, ensure that you have:

• An OpenSearch application with at least one table

• The endpoint of the OpenSearch API in your region

• An AccessKey pair (AccessKey ID and AccessKey secret) for authentication. Use a Resource Access Management (RAM) user rather than the root account. See Create a RAM user and Create an AccessKey pair

• Required permissions granted to the AliyunServiceRoleForOpenSearch role. See AliyunServiceRoleForOpenSearch and Access authorization rules

Set environment variables

Set ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET before running the sample code.

• Linux and macOS — Run the following commands. Replace <access_key_id> and <access_key_secret> with the AccessKey ID and AccessKey secret of your RAM user.

  export ALIBABA_CLOUD_ACCESS_KEY_ID=<access_key_id>
  export ALIBABA_CLOUD_ACCESS_KEY_SECRET=<access_key_secret>

• Windows — Create an environment variable file, add the two variables with their values, then restart Windows for the changes to take effect.

Implement iterative scroll queries

The following example uses DeepPageingIterator to scroll through all results for the query name:'opensearch', sorted by id in descending order. Each page returns 5 documents in FULLJSON format.

package com.aliyun.opensearch;

import com.aliyun.opensearch.sdk.dependencies.com.google.common.collect.Lists;
import com.aliyun.opensearch.sdk.generated.OpenSearch;
import com.aliyun.opensearch.sdk.generated.search.*;
import com.aliyun.opensearch.search.DeepPageingIterator;
import java.nio.charset.Charset;

public class testScrollIterator {

    // Replace these values with your actual OpenSearch application details.
    private static String appName = "<your-app-name>";
    private static String tableName = "<your-table-name>";
    private static String host = "<your-opensearch-api-endpoint>";

    public static void main(String[] args) {

        // Read credentials from environment variables — never hardcode them.
        String accesskey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String secret = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");

        // Log the file encoding for debugging purposes.
        System.out.println(String.format("file.encoding: %s", System.getProperty("file.encoding")));
        System.out.println(String.format("defaultCharset: %s", Charset.defaultCharset().name()));

        // Initialize the client chain: OpenSearch → OpenSearchClient → SearcherClient.
        OpenSearch openSearch = new OpenSearch(accesskey, secret, host);
        OpenSearchClient serviceClient = new OpenSearchClient(openSearch);
        SearcherClient searcherClient = new SearcherClient(serviceClient);

        // Configure the config clause: application name, page size, response format, and fetch fields.
        Config config = new Config(Lists.newArrayList(appName));
        config.setStart(0);       // Has no effect in scroll queries; the default value 0 is always used.
        config.setHits(5);        // Number of documents per page.
        config.setSearchFormat(SearchFormat.FULLJSON);  // Supported formats: JSON and FULLJSON.
        config.setFetchFields(Lists.newArrayList("id", "name", "phone", "int_arr", "literal_arr", "float_arr", "cate_id"));

        // Build the search query. Specifying multiple index fields in separate setQuery calls
        // overwrites the previous query — include all index fields in a single setQuery call.
        SearchParams searchParams = new SearchParams(config);
        searchParams.setQuery("name:'opensearch'");

        // Optional: add a filter condition.
        // searchParams.setFilter("cate_id<=3");

        // Scroll queries support sorting on a single field only.
        Sort sorter = new Sort();
        sorter.addToSortFields(new SortField("id", Order.DECREASE));
        searchParams.setSort(sorter);

        // Enable deep paging.
        DeepPaging deep = new DeepPaging();
        searchParams.setDeepPaging(deep);

        // DeepPageingIterator manages scroll IDs automatically — no manual tracking needed.
        DeepPageingIterator pagesIterator = new DeepPageingIterator(searcherClient, searchParams);

        // Optional: set a custom iteration interval in milliseconds (default: 100 ms).
        // Increase this value to reduce pressure on the OpenSearch service.
        // pagesIterator.setPagingIntervals(80);

        // Check for errors using the error code and message, not the status field.
        try {
            System.out.println("Starting iterative scroll query...");
            while (pagesIterator.hasNext()) {
                System.out.println(pagesIterator.next());
            }
        } catch (Exception ex) {
            System.out.println("Error: " + ex.getMessage());
        }
    }
}

What's next

• Error codes — Look up error codes returned during scroll queries.

• AliyunServiceRoleForOpenSearch — Review the service role and its required permissions.

• Access authorization rules — Configure permissions for RAM users.