Perform scroll searches to export large sets of documents - OpenSearch - Alibaba Cloud - OpenSearch

Regular searches return a maximum of 5,000 documents. If you need more results for analysis, use scroll searches to retrieve larger result sets.

Parameters

Request parameters

Parameter	Type	Required	Valid value	Default value	Description
scroll	STRING	Yes	You can set a value in units of weeks, days, hours, minutes, or seconds.		The validity period of the scroll ID to be returned for the next execution of the scroll search, in units of weeks (w), days (d), hours (h), minutes (m), or seconds (s). Example: 1m.
search_type	STRING	Yes	scan		The type of the scroll search. Specify this parameter only for the first scroll search. For subsequent searches, use the scroll_id parameter instead.
scroll_id	STRING	Yes			The ID returned by the scroll search. The first scroll search returns only a scroll ID without results. Use this ID to execute the scroll search again to obtain results. For subsequent searches, this ID is both a required request parameter and a returned response parameter.
fetch_fields	STRING				The fields to be returned in search results.

Response parameters

Parameter	Type	Description
status	STRING	The request execution result. Valid values: OK and FAIL. If the value is FAIL, troubleshoot errors based on the error code.
request_id	STRING	The request ID, used for troubleshooting.
result	STRING	The search results, including the searchtime, total, num, viewtotal, items, facet, and scroll_id parameters.
errors	STRING	The error information. The error_message parameter indicates the error message. For more information, see Error codes.

Note

Return results of scroll searches support only the fullJSON and JSON formats.

Sample responses

Response for the first execution of a scroll search:

{
    "status": "OK",
    "request_id": "150150574119953661605242",
    "result": {
        "searchtime": 0.005029,
        "total": 1,
        "num": 0,
        "viewtotal": 1,
        "scroll_id": "eJxtUMtuhDAM/BrvOYQC5cABdulvRFFIirsm2TpBavv1Ndut1EMlS36NZ0Y2ZHMxbueceAjIuWCMnrPjRITLyfzZm83y9V QVGT8x80U3PxQNUqieVZV1/an4ItbTUBPSx5wgXqKdvOSbmuKR8ZYjGWWirB4tvToAiX7u3G2eCNK77vnz8GlGPAV6suKBeqxAn0OiTd7NGEnesspyoyFLF6hecn4JUKjVgp0K3FnkfMfIyPoDuYWegX9GeYOpicY9TG8gwOSuBL04X1 MMg3ROwCesLlG6X7a2o=",
        "items": [],
        "facet": []
    },
    "errors": [],
    "tracer": ""
}

Response for a subsequent execution of the scroll search:

{
    "status": "OK",
    "request_id": "150150574119952551519970",
    "result": {
        "searchtime": 0.006293,
        "total": 1,
        "num": 1,
        "viewtotal": 1,
        "scroll_id": "eJxNT9tugzAM/RrznIRC4YEHaNlvRFFIhteQtE6Qtn39TNdJk2z5dnx8rIPJRdudcqKhl60Uir2Vp06ISv8b6s3QbZCVzpaCdp93XXBzg2wEW9MJ2dWq8q7YVXt0YckDLlBP0WyOw31N8YgYizZEnAUsjkx4VT4k8zexpjiNS/XYHX0NNkWP71BfVyxQjxLUxSfazFH4PYSPnCL3iMniDZq3jN98aFRCgGrZniy8/itkBHWGuYVeQH+B+QzTCUZ1NJ9gj4FVMfrQPr8Y+Hk+dgU14fIDVhtfTw==",
        "items": [
            {
                "fields": {
                    "cate_id": "0",
                    "float_arr": "0",
                    "id": "1",
                    "int_arr": "0",
                    "literal_arr": "Search",
                    "name": "Search",
                    "phone": "1381111****",
                    "index_name": "app_schema_demo"
                },
                "property": {},
                "attribute": {},
                "variableValue": {},
                "sortExprValues": [
                    "1"
                ]
            }
        ],
        "facet": []
    },
    "errors": [],
    "tracer": ""
}

Usage notes

Scroll searches support sorting based on a single field of the INT type. The OpenSearch API and OpenSearch SDKs version must be V3.
Scroll searches retrieve all matched results and do not support the aggregate and distinct clauses, rank clauses, and query analysis.
The start parameter in the config clause does not take effect in scroll searches. The default value 0 is used, meaning you cannot skip pages. Each result set can contain a maximum of 500 documents.
Specify the hit parameter in the first scroll search to set the number of documents per result set. Changes to the hit parameter in subsequent scroll searches do not take effect.
The first scroll search returns only a scroll ID. Use this ID to run the scroll search again to obtain document data.
To determine whether an error occurred, check the error code and message rather than the status information. For more information, see Error codes.
If a scroll search returns an error message indicating expiration, the scroll request validity period has expired. Modify the scroll parameter.

SDK demo code

Note

Note:

1. The start parameter in the config clause does not take effect in scroll searches. Use the hit parameter in the config clause to specify the number of documents per result set.

2. Scroll searches do not support the aggregate, distinct, and rank clauses. Scroll searches support sorting based on a single field of the INT type.

3. Scroll searches across applications are not supported.

4. If the value of the scroll_id parameter in a request is invalid, an error occurs.

5. Return results of scroll searches support only the fullJSON and JSON formats.

6. The first scroll search returns only a scroll ID. Use this ID to execute the scroll search again to obtain document data.