Use Tablestore SDK for .NET to perform a parallel scan - Tablestore

If you do not have requirements on the order of query results, you can use the parallel scan feature to obtain query results in an efficient manner.

Prerequisites

An OTSClient instance is initialized. For more information, see Initialize an OTSClient instance.
A data table is created and data is written to the data table. For more information, see Create data tables and Write data.
A search index is created for the data table. For more information, see Create search indexes.

Parameters

A parallel scan requires two coordinating operations: call ComputeSplits to get the session ID and the maximum number of parallel tasks (MaxParallel), then launch one ParallelScan task per parallel slot — each with a unique CurrentParallelId ranging from 0 to MaxParallel - 1. For example, if MaxParallel is 4, start four scan tasks with CurrentParallelId values of 0, 1, 2, and 3 to cover the complete dataset.

Parameter		Description
TableName		The name of the data table.
IndexName		The name of the search index.
ScanQuery	Query	The query condition. Supported types: term query, fuzzy query, range query, geo query, and nested query — the same types supported by the Search operation.
	Limit	The maximum number of rows returned per ParallelScan call.
	MaxParallel	The maximum number of parallel scan tasks per request. The maximum value depends on data volume — larger datasets support more parallel tasks. Call ComputeSplits to get this value before starting a scan. Each ParallelScan request uses one CurrentParallelId in the range [0, MaxParallel), and all IDs in that range must be covered to scan the complete dataset. For example, if MaxParallel is 4, start four tasks with CurrentParallelId values of 0, 1, 2, and 3. MaxParallel and CurrentParallelId must be used together.
	CurrentParallelId	The ID of this parallel scan task. Valid values: [0, MaxParallel). Each concurrent task must use a unique ID. Must be used together with MaxParallel.
	Token	The pagination token for fetching the next page of results. Each ParallelScan response includes a token for the next page. Pass it in the subsequent request to continue reading. When the token is null, all data for this task has been retrieved.
	AliveTime	The validity period of the session and its associated token, in seconds. Default value: 60. Use the default value. If no request is sent within the validity period, the session expires and no further data can be retrieved. The validity period resets with each request. Note Sessions can expire early if the search index schema is dynamically modified, a single server fails, or server load balancing is performed. In these cases, recreate the session.
ColumnsToGet		The columns to return. Set the Columns parameter to specify column names. To return all columns in the search index, set ReturnAllFromIndex to true. Important The ReturnAll parameter is not supported.
SessionId		The session ID for the parallel scan. Call ComputeSplits to create a session and get both the session ID and the maximum number of supported parallel tasks (MaxParallel).

Example

The following example scans all data using a single thread. It calls ComputeSplits to get the session ID and MaxParallel, then pages through results using NextToken until all rows are retrieved.

/// <summary>
/// Scans all data in a search index using a single thread.
/// Calls ComputeSplits to get the session ID and MaxParallel,
/// then pages through results until NextToken is null.
/// </summary>
public class ParallelScan
{
    public static void ParallelScanwithSingleThread(OTSClient otsClient)
    {
        SearchIndexSplitsOptions options = new SearchIndexSplitsOptions
        {
            IndexName = IndexName
        };

        ComputeSplitsRequest computeSplitsRequest = new ComputeSplitsRequest
        {
            TableName = TableName,
            SplitOptions = options
        };

        ComputeSplitsResponse computeSplitsResponse = otsClient.ComputeSplits(computeSplitsRequest);

        MatchAllQuery matchAllQuery = new MatchAllQuery();

        ScanQuery scanQuery = new ScanQuery();
        scanQuery.AliveTime = 60;
        scanQuery.Query = matchAllQuery;
        scanQuery.MaxParallel = computeSplitsResponse.SplitsSize;
        scanQuery.Limit = 10;

        ParallelScanRequest parallelScanRequest = new ParallelScanRequest();
        parallelScanRequest.TableName = TableName;
        parallelScanRequest.IndexName = IndexName;
        parallelScanRequest.ScanQuery = scanQuery;
        parallelScanRequest.ColumnToGet = new ColumnsToGet { ReturnAllFromIndex = true };
        parallelScanRequest.SessionId = computeSplitsResponse.SessionId;

        int total = 0;
        List<Row> result = new List<Row>();

        ParallelScanResponse parallelScanResponse = otsClient.ParallelScan(parallelScanRequest);

        while (parallelScanResponse.NextToken != null)
        {
            List<Row> rows = new List<Row>(parallelScanResponse.Rows);

            total += rows.Count;
            result.AddRange(rows);

            parallelScanRequest.ScanQuery.Token = parallelScanResponse.NextToken;

            parallelScanResponse = otsClient.ParallelScan(parallelScanRequest);
        }

        foreach (Row row in result)
        {
            Console.WriteLine(JsonConvert.SerializeObject(row));
        }
        Console.WriteLine("Total Row Count: {0}", total);
    }
}