Bulk export rows by primary key range - Tablestore SDK for Java - Tablestore

Prerequisites

Install the Tablestore SDK for Java and initialize the client.

Feature description

public BulkExportResponse bulkExport(BulkExportRequest bulkExportRequest) throws TableStoreException, ClientException

Scans contiguous rows from a data table by primary key range. The response includes rows whose primary keys are greater than or equal to the start key and less than the end key. Use BulkExportQueryCriteria (in the com.alicloud.openservices.tablestore.model.tunnel subpackage) to set the start and end primary keys, the columns to return, the filter, and the data block encoding type. The response returns row data as ByteBuffer-encoded bytes that you decode into a row list with the matching parser: DBT_PLAIN_BUFFER pairs with PlainBufferBlockParser, and DBT_SIMPLE_ROW_MATRIX pairs with SimpleRowMatrixBlockParser. If getNextStartPrimaryKey() returns a non-null value, more rows remain; continue the scan from that key.

The following example reads from the bulk_export_demo table over the primary key range [row00, row99) in a single call. It uses DBT_PLAIN_BUFFER encoding and decodes the result with the matching parser.

String tableName = "bulk_export_demo";

// Start primary key (inclusive)
PrimaryKey startPk = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn("pk", PrimaryKeyValue.fromString("row00"))
        .build();
// End primary key (exclusive)
PrimaryKey endPk = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn("pk", PrimaryKeyValue.fromString("row99"))
        .build();

// BulkExportQueryCriteria is located in the model.tunnel subpackage
BulkExportQueryCriteria criteria = new BulkExportQueryCriteria(tableName);
criteria.setInclusiveStartPrimaryKey(startPk);
criteria.setExclusiveEndPrimaryKey(endPk);
criteria.setDataBlockType(DataBlockType.DBT_PLAIN_BUFFER);
criteria.addColumnsToGet("pk");
criteria.addColumnsToGet("col1");

BulkExportRequest request = new BulkExportRequest(criteria);
BulkExportResponse response = client.bulkExport(request);

// response.getRows() returns a ByteBuffer; parse it into List<Row> using the corresponding parser
// DBT_PLAIN_BUFFER       -> PlainBufferBlockParser
// DBT_SIMPLE_ROW_MATRIX  -> SimpleRowMatrixBlockParser
PlainBufferBlockParser parser = new PlainBufferBlockParser(response.getRows());
List<Row> rows = parser.getRows();
System.out.println("Rows returned: " + rows.size());
for (Row row : rows) {
    System.out.println(row);
}

// A non-null nextStartPrimaryKey indicates that more data remains and pagination must continue
System.out.println("Has next: " + (response.getNextStartPrimaryKey() != null));

Key parameters:

tableName (required): the name of the data table.
inclusiveStartPrimaryKey / exclusiveEndPrimaryKey (required): the start and end primary keys of the scan range, left-inclusive and right-exclusive.
dataBlockType (optional): the data block encoding of the response, which must match the parser used to decode the result.
columnsToGet / filter (optional): the columns to return and the server-side filter condition.

Parameters

Name	Type	Description
tableName (required)	String	The name of the data table.
inclusiveStartPrimaryKey (required)	PrimaryKey	The start primary key of the scan range. If a row with this key exists, it is included in the response. Accepts a valid primary key or a virtual point built from `INF_MIN` and `INF_MAX`. A virtual point must have the same number of columns as the table primary key. `INF_MIN` denotes negative infinity and compares smaller than any value of any column type; `INF_MAX` denotes positive infinity and compares larger than any value of any column type.
exclusiveEndPrimaryKey (required)	PrimaryKey	The end primary key of the scan range. The row with this key is never included in the response, whether it exists or not. Accepts a valid primary key or a virtual point built from `INF_MIN` and `INF_MAX`. A virtual point must have the same number of columns as the table primary key.
columnsToGet (optional)	Set<String>	The set of column names to return. If not set, all columns of each row are returned. When set, the response includes only the existing columns of each row. If a row contains none of the specified columns, the response excludes that row.
filter (optional)	Filter	A server-side filter applied after column selection. If you specify both `columnsToGet` and `filter`, the server first retrieves the columns named in `columnsToGet`, then applies `filter` to the result.
dataBlockType (optional)	DataBlockType	The data block encoding of the response. Valid values: `DBT_PLAIN_BUFFER` or `DBT_SIMPLE_ROW_MATRIX` (API default: `DBT_SIMPLE_ROW_MATRIX`). The encoding must match the parser used to decode the result.

Scenario examples

Paginate the entire table with nextStartPrimaryKey

A single bulkExport call often returns only part of the range. To scan the entire table, pass the nextStartPrimaryKey from each response as the start primary key of the next call. Stop when nextStartPrimaryKey is null.

String tableName = "bulk_export_demo";

PrimaryKey startPk = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn("pk", PrimaryKeyValue.fromString("row00"))
        .build();
PrimaryKey endPk = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn("pk", PrimaryKeyValue.fromString("row99"))
        .build();

int totalRows = 0;
while (startPk != null) {
    BulkExportQueryCriteria criteria = new BulkExportQueryCriteria(tableName);
    criteria.setInclusiveStartPrimaryKey(startPk);
    criteria.setExclusiveEndPrimaryKey(endPk);
    criteria.setDataBlockType(DataBlockType.DBT_PLAIN_BUFFER);

    BulkExportResponse response = client.bulkExport(new BulkExportRequest(criteria));

    PlainBufferBlockParser parser = new PlainBufferBlockParser(response.getRows());
    List<Row> rows = parser.getRows();
    totalRows += rows.size();

    // A null nextStartPrimaryKey indicates that all data has been read
    startPk = response.getNextStartPrimaryKey();
}
System.out.println("Total rows scanned: " + totalRows);

Prerequisites

Feature description

Parameters

Scenario examples

Paginate the entire table with nextStartPrimaryKey

Related topics