Bulk import offline data - Tablestore SDK for Java - Tablestore

Prerequisites

Install the Tablestore SDK for Java and initialize the client.

Description

Call bulkImport to submit multiple row operations in a single request. The server processes each row independently and returns per-row results, so a failure on one row does not affect operations on other rows. This method is intended for offline bulk imports in big-data scenarios.

public BulkImportResponse bulkImport(BulkImportRequest bulkImportRequest) throws TableStoreException, ClientException

The following example inserts five rows into the bulk_import_demo table in a single bulk request and prints the count of succeeded and failed rows.

String tableName = "bulk_import_demo";

BulkImportRequest request = new BulkImportRequest(tableName);

List<RowChange> rowChanges = new ArrayList<RowChange>();
for (int i = 0; i < 5; i++) {
    PrimaryKey pk = PrimaryKeyBuilder.createPrimaryKeyBuilder()
            .addPrimaryKeyColumn("pk", PrimaryKeyValue.fromString("row" + i))
            .build();
    RowPutChange put = new RowPutChange(tableName, pk);
    put.addColumn(new Column("col1", ColumnValue.fromString("v" + i)));
    rowChanges.add(put);
}
request.addRowChanges(rowChanges);

BulkImportResponse response = client.bulkImport(request);

// Retrieve per-row results into succeedRows and failedRows
List<BulkImportResponse.RowResult> succeedRows = new ArrayList<BulkImportResponse.RowResult>();
List<BulkImportResponse.RowResult> failedRows = new ArrayList<BulkImportResponse.RowResult>();
response.getResult(succeedRows, failedRows);

System.out.println("All succeed: " + response.isAllSucceed());
System.out.println("Succeed: " + succeedRows.size() + ", Failed: " + failedRows.size());

Parameters

Name

Type

Description

tableName (required)

String

The name of the table.

rowChanges (required)

List<RowChange>

The list of row operations. You can mix the following operations in a single request:

RowPutChange: Writes a row. For configuration details, see Write a row.
RowUpdateChange: Updates a row. For configuration details, see Update a row.
RowDeleteChange: Deletes a row. For configuration details, see Delete a row.

Response

Import result

BulkImportResponse contains the following operation-specific field.

Name	Type	Description
rowResults	List<BulkImportResponse.RowResult>	The result of each row operation in the same order as `rowChanges` in the request. Call `getRowResults()` to obtain the list.

Row result

Each element in BulkImportResponse.rowResults[] is of the BulkImportResponse.RowResult type.

Name	Type	Description
succeed	boolean	Indicates whether the row operation succeeded. Call `isSucceed()` to obtain the value.
error	Error	The error for a failed row operation. The value is `null` when the operation succeeds.
index	int	The index of the row operation in `rowChanges` of the request.

Examples

Combine put, update, and delete in a single request

Use one BulkImportRequest to submit put, update, and delete operations together.

String tableName = "bulk_import_demo";

BulkImportRequest request = new BulkImportRequest(tableName);

// Insert a new row
PrimaryKey pkPut = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn("pk", PrimaryKeyValue.fromString("mixed_put"))
        .build();
RowPutChange put = new RowPutChange(tableName, pkPut);
put.addColumn(new Column("col1", ColumnValue.fromString("put_value")));
request.addRowChange(put);

// Update an existing row (append a column)
PrimaryKey pkUpdate = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn("pk", PrimaryKeyValue.fromString("row0"))
        .build();
RowUpdateChange update = new RowUpdateChange(tableName, pkUpdate);
update.put(new Column("col2", ColumnValue.fromLong(100)));
request.addRowChange(update);

// Delete an existing row
PrimaryKey pkDelete = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn("pk", PrimaryKeyValue.fromString("row1"))
        .build();
RowDeleteChange delete = new RowDeleteChange(tableName, pkDelete);
request.addRowChange(delete);

BulkImportResponse response = client.bulkImport(request);
System.out.println("Mixed all succeed: " + response.isAllSucceed());

Retry failed rows

Use createRequestForRetry(failedRows) to build a new request from the failed rows. The retry contains only the failed rows, so rows that already succeeded are not re-applied.

String tableName = "bulk_import_demo";

BulkImportRequest request = new BulkImportRequest(tableName);

PrimaryKey pk = PrimaryKeyBuilder.createPrimaryKeyBuilder()
        .addPrimaryKeyColumn("pk", PrimaryKeyValue.fromString("good"))
        .build();
RowPutChange row = new RowPutChange(tableName, pk);
row.addColumn(new Column("col1", ColumnValue.fromString("ok")));
request.addRowChange(row);

BulkImportResponse response = client.bulkImport(request);

List<BulkImportResponse.RowResult> succeedRows = new ArrayList<BulkImportResponse.RowResult>();
List<BulkImportResponse.RowResult> failedRows = new ArrayList<BulkImportResponse.RowResult>();
response.getResult(succeedRows, failedRows);

// If any rows failed, build a retry request that contains only the failed rows
if (!failedRows.isEmpty()) {
    BulkImportRequest retryRequest = request.createRequestForRetry(failedRows);
    BulkImportResponse retryResponse = client.bulkImport(retryRequest);
    System.out.println("Retry all succeed: " + retryResponse.isAllSucceed());
}