Data upload by using BufferedWriter - MaxCompute - Alibaba Cloud Documentation Center

Use the BufferedWriter interface in the MaxCompute Tunnel SDK to write records from a Java application directly to a MaxCompute table. This approach buffers records in memory and flushes them when you close the writer, reducing the number of network round trips compared to writing one record at a time.

Prerequisites

Before you begin, make sure you have:

MaxCompute and MaxCompute Tunnel initialized in your Java project
An upload session created for the target table
The target table schema (column names and data types)

How it works

The upload flow follows these steps:

Create an upload session with tunnel.createUploadSession(projectName, tableName).
Open a BufferedWriter with uploadSession.openBufferedWriter().
Create a Record object using uploadSession.newRecord().
Set field values on the record and call writer.write(record) for each row.
Close the writer in a finally block to flush buffered data to MaxCompute.
Commit the session with uploadSession.commit() to finalize the upload.

Upload data

The following example assumes that MaxCompute and MaxCompute Tunnel are already initialized. Replace items, name, and id with your actual data and column definitions.

// Initialize MaxCompute and MaxCompute Tunnel before running this example.
RecordWriter writer = null;
TableTunnel.UploadSession uploadSession = tunnel.createUploadSession(projectName, tableName);
try {
  int i = 0;
  // Open a BufferedWriter for the upload session.
  writer = uploadSession.openBufferedWriter();
  Record product = uploadSession.newRecord();
  for (String item : items) {
    product.setString("name", item);
    product.setBigint("id", i);
    // Write each record to the buffer.
    writer.write(product);
    i += 1;
  }
} finally {
  if (writer != null) {
    // Close the writer to flush buffered data.
    writer.close();
  }
}
// Commit the session to complete the upload.
uploadSession.commit();

Parameters

Variable	Type	Description
`items`	String array	The data to upload. Define as a STRING array. Example: `String[] arr = {"s1", "s2"}`
`name`	STRING column	The column to write STRING data to. Specify the column name and data type based on your business requirements.
`id`	BIGINT column	The column to write BIGINT data to. Specify the column name and data type based on your business requirements.

Setter methods and column types

Each MaxCompute column type has a corresponding setter method on the Record object. The following table lists the methods used in this example:

Setter method	Column type	Example value
`setString(columnName, value)`	STRING	`"example"`
`setBigint(columnName, value)`	BIGINT	`42L`

For a complete list of setter methods and supported column types, see the MaxCompute Tunnel SDK reference.

What's next

For the full initialization code and a complete runnable example, see the Tunnel SDK documentation.
To upload data using other methods, see the data upload and download documentation.