MaxCompute Tunnel is a tunnel service that you can use to upload data to or download data from MaxCompute. This topic provides examples on how to upload data to and download data from MaxCompute by using Tunnel SDK for Python.
- The following sections provide examples on how to upload data to and download data from MaxCompute by using the SDK for Python. For more information about examples in other scenarios, see SDK for Python documentation.
- In a Cython environment, PyODPS compiles C code during installation to accelerate the Tunnel-based data upload and download.
Example of data upload
from odps.tunnel import TableTunnel from odps import ODPS o = ODPS('**your-access-id**', '**your-secret-access-key**', '**your-default-project**', endpoint='**your-end-point**') table = o.get_table('my_table') tunnel = TableTunnel(odps) upload_session = tunnel.create_upload_session(table.name, partition_spec='pt=test') with upload_session.open_record_writer(0) as writer: record = table.new_record() record = 'test1' record = 'id1' writer.write(record) record = table.new_record(['test2', 'id2']) writer.write(record) upload_session.commit()
Example of data download
from odps.tunnel import TableTunnel tunnel = TableTunnel(odps) download_session = tunnel.create_download_session('my_table', partition_spec='pt=test') with download_session.open_record_reader(0, download_session.count) as reader: for record in reader: # Process each record. with download_session.open_arrow_reader(0, download_session.count) as reader: for batch in reader: # Process each Arrow RecordBatch.