PyODPS resource operations - MaxCompute - MaxCompute - Alibaba Cloud Documentation Center

Quick start

Create a file resource and read its content:

# Create a resource from a string
resource = o.create_resource('test_py_resource.py', 'py', file_obj='import this')

# Read the resource
with resource.open('r') as fp:
    content = fp.read()
    print(content)

Method reference

Resource operations available through the MaxCompute entry object (o).

Method	Description	Return type
`o.list_resources()`	List all resources in a project	Iterator
`o.exist_resource(name)`	Check whether a resource exists	`bool`
`o.create_resource(name, type, ...)`	Create a resource	`Resource`
`o.open_resource(name, mode)`	Open a resource for reading or writing	File-like object
`o.get_resource(name)`	Get a resource object by name	`Resource`
`o.delete_resource(name)`	Delete a resource	`None`
`resource.drop()`	Delete a resource (alias)	`None`

File resources

File resources include plain files, .py files, .jar files, and archives.

Create a file resource

Call create_resource() with the resource name, file type, and either a file-like object or a string.

# Create from a file on disk
# Open compressed packages and binary files in binary mode ('rb')
resource = o.create_resource('test_file_resource', 'file', file_obj=open('/to/path/file', 'rb'))

# Create from a string
resource = o.create_resource('test_py_resource.py', 'py', file_obj='import this')

Use binary mode ('rb') for compressed packages and binary files. Text files and Python scripts accept string input directly. For .py, .jar, and archive resources, the name must include the file extension.

Read and modify a file resource

Open a file resource using one of two approaches:

Call resource.open(mode) on a resource object.
Call o.open_resource(name, mode) on the MaxCompute entry object.

Both return a file-like object with the same interface as Python's built-in open().

Read a resource:

with resource.open('r') as fp:
    content = fp.read()       # Read all content
    fp.seek(0)                # Return to the beginning
    lines = fp.readlines()    # Read multiple lines

Read and write a resource:

with o.open_resource('test_file_resource', mode='r+') as fp:
    fp.read()
    fp.tell()                              # Get the current position
    fp.seek(10)
    fp.truncate()                          # Truncate at the current position
    fp.writelines(['Hello\n', 'World\n'])  # Write multiple lines
    fp.write('Hello World')
    fp.flush()                             # Submit the update to MaxCompute

Open modes

Text modes:

Mode	Description
`r`	Read-only. Cannot write.
`w`	Write-only. Clears existing content.
`a`	Append. Writes to the end of the file.
`r+`	Read and write.
`w+`	Read and write. Clears existing content.
`a+`	Read and append.

Binary modes (for binary data):

Mode	Description
`rb`	Binary read-only.
`r+b`	Binary read and write.

File-like object methods

Method	Description
`read()`	Read all content
`readlines()`	Read all lines into a list
`write(data)`	Write a string
`writelines(lines)`	Write a list of strings
`seek(pos)`	Move to a specific position
`tell()`	Get the current position
`truncate()`	Truncate at the current position. Pass a length to truncate to a specific size.
`flush()`	Submit the update to MaxCompute immediately

Table resources

Table resources reference an existing MaxCompute table, and optionally a specific partition.

Create a table resource

o.create_resource('test_table_resource', 'table', table_name='my_table', partition='pt=test')

Update a table resource

Change the referenced partition or project:

table_resource = o.get_resource('test_table_resource')
table_resource.update(partition='pt=test2', project_name='my_project2')

Get table and partition metadata

table_resource = o.get_resource('test_table_resource')

table = table_resource.table
print(table.name)            # Table name

partition = table_resource.partition
print(partition.spec)        # Partition spec, e.g. 'pt=test'

Read and write table data

Write records to a table resource, then read them back:

table_resource = o.get_resource('test_table_resource')

# Write records
with table_resource.open_writer() as writer:
    writer.write([0, 'aaaa'])
    writer.write([1, 'bbbbb'])

# Read records
with table_resource.open_reader() as reader:
    for rec in reader:
        print(rec)