List objects in a bucket using Python SDK V1 - Object Storage Service

Object Storage Service (OSS) SDK for Python provides encapsulated iterators to list objects in a bucket. You can use these iterators to filter objects by prefix, starting position, and other criteria.

Usage

OSS SDK for Python provides two iterators for listing objects: ObjectIterator, which is based on the GetBucket(ListObjects) operation, and ObjectIteratorV2, which is based on the ListObjectsV2 (GetBucketV2) operation. The main differences are:

ObjectIterator: Returns the owner information by default.
ObjectIteratorV2: Uses the fetch_ownerparameter to specify whether to return owner information.
Using ObjectIteratorV2 requires OSS SDK for Python 2.12.0 or later.

We recommend ObjectIteratorV2 for its improved support for versioning-enabled buckets.

ObjectIteratorV2 constructor

oss2.ObjectIteratorV2(bucket, prefix='', delimiter='', continuation_token='', start_after='', fetch_owner=False, encoding_type = 'url', max_keys=100, max_retries=None, headers=None)

Click to view the parameter descriptions

Parameter	Type	Required	Description
bucket	oss2.Bucket	Yes	The initialized bucket.
prefix	str	No	Specifies a prefix to filter object names. The default value is an empty string, which returns all objects.
delimiter	str	No	The delimiter used to simulate a folder structure. It is typically set to a forward slash (`/`).
continuation_token	str	No	The pagination token. Pass an empty string for the first request. For subsequent requests, use the `next_continuation_token` value from the previous response.
start_after	str	No	Specifies the starting point for a paginated list operation. The operation returns all objects whose names are lexicographically greater than the specified string. Even if an object with the same name as `start_after` exists in the bucket, it is not included in the result.
fetch_owner	bool	No	Specifies whether to include owner information in the results. Valid values: - `True`: Returns owner information. - `False` (default): Does not return owner information.
max_keys	int	No	The maximum number of objects to return in a single response. The default is 100, and the maximum is 1000.
max_retries	int	No	The maximum number of retries for a failed request.

ObjectIterator constructor

oss2.ObjectIterator(bucket, prefix='', delimiter='', marker='', max_keys=100, max_retries=None, headers=None)

Click to view the parameter descriptions

Parameter	Type	Required	Description
bucket	oss2.Bucket	Yes	The initialized bucket.
prefix	str	No	Specifies a prefix to filter object names. The default value is an empty string, which returns all objects.
delimiter	str	No	The delimiter used to simulate a folder structure. It is typically set to a forward slash (`/`).
marker	str	No	Specifies the starting point for a paginated list operation. The operation returns all objects whose names are lexicographically greater than the specified string. Even if an object with the same name as `marker` exists in the bucket, it is not included in the result.
max_keys	int	No	The maximum number of objects to return in a single response. The default is 100, and the maximum is 1000.
max_retries	int	No	The maximum number of retries for a failed request.

Code examples

Before you run the code, install OSS SDK for Python and configure environment variables for access credentials. For more information, see Quick start with OSS SDK for Python. An Alibaba Cloud account has all permissions by default. If you use a Resource Access Management (RAM) user or a RAM role, make sure that the user or role has the oss:ListObjects permission.

List all objects

ObjectIteratorV2 (Recommended)

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)

    # List all objects in the bucket.
    print("List all objects:")
    for obj in oss2.ObjectIteratorV2(bucket):
        print(f"Object name: {obj.key}, Size: {obj.size} bytes")


if __name__ == "__main__":
    main()

To retrieve the owner information for the objects, set fetch_owner=True.

# List all objects and get their owner information.
for obj in oss2.ObjectIteratorV2(bucket, fetch_owner=True):
    print(f"Object name: {obj.key}")
    print(f"Owner name: {obj.owner.display_name}")
    print(f"Owner ID: {obj.owner.id}")

ObjectIterator

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)

    # List all objects in the bucket.
    print("List all objects:")
    for obj in oss2.ObjectIterator(bucket):
        print(f"Object name: {obj.key}, Size: {obj.size} bytes")


if __name__ == "__main__":
    main()

List a specific number of objects

ObjectIteratorV2 (Recommended)

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from itertools import islice

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)

    # List the first 10 objects in the bucket.
    print("List the first 10 objects:")
    for obj in islice(oss2.ObjectIteratorV2(bucket), 10):
        print(f"Object name: {obj.key}")


if __name__ == "__main__":
    main()

ObjectIterator

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from itertools import islice

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)

    # List the first 10 objects in the bucket.
    print("List the first 10 objects:")
    for obj in islice(oss2.ObjectIterator(bucket), 10):
        print(f"Object name: {obj.key}")


if __name__ == "__main__":
    main()

List objects with a specific prefix

Assume that a bucket contains four objects: oss.jpg, fun/test.jpg, fun/movie/001.avi, and fun/movie/007.avi. The forward slash (/) is used as a folder separator.

ObjectIteratorV2 (Recommended)

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
    
    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)
    
    # List all objects in the fun folder, including objects in its subdirectories.
    print("List all objects with the prefix fun/:")
    for obj in oss2.ObjectIteratorV2(bucket, prefix='fun/'):
        print(f"Object name: {obj.key}")

if __name__ == "__main__":
    main()

Expected output:

List all objects with the prefix fun/:
Object name: fun/
Object name: fun/movie/
Object name: fun/movie/001.avi
Object name: fun/movie/007.avi
Object name: fun/test.jpg

ObjectIterator

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
    
    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)
    
    # List all objects in the fun folder, including objects in its subdirectories.
    print("List all objects with the prefix fun/:")
    for obj in oss2.ObjectIterator(bucket, prefix='fun/'):
        print(f"Object name: {obj.key}")

if __name__ == "__main__":
    main()

Expected output:

List all objects with the prefix fun/:
Object name: fun/
Object name: fun/movie/
Object name: fun/movie/001.avi
Object name: fun/movie/007.avi
Object name: fun/test.jpg

List objects and subdirectories in a specific directory

OSS does not have the concept of folders. All elements are stored as objects. To create a folder, you can create a zero-byte object that ends with a forward slash (/). The OSS console displays objects that end with a forward slash (/) as folders.

You can use the delimiter and prefix parameters to simulate folder functionality.

If you set the prefix parameter to a folder name, the operation lists all objects that start with this prefix. This includes all objects and subfolders within that folder, and all objects within the subfolders.
If you also set the delimiter parameter to a forward slash (/), the operation lists only the objects and subfolder names in that folder. Objects within the subfolders are not listed.

ObjectIteratorV2 (Recommended)

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
    
    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)
    
    # List the objects and subfolder names in the fun folder, but not the objects within the subfolders.
    print("List objects and subdirectories in the fun directory:")
    for obj in oss2.ObjectIteratorV2(bucket, prefix='fun/', delimiter='/', start_after='fun/'):
        # Use the is_prefix method to determine whether the obj is a folder.
        if obj.is_prefix():
            print(f"Subdirectory: {obj.key}")
        else:
            print(f"Object: {obj.key}")

if __name__ == "__main__":
    main()

Expected output:

List objects and subdirectories in the fun directory:
Subdirectory: fun/movie/
Object: fun/test.jpg

ObjectIterator

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
    
    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)
    
    # List the objects and subfolder names in the fun folder, but not the objects within the subfolders.
    print("List objects and subdirectories in the fun directory:")
    for obj in oss2.ObjectIterator(bucket, prefix='fun/', delimiter='/', marker='fun/'):
        # Use the is_prefix method to determine whether the obj is a folder.
        if obj.is_prefix():
            print(f"Subdirectory: {obj.key}")
        else:
            print(f"Object: {obj.key}")

if __name__ == "__main__":
    main()

Expected output:

List objects and subdirectories in the fun directory:
Subdirectory: fun/movie/
Object: fun/test.jpg

List objects after a specific start position

Assume that a bucket contains the following four objects: x1.txt, x2.txt, z1.txt, and z2.txt.

ObjectIteratorV2 (Recommended)

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
    
    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)
    
    # List all objects after x2.txt (excluding x2.txt itself).
    print("List all objects after x2.txt:")
    for obj in oss2.ObjectIteratorV2(bucket, start_after="x2.txt"):
        print(f"Object name: {obj.key}")

if __name__ == "__main__":
    main()

Expected output:

List all objects after x2.txt:
Object name: z1.txt
Object name: z2.txt

ObjectIterator

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
    
    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)
    
    # List all objects after x2.txt (excluding x2.txt itself).
    print("List all objects after x2.txt:")
    for obj in oss2.ObjectIterator(bucket, marker="x2.txt"):
        print(f"Object name: {obj.key}")

if __name__ == "__main__":
    main()

Expected output:

List all objects after x2.txt:
Object name: z1.txt
Object name: z2.txt

Get the size of objects in a specific directory

ObjectIteratorV2 (Recommended)

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def calculate_folder_size(bucket, folder):
    """Calculate the total size of a specified folder."""
    total_size = 0
    for obj in oss2.ObjectIteratorV2(bucket, prefix=folder):
        total_size += obj.size
    return total_size

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
    
    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)
    
    # List all objects and folders in the root directory and display their sizes.
    print("List the sizes of objects and folders in the root directory:")
    for obj in oss2.ObjectIteratorV2(bucket, delimiter='/'):
        if obj.is_prefix():
            # Calculate the total size of the folder.
            folder_size = calculate_folder_size(bucket, obj.key)
            print(f"Directory: {obj.key}, Size: {folder_size} bytes")
        else:
            # Display the object size directly.
            print(f"Object: {obj.key}, Size: {obj.size} bytes")

if __name__ == "__main__":
    main()

ObjectIterator

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

def calculate_folder_size(bucket, folder):
    """Calculate the total size of a specified folder."""
    total_size = 0
    for obj in oss2.ObjectIterator(bucket, prefix=folder):
        total_size += obj.size
    return total_size

def main():
    # Get access credentials from environment variables.
    auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
    
    # Set the endpoint and region where the bucket is located.
    endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
    region = "cn-hangzhou"
    # Initialize the bucket.
    bucket = oss2.Bucket(auth, endpoint, "example-bucket", region=region)
    
    # List all objects and folders in the root directory and display their sizes.
    print("List the sizes of objects and folders in the root directory:")
    for obj in oss2.ObjectIterator(bucket, delimiter='/'):
        if obj.is_prefix():
            # Calculate the total size of the folder.
            folder_size = calculate_folder_size(bucket, obj.key)
            print(f"Directory: {obj.key}, Size: {folder_size} bytes")
        else:
            # Display the object size directly.
            print(f"Object: {obj.key}, Size: {obj.size} bytes")

if __name__ == "__main__":
    main()