Use MetaSearch with OSS SDK for Python - Object Storage Service

Scalar search lets you query objects in a bucket by metadata conditions — filtering by size, storage class, tags, or custom attributes — without iterating through the entire object list. OSS builds and maintains a metadata index for the bucket, so queries return results quickly even across millions of objects.

Prerequisites

Before you begin, make sure you have:

An OSS bucket
The alibabacloud_oss_v2 Python SDK installed
AccessKey credentials stored in environment variables (loaded via EnvironmentVariableCredentialsProvider)

Usage notes

The sample code uses cn-hangzhou as the region and a public endpoint by default. To access OSS from another Alibaba Cloud service in the same region, use an internal endpoint instead. For a full list of regions and endpoints, see Regions and endpoints.

Set up the client

All examples use the same client initialization pattern. Initialize it once and reuse it across all operations below.

import alibabacloud_oss_v2 as oss

credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = "<region-id>"          # e.g., cn-hangzhou
# cfg.endpoint = "<custom-endpoint>" # Uncomment to use an internal or custom endpoint

client = oss.Client(cfg)

Replace <region-id> with the region where your bucket is located.

Sample code

Enable metadata management

When you enable metadata management for a bucket, OSS creates a metadata index library and indexes all existing objects. After setup, OSS performs near-real-time incremental scans on new objects as they are uploaded.

result = client.open_meta_query(oss.OpenMetaQueryRequest(
    bucket="<bucket-name>",
))

print(f"status code: {result.status_code}, request id: {result.request_id}")

For the complete runnable sample, see open_meta_query.py.

Get metadata index status

Use get_meta_query_status to check whether the metadata index library is ready and retrieve its current state.

result = client.get_meta_query_status(oss.GetMetaQueryStatusRequest(
    bucket="<bucket-name>",
))

print(f"status code: {result.status_code}")
print(f"request id:  {result.request_id}")
print(f"create time: {result.meta_query_status.create_time}")
print(f"update time: {result.meta_query_status.update_time}")
print(f"state:       {result.meta_query_status.state}")
print(f"phase:       {result.meta_query_status.phase}")

For the complete runnable sample, see get_meta_query_status.py.

Query objects by condition

Use do_meta_query to filter objects by a query condition, sort the results, and run aggregations. The example below finds all objects larger than 1 MiB, sorted by size in descending order, and computes the total and maximum size.

Query condition format

The query parameter takes a JSON string with three fields:

Field	Description	Example
`Field`	Metadata attribute to filter on	`"Size"`
`Value`	Comparison value	`"1048576"`
`Operation`	Comparison operator	`"gt"`

For supported fields and operators, see the scalar search documentation.

result = client.do_meta_query(oss.DoMetaQueryRequest(
    bucket="<bucket-name>",
    meta_query=oss.MetaQuery(
        query='{"Field": "Size", "Value": "1048576", "Operation": "gt"}',
        sort="Size",
        order=oss.MetaQueryOrderType.DESC,
        max_results=80369,
        next_token="",          # Pass the token from the previous response to paginate
        aggregations=oss.MetaQueryAggregations(
            aggregations=[
                oss.MetaQueryAggregation(field="Size", operation="sum"),
                oss.MetaQueryAggregation(field="Size", operation="max"),
            ],
        ),
    ),
))

print(f"status code: {result.status_code}, request id: {result.request_id}")
print(f"next token:  {result.next_token}")

# Print aggregation results
for agg in result.aggregations.aggregations:
    print(f"aggregation — field: {agg.field}, operation: {agg.operation}")

# Print tags and user metadata for matching objects
if result.files:
    if result.files.file.oss_tagging.taggings:
        for tag in result.files.file.oss_tagging.taggings:
            print(f"tag: {tag.key} = {tag.value}")
    if result.files.file.oss_user_meta.user_metas:
        for meta in result.files.file.oss_user_meta.user_metas:
            print(f"user meta: {meta.key} = {meta.value}")

Result fields

Key fields available on result.files.file:

Field	Description
`filename`	Object name
`size`	Object size in bytes
`oss_storage_class`	Storage class (e.g., `Standard`, `IA`, `Archive`)
`oss_object_type`	Object type
`etag`	ETag of the object
`object_acl`	Object ACL
`server_side_encryption`	Server-side encryption algorithm
`file_modified_time`	Last modified time
`oss_tagging_count`	Number of object tags
`oss_tagging.taggings`	List of tag key-value pairs
`oss_user_meta.user_metas`	List of user metadata key-value pairs
`oss_crc64`	CRC-64 checksum

For the complete runnable sample, see do_meta_query.py.

Disable metadata management

Use close_meta_query to disable the metadata management feature for a bucket.

result = client.close_meta_query(oss.CloseMetaQueryRequest(
    bucket="<bucket-name>",
))

print(f"status code: {result.status_code}, request id: {result.request_id}")

For the complete runnable sample, see close_meta_query.py.