All Products
Search
Document Center

Object Storage Service:Filter OSS objects whose last modified time is within a specific period of time

Last Updated:Mar 23, 2025

If you want to filter Object Storage Service (OSS) objects whose last modified time is within a specific period of time, you can use the data indexing feature. You can specify the start date and end date of the last modified time of an object as index conditions to improve object query efficiency. This feature is suitable for the accurate retrieval of time-sensitive objects in audit trails, data synchronization, periodic backup, cost analysis, or other business scenarios.

Scenarios

  • Audit and compliance requirements

    Enterprises or organizations need to periodically review data activities within a specific period of time to ensure compliance. Querying objects uploaded within a specific period of time can help you manage data lifecycle, track data generation and change history, and audit data.

  • Data backup and restoration

    When you perform a backup or recovery operation with backup policies specified, querying for newly added or updated objects within a specified time range helps identify backup targets, thereby reducing storage usage and transmission costs.

  • Data analysis and report generation

    For applications that involve big data analysis, log processing, or business reporting, new data generated within a specific period of time are required to be processed on a regular basis, such as daily, weekly, or monthly. Querying objects generated within a specific period of time can help you quickly identify the dataset that you want to analyze and simplify the data extraction process.

  • Data synchronization performed by a content management system

    The content management system is required to regularly synchronize objects uploaded or updated within a specific period of time to maintain the timeliness and integrity of website content.

  • Cost optimization and resource deletion

    Enterprises need to periodically review and delete objects that are no longer needed. Querying and listing objects that have not been updated within a specific period of time helps you identify objects that are no longer needed to be retained. This ensures the efficient use of resources.

  • Troubleshooting and issue tracing

    When system failures, data loss, or unusual behaviors, querying objects uploaded within a specific period of time helps engineers quickly locate objects or operations that may cause issues.

  • Collaboration and versioning

    In a multi-user collaboration environment, querying objects uploaded within a specific period of time helps you browse, compare, or restore historical versions to a specific point in time.

Methods

Use the OSS console

  1. Log on to the OSS console.

  2. In the left-side navigation pane, click Buckets. On the Buckets page, find and click the desired bucket.

  3. In the left-side navigation tree, choose Object Management > Data Indexing.

  4. On the Data Indexing page, turn on Metadata Management.

    The amount of time required for metadata management to take effect varies based on the number of objects in the bucket.

  5. In the OSS metadata conditions section, specify Start Date and End Date to the right of Last Modified At. The value of Start Date and End Date are accurate to seconds. Retain the default settings for other parameters.

  6. In the Object Sort Order section, set Sort Order to Ascending and Sorted By to Object Name.

  7. Click Query.

Use OSS SDKs

Only OSS SDK for Java, OSS SDK for Python, and OSS SDK for Go allow you to use the data indexing feature to query objects that meet specific conditions. Before you use the data indexing feature to query objects in a bucket, you must enable the metadata management feature for the bucket.

Java

package com.aliyun.sts.sample;

import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSS;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.model.*;

public class Demo {

    // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
    private static String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
    // Specify the name of the bucket. Example: examplebucket. 
    private static String bucketName = "examplebucket";

    public static void main(String[] args) throws Exception {
        // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Create an OSSClient instance. 
        OSS ossClient = new OSSClientBuilder().build(endpoint, credentialsProvider);

        try {
            // Set the maximum number of objects that you want to return to 100. 
            int maxResults = 100;
            // Specify the start date and end date of the last modified time of the objects that you want to query. In this example, the start date is set to December 1, 2023 and the end date is set to March 31, 2024. 
            String query = "{\n" +
                    "  \"SubQueries\":[\n" +
                    "    {\n" +
                    "      \"Field\":\"FileModifiedTime\",\n" +
                    "      \"Value\": \"2023-12-01T00:00:00.000+08:00\",\n" +
                    "      \"Operation\":\"gt\"\n" +
                    "    },         \n" +
                    "    {\n" +
                    "      \"Field\":\"FileModifiedTime\",\n" +
                    "      \"Value\": \"2024-03-31T23:59:59.000+08:00\",\n" +
                    "      \"Operation\":\"lt\"\n" +
                    "    }\n" +
                    "  ],\n" +
                    "  \"Operation\":\"and\"\n" +
                    "}";
            // Specify that the returned results are sorted in ascending order by object name. 
            String sort = "Filename";
            DoMetaQueryRequest doMetaQueryRequest = new DoMetaQueryRequest(bucketName, maxResults, query, sort);
            doMetaQueryRequest.setOrder(SortOrder.ASC);
            DoMetaQueryResult doMetaQueryResult = ossClient.doMetaQuery(doMetaQueryRequest);

            if (doMetaQueryResult.getFiles() != null) {
                for (ObjectFile file : doMetaQueryResult.getFiles().getFile()) {
                    System.out.println("Filename: " + file.getFilename());
                    // Query the ETag values that are used to identify the content of the objects. 
                    System.out.println("ETag: " + file.getETag());
                    // Query the access control list (ACL) of the objects.
                    System.out.println("ObjectACL: " + file.getObjectACL());
                    // Query the type of the objects. 
                    System.out.println("OssObjectType: " + file.getOssObjectType());
                    // Query the storage class of the objects. 
                    System.out.println("OssStorageClass: " + file.getOssStorageClass());
                    // Query the number of tags of the objects. 
                    System.out.println("TaggingCount: " + file.getOssTaggingCount());
                    if (file.getOssTagging() != null) {
                        for (Tagging tag : file.getOssTagging().getTagging()) {
                            System.out.println("Key: " + tag.getKey());
                            System.out.println("Value: " + tag.getValue());
                        }
                    }
                    if (file.getOssUserMeta() != null) {
                        for (UserMeta meta : file.getOssUserMeta().getUserMeta()) {
                            System.out.println("Key: " + meta.getKey());
                            System.out.println("Value: " + meta.getValue());
                        }
                    }
                }
            }
        } catch (OSSException oe) {
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Error Message: " + ce.getMessage());
        } finally {
            // Shut down the OSSClient instance. 
            ossClient.shutdown();
        }
    }
}

Python

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import MetaQuery, AggregationsRequest
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuth(EnvironmentVariableCredentialsProvider())

# In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
# Specify the name of the bucket. Example: examplebucket. 
bucket = oss2.Bucket(auth, 'http://oss-cn-hangzhou.aliyuncs.com', 'examplebucket0703')

# Specify the start date and end date of the last modified time of the objects that you want to query. In this example, the start date is set to December 1, 2023 and the end date is set to March 31, 2024. 
do_meta_query_request = MetaQuery(max_results=100, query='{"SubQueries":[{"Field":"FileModifiedTime","Value": "2023-12-01T00:00:00.000+08:00","Operation":"gt"}, {"Field":"FileModifiedTime","Value": "2024-03-31T23:59:59.000+08:00","Operation":"lt"}],"Operation":"and"}', sort='Filename', order='asc')
result = bucket.do_bucket_meta_query(do_meta_query_request)

for s in result.files:
    print(s.file_name)
    print(s.etag)
    print(s.oss_object_type)
    print(s.oss_storage_class)
    print(s.oss_crc64)
    print(s.object_acl)

Go

package main

import (
	"fmt"
	"github.com/aliyun/aliyun-oss-go-sdk/oss"
	"os"
)

func main() {
	// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Create an OSSClient instance. 
	// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
	client, err := oss.New("oss-cn-hangzhou.aliyuncs.com", "", "", oss.SetCredentialsProvider(&provider))
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	// Specify the start date and end date of the last modified time of the objects that you want to query. In this example, the start date is set to December 1, 2023 and the end date is set to March 31, 2024. 
	query := oss.MetaQuery{
		NextToken:  "",
		MaxResults: 100,
		Query: `{
  "SubQueries":[
    {
      "Field":"FileModifiedTime",
      "Value": "2023-12-01T00:00:00.000+08:00",
      "Operation":"gt"
    },         
    {
      "Field":"FileModifiedTime",
      "Value": "2024-03-31T23:59:59.000+08:00",
      "Operation":"lt"
    }
  ],
  "Operation":"and"
}`,
                // Specify that the returned results are sorted in ascending order by object name. 
		Sort:  "Filename",
		Order: "asc",
	}
	// Query objects that match specific conditions and list object information based on specific fields and sorting methods. 
	result, err := client.DoMetaQuery("examplebucket", query)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	fmt.Printf("NextToken:%s\n", result.NextToken)
	for _, file := range result.Files {
		fmt.Printf("File name: %s\n", file.Filename)
		fmt.Printf("size: %d\n", file.Size)
		fmt.Printf("File Modified Time:%s\n", file.FileModifiedTime)
		fmt.Printf("Oss Object Type:%s\n", file.OssObjectType)
		fmt.Printf("Oss Storage Class:%s\n", file.OssStorageClass)
		fmt.Printf("Object ACL:%s\n", file.ObjectACL)
		fmt.Printf("ETag:%s\n", file.ETag)
		fmt.Printf("Oss CRC64:%s\n", file.OssCRC64)
		fmt.Printf("Oss Tagging Count:%d\n", file.OssTaggingCount)
		for _, tagging := range file.OssTagging {
			fmt.Printf("Oss Tagging Key:%s\n", tagging.Key)
			fmt.Printf("Oss Tagging Value:%s\n", tagging.Value)
		}
		for _, userMeta := range file.OssUserMeta {
			fmt.Printf("Oss User Meta Key:%s\n", userMeta.Key)
			fmt.Printf("Oss User Meta Key Value:%s\n", userMeta.Value)
		}
	}
}

Use the OSS API

If your business requires a high level of customization, you can directly call RESTful APIs. To directly call an API, you must include the signature calculation in your code. For more information, see DoMetaQuery.

Reference

The data indexing feature supports multiple filtering conditions, such as the storage class, ACL, and object size, and you can filter objects that meet specific conditions efficiently. For example, if you want to query the objects whose ACL is public-read or whose size is less than 64 KB in a bucket. For more information, see Data indexing.