:Filter OSS objects by time range

Use cases

• Auditing and compliance checks

  Enterprises and organizations must periodically review data activities to ensure compliance. Querying objects uploaded within a specific time range helps manage the data lifecycle, track data creation and modification history, and support data auditing processes.

• Data backup and recovery

  When performing backup or recovery operations based on a defined policy, querying new or updated objects within a specified time range helps identify the exact backup targets, saving storage space and data transfer costs.

• Data analysis and reporting

  For applications that involve big data analytics, log processing, or business reporting, you often need to process new data generated within a specific period, such as daily, weekly, or monthly. Querying objects within a specified time range helps you quickly locate the dataset for analysis and simplifies data extraction.

• Content management system synchronization

  A content management system must regularly synchronize new or updated objects to keep website content current and complete.

• Cost optimization and resource cleanup

  Enterprises must periodically clean up obsolete objects. Querying for objects that have not been updated within a specified time frame helps identify which objects to delete, ensuring efficient resource utilization.

• Troubleshooting and issue tracing

  When you troubleshoot system failures, data loss, or abnormal behavior, querying for objects within a specific time period helps you quickly locate problematic objects or operation records.

• Collaboration and version control

  In a multi-user collaboration environment, querying objects within a time range lets you browse historical versions, compare changes, or restore data to a specific point in time.

Procedure

package com.aliyun.sts.sample;

import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSS;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.model.*;

public class Demo {

    // This example uses the endpoint of the China (Hangzhou) region. Replace the value with your actual endpoint.
    private static String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
    // Specify the region that corresponds to your endpoint. For example, if your endpoint is oss-cn-hangzhou.aliyuncs.com, set the region to cn-hangzhou.
    private static String region = "cn-hangzhou";
    
    // Specify the bucket name. Example: examplebucket.
    private static String bucketName = "examplebucket";

    public static void main(String[] args) throws Exception {
        // Obtain access credentials from environment variables. Before you run this code, make sure that you have configured the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Create an OSSClient instance.
        // Call the shutdown method to release resources when the OSSClient instance is no longer needed.
        ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
        // Explicitly declare that the V4 signature algorithm is used.
        clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);
        OSS ossClient = OSSClientBuilder.create()
                .endpoint(endpoint)
                .credentialsProvider(credentialsProvider)
                .clientConfiguration(clientBuilderConfiguration)
                .region(region)
                .build();

        try {
            // Set the maximum number of objects to return to 100.
            int maxResults = 100;
            // Query objects that were last modified between December 1, 2023 and March 31, 2024.
            String query = "{\n" +
                    "  \"SubQueries\":[\n" +
                    "    {\n" +
                    "      \"Field\":\"FileModifiedTime\",\n" +
                    "      \"Value\": \"2023-12-01T00:00:00.000+08:00\",\n" +
                    "      \"Operation\":\"gt\"\n" +
                    "    },         \n" +
                    "    {\n" +
                    "      \"Field\":\"FileModifiedTime\",\n" +
                    "      \"Value\": \"2024-03-31T23:59:59.000+08:00\",\n" +
                    "      \"Operation\":\"lt\"\n" +
                    "    }\n" +
                    "  ],\n" +
                    "  \"Operation\":\"and\"\n" +
                    "}";
            // Sort the results by object name in ascending order.
            String sort = "Filename";
            DoMetaQueryRequest doMetaQueryRequest = new DoMetaQueryRequest(bucketName, maxResults, query, sort);
            doMetaQueryRequest.setOrder(SortOrder.ASC);
            DoMetaQueryResult doMetaQueryResult = ossClient.doMetaQuery(doMetaQueryRequest);

            if (doMetaQueryResult.getFiles() != null) {
                for (ObjectFile file : doMetaQueryResult.getFiles().getFile()) {
                    System.out.println("Filename: " + file.getFilename());
                    // Obtain the object's ETag.
                    System.out.println("ETag: " + file.getETag());
                    // Obtain the access control list (ACL) of the object.
                    System.out.println("ObjectACL: " + file.getObjectACL());
                    // Obtain the object type.
                    System.out.println("OssObjectType: " + file.getOssObjectType());
                    // Obtain the object storage class.
                    System.out.println("OssStorageClass: " + file.getOssStorageClass());
                    // Obtain the number of object tags.
                    System.out.println("TaggingCount: " + file.getOssTaggingCount());
                    if (file.getOssTagging() != null) {
                        for (Tagging tag : file.getOssTagging().getTagging()) {
                            System.out.println("Key: " + tag.getKey());
                            System.out.println("Value: " + tag.getValue());
                        }
                    }
                    if (file.getOssUserMeta() != null) {
                        for (UserMeta meta : file.getOssUserMeta().getUserMeta()) {
                            System.out.println("Key: " + meta.getKey());
                            System.out.println("Value: " + meta.getValue());
                        }
                    }
                }
            }
        } catch (OSSException oe) {
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Error Message: " + ce.getMessage());
        } finally {
            // Shut down the OSSClient instance.
            ossClient.shutdown();
        }
    }
}

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import MetaQuery, AggregationsRequest
# Obtain access credentials from environment variables. Before you run this code, make sure that you have configured the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.
auth = oss2.ProviderAuth(EnvironmentVariableCredentialsProvider())

# This example uses the endpoint of the China (Hangzhou) region. Replace the value with your actual endpoint.
# Specify the bucket name. Example: examplebucket.
bucket = oss2.Bucket(auth, 'http://oss-cn-hangzhou.aliyuncs.com', 'examplebucket0703')

# Query objects last modified between December 1, 2023 and March 31, 2024, and sort the results by object name in ascending order.
do_meta_query_request = MetaQuery(max_results=100, query='{"SubQueries":[{"Field":"FileModifiedTime","Value": "2023-12-01T00:00:00.000+08:00","Operation":"gt"}, {"Field":"FileModifiedTime","Value": "2024-03-31T23:59:59.000+08:00","Operation":"lt"}],"Operation":"and"}', sort='Filename', order='asc')
result = bucket.do_bucket_meta_query(do_meta_query_request)

for s in result.files:
    print(s.file_name)
    print(s.etag)
    print(s.oss_object_type)
    print(s.oss_storage_class)
    print(s.oss_crc64)
    print(s.object_acl)

package main

import (
	"fmt"
	"github.com/aliyun/aliyun-oss-go-sdk/oss"
	"os"
)

func main() {
	// Obtain access credentials from environment variables. Before you run this code, make sure that you have configured the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Create an OSSClient instance.
	// This example uses the endpoint for the China (Hangzhou) region. Replace the value with your actual endpoint.
	client, err := oss.New("oss-cn-hangzhou.aliyuncs.com", "", "", oss.SetCredentialsProvider(&provider))
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	// Query objects that were last modified between December 1, 2023 and March 31, 2024.
	query := oss.MetaQuery{
		NextToken:  "",
		MaxResults: 100,
		Query: `{
  "SubQueries":[
    {
      "Field":"FileModifiedTime",
      "Value": "2023-12-01T00:00:00.000+08:00",
      "Operation":"gt"
    },         
    {
      "Field":"FileModifiedTime",
      "Value": "2024-03-31T23:59:59.000+08:00",
      "Operation":"lt"
    }
  ],
  "Operation":"and"
}`,
                // Sort the results by object name in ascending order.
		Sort:  "Filename",
		Order: "asc",
	}
	// Query for objects that meet the specified conditions and list the object information based on the specified fields and sort order.
	result, err := client.DoMetaQuery("examplebucket", query)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	fmt.Printf("NextToken:%s\n", result.NextToken)
	for _, file := range result.Files {
		fmt.Printf("File name: %s\n", file.Filename)
		fmt.Printf("size: %d\n", file.Size)
		fmt.Printf("File Modified Time:%s\n", file.FileModifiedTime)
		fmt.Printf("Oss Object Type:%s\n", file.OssObjectType)
		fmt.Printf("Oss Storage Class:%s\n", file.OssStorageClass)
		fmt.Printf("Object ACL:%s\n", file.ObjectACL)
		fmt.Printf("ETag:%s\n", file.ETag)
		fmt.Printf("Oss CRC64:%s\n", file.OssCRC64)
		fmt.Printf("Oss Tagging Count:%d\n", file.OssTaggingCount)
		for _, tagging := range file.OssTagging {
			fmt.Printf("Oss Tagging Key:%s\n", tagging.Key)
			fmt.Printf("Oss Tagging Value:%s\n", tagging.Value)
		}
		for _, userMeta := range file.OssUserMeta {
			fmt.Printf("Oss User Meta Key:%s\n", userMeta.Key)
			fmt.Printf("Oss User Meta Key Value:%s\n", userMeta.Value)
		}
	}
}

References

Data indexing supports multiple filter conditions, such as storage class, read/write permissions, and object size, to help you quickly find the data you need. For example, you can query for all objects with public-read permissions or all objects smaller than 64 KB. For more information, see Data indexing.