Utiliser MetaSearch avec le SDK OSS pour Python - Object Storage Service

MetaSearch, une fonctionnalité d'Object Storage Service (OSS), indexe les objets selon leurs métadonnées. Vous pouvez utiliser ces métadonnées comme conditions de recherche pour interroger vos objets. Cette approche vous permet de gérer et comprendre les structures de données, d'exécuter des requêtes, de collecter des statistiques et de gérer les objets efficacement.

Notes d'utilisation

L'exemple de code de cette rubrique utilise l'ID de région cn-hangzhou de la région Chine (Hangzhou). Par défaut, le point de terminaison public permet d'accéder aux ressources d'un bucket. Pour accéder aux ressources du bucket depuis d'autres services Alibaba Cloud situés dans la même région, utilisez un point de terminaison interne. Pour plus d'informations sur les régions et les points de terminaison OSS, consultez Régions et points de terminaison.

Exemples de code

Activer la fonctionnalité de gestion des métadonnées

L'exemple de code suivant active la fonctionnalité de gestion des métadonnées pour un bucket spécifique. Une fois activée, OSS crée une bibliothèque d'index de métadonnées pour ce bucket et génère les index de tous les objets qu'il contient. Ensuite, OSS analyse en quasi temps réel les nouveaux objets ajoutés au bucket et crée leurs index de métadonnées.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line parameter parser and describe the purpose of the script.
parser = argparse.ArgumentParser(description="open meta query sample")
# Specify the --region parameter to indicate the region in which the bucket is located. This parameter is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Specify the --bucket parameter to indicate the bucket. This parameter is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Specify the --endpoint parameter to indicate the endpoint for accessing OSS. This parameter is required.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')

def main():
    # Parse the command-line arguments.
    args = parser.parse_args()

    # From the environment variables, load the authentication information required to access OSS.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configuration of the SDK.
    cfg = oss.config.load_default()
    # Specify the credential provider.
    cfg.credentials_provider = credentials_provider
    # Set the region to the one provided from the command line.
    cfg.region = args.region
    # If a custom endpoint is provided, update the endpoint attribute of the cfg object with the provided endpoint.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client.
    client = oss.Client(cfg)

    # Initiate a request to enable metadata-based query.
    result = client.open_meta_query(oss.OpenMetaQueryRequest(
            bucket=args.bucket,
    ))

    # Display the HTTP status code and request ID.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          )

# Call the main function when the script is directly run.
if __name__ == "__main__":
    main()

Interroger la bibliothèque d'index de métadonnées d'un bucket

L'exemple de code suivant interroge la bibliothèque d'index de métadonnées d'un bucket :

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line parameter parser and define the parameters.
parser = argparse.ArgumentParser(description="get meta query status sample")
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')

def main():
    # Parse the command-line arguments.
    args = parser.parse_args()

    # Obtain access credentials from environment variables.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configuration of the SDK.
    cfg = oss.config.load_default()
    # Set the credential provider to the credential provider obtained from the environment variables.
    cfg.credentials_provider = credentials_provider
    # Set the region in which the bucket is located.
    cfg.region = args.region
    # If a custom endpoint is provided, update the endpoint attribute of the cfg object with the provided endpoint.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client.
    client = oss.Client(cfg)

    # Call the get_meta_query_status method to obtain the metadata query status of the specified bucket.
    result = client.get_meta_query_status(oss.GetMetaQueryStatusRequest(
            bucket=args.bucket,
    ))

    # Print the response information, including the status code, request ID, creation time, update time, status, and phase.
    print(f'status code: {result.status_code},'
            f' request id: {result.request_id},'
            f' create time: {result.meta_query_status.create_time},'
            f' update time: {result.meta_query_status.update_time},'
            f' state: {result.meta_query_status.state},'
            f' phase: {result.meta_query_status.phase},'
    )

# Call the main function when the script is directly run.
if __name__ == "__main__":
    main()

Interroger les objets répondant à des conditions spécifiques

L'exemple de code suivant interroge les objets répondant à des conditions spécifiques et liste leurs informations selon un champ et un ordre de tri donnés :

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line parameter parser for parsing arguments from the command line.
parser = argparse.ArgumentParser(description="do meta query sample")
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')

def main():
    # Parse the command-line arguments.
    args = parser.parse_args()

    # Obtain access credentials from environment variables.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configuration of the SDK.
    cfg = oss.config.load_default()
    # Specify the credential provider.
    cfg.credentials_provider = credentials_provider
    # Set the region to the one provided from the command line.
    cfg.region = args.region
    # If a custom endpoint is provided, update the endpoint attribute of the cfg object with the provided endpoint.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client.
    client = oss.Client(cfg)

    # Perform the metadata-based query.
    result = client.do_meta_query(oss.DoMetaQueryRequest(
            bucket=args.bucket,  # The bucket that stores the objects to be queried.
            meta_query=oss.MetaQuery(  # Specify query settings.
                aggregations=oss.MetaQueryAggregations(  # Specify aggregatios.
                    aggregations=[  # The aggregation list.
                        oss.MetaQueryAggregation(  # The first aggregation: the total object size.
                            field='Size',
                            operation='sum',
                        ),
                        oss.MetaQueryAggregation(  # The second aggregation: the maximum object size.
                            field='Size',
                            operation='max',
                        )
                    ],
                ),
                next_token='',  # The pagination token.
                max_results=80369,  # The maximum number of entries that can be returned.
                query='{"Field": "Size","Value": "1048576","Operation": "gt"}',  # The query condition.
                sort='Size',  # The sorting field.
                order=oss.MetaQueryOrderType.DESC,  # The sorting order.
            ),
    ))

    # Display the basic response information.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          # You can uncomment the one or more of the following lines to display more details.
          # f' files: {result.files},'
          # f' file: {result.files.file},'
          # f' file modified time: {result.files.file.file_modified_time},'
          # f' etag: {result.files.file.etag},'
          # f' server side encryption: {result.files.file.server_side_encryption},'
          # f' oss tagging count: {result.files.file.oss_tagging_count},'
          # f' oss tagging: {result.files.file.oss_tagging},'
          # f' key: {result.files.file.oss_tagging.taggings[0].key},'
          # f' value: {result.files.file.oss_tagging.taggings[0].value},'
          # f' key: {result.files.file.oss_tagging.taggings[1].key},'
          # f' value: {result.files.file.oss_tagging.taggings[1].value},'
          # f' oss user meta: {result.files.file.oss_user_meta},'
          # f' key: {result.files.file.oss_user_meta.user_metas[0].key},'
          # f' value: {result.files.file.oss_user_meta.user_metas[0].value},'
          # f' key: {result.files.file.oss_user_meta.user_metas[1].key},'
          # f' value: {result.files.file.oss_user_meta.user_metas[1].value},'
          # f' filename: {result.files.file.filename},'
          # f' size: {result.files.file.size},'
          # f' oss object type: {result.files.file.oss_object_type},'
          # f' oss storage class: {result.files.file.oss_storage_class},'
          # f' object acl: {result.files.file.object_acl},'
          # f' oss crc64: {result.files.file.oss_crc64},'
          # f' server side encryption customer algorithm: {result.files.file.server_side_encryption_customer_algorithm},'
          # f' aggregations: {result.aggregations},'
          f' field: {result.aggregations.aggregations[0].field},'
          f' operation: {result.aggregations.aggregations[0].operation},'
          f' field: {result.aggregations.aggregations[1].field},'
          f' operation: {result.aggregations.aggregations[1].operation},'
          f' next token: {result.next_token},'
    )

    # If matched objects are found, display tags and user metadata.
    if result.files:
        if result.files.file.oss_tagging.taggings:
            for r in result.files.file.oss_tagging.taggings:
                print(f'result: key: {r.key}, value: {r.value}')
        if result.files.file.oss_user_meta.user_metas:
            for r in result.files.file.oss_user_meta.user_metas:
                print(f'result: key: {r.key}, value: {r.value}')
    # Display all aggregations.
    if result.aggregations.aggregations:
        for r in result.aggregations.aggregations:
            print(f'result: field: {r.field}, operation: {r.operation}')

if __name__ == "__main__":
    main()

Désactiver la fonctionnalité de gestion des métadonnées

L'exemple de code suivant désactive la fonctionnalité de gestion des métadonnées pour un bucket spécifique :

import argparse
import alibabacloud_oss_v2 as oss

# Create an ArgumentParser object for processing command-line arguments.
parser = argparse.ArgumentParser(description="close meta query sample")
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')

def main():
    # Parse the command-line arguments.
    args = parser.parse_args()

    # Obtain access credentials from environment variables.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configuration of the SDK.
    cfg = oss.config.load_default()
    # Set the credential provider to the credential provider obtained from the environment variables.
    cfg.credentials_provider = credentials_provider
    # Set the region in the configuration to the one specified in the command line.
    cfg.region = args.region
    # If a custom endpoint is provided, update the endpoint attribute of the cfg object with the provided endpoint.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Create an OSS client.
    client = oss.Client(cfg)

    # Call the close_meta_query method to disable the metadata query feature for the specified bucket.
    result = client.close_meta_query(oss.CloseMetaQueryRequest(
            bucket=args.bucket,
    ))

    # Display the HTTP status code and request ID.
    print(f'status code: {result.status_code}, request id: {result.request_id}')

# Call the main function when the script is directly run.
if __name__ == "__main__":
    main()

Références

Pour obtenir l'exemple de code complet permettant d'activer la gestion des métadonnées, consultez open_meta_query.py.
Pour obtenir l'exemple de code complet permettant d'interroger la bibliothèque d'index de métadonnées d'un bucket, consultez get_meta_query_status.py.
Pour obtenir l'exemple de code complet permettant d'interroger des objets selon des conditions et de lister les informations des objets correspondants par champ spécifié dans un ordre donné, consultez do_meta_query.py.
Pour obtenir l'exemple de code complet permettant de désactiver la gestion des métadonnées, consultez close_meta_query.py.