how to use oss python sdk v2 to upload files - Object Storage Service

Cette rubrique explique comment utiliser le nouveau module Uploader du Python SDK V2 pour charger des fichiers.

Notes

L'exemple de code de cette rubrique utilise l'endpoint public de la région Chine (Hangzhou). L'ID de région est cn-hangzhou. Pour accéder à OSS depuis d'autres services Alibaba Cloud situés dans la même région, utilisez un endpoint interne. Pour plus d'informations sur les régions et endpoints pris en charge par OSS, consultez Régions et endpoints OSS.
Pour charger un fichier, vous devez disposer de l'autorisation oss:PutObject. Pour plus d'informations, consultez Accorder des autorisations personnalisées à un utilisateur RAM.

Définition de la méthode

Présentation du gestionnaire de chargement

Le nouveau module Uploader du Python SDK V2 propose une méthode de chargement unifiée qui masque les détails d'implémentation sous-jacents afin de simplifier le chargement des fichiers.

L'Uploader utilise le chargement multipart pour diviser un fichier ou un flux en plusieurs parties, puis charge ces parties simultanément. Ce processus améliore les performances de chargement.
L'Uploader offre également une fonctionnalité de chargement avec reprise. Pendant le chargement, l'Uploader enregistre l'état des parties terminées. Si le chargement est interrompu par un problème tel qu'une erreur réseau ou une fermeture inattendue du programme, vous pouvez reprendre le chargement à partir des points d'arrêt enregistrés.

Le tableau suivant décrit les méthodes courantes de l'Uploader.

class Uploader:
  ...

def uploader(self, **kwargs) -> Uploader:
  ...

def upload_file(self, request: models.PutObjectRequest, filepath: str, **kwargs: Any) -> UploadResult:
  ...
  
def upload_from(self, request: models.PutObjectRequest, reader: IO[bytes], **kwargs: Any) -> UploadResult:
  ...

Paramètres de requête

Paramètre

Type

Description

request

PutObjectRequest

Les paramètres de requête pour le chargement d'un objet. Ces paramètres sont identiques à ceux de la méthode PutObject. Pour plus d'informations, consultez PutObjectRequest

reader

IO[bytes]

Le flux de données à charger

filepath

str

Le chemin du fichier local

**kwargs

Any

(Facultatif) Tout paramètre. Le type est un dictionnaire.

Paramètres de réponse

Type

Description

UploadResult

Les paramètres de réponse pour le chargement d'un objet. Pour plus d'informations, consultez UploadResult

Lorsque vous utilisez client.uploader pour initialiser une instance du gestionnaire de chargement, vous pouvez spécifier des options de configuration afin de personnaliser le comportement du chargement. Vous pouvez également définir ces options pour chaque appel d'API de chargement afin d'adapter le comportement au chargement d'un objet spécifique. Par exemple, vous pouvez spécifier la taille des parties.

Définissez les paramètres de configuration pour l'uploader
```
uploader = client.uploader(part_size=10  * 1024 * 1024)
```

Définissez les paramètres de configuration pour chaque requête de chargement

result = uploader.upload_file(oss.PutObjectRequest(
        bucket="example_bucket",
        key="example_key",
    ),
    filepath="/local/dir/example",
    part_size=10 * 1024 * 1024,
)

Le tableau suivant décrit les options de configuration courantes.

Paramètre	Type	Description
part_size	int	Spécifie la taille des parties. La valeur par défaut est de 6 MiB.
parallel_num	int	Spécifie le nombre de tâches de chargement simultanées. La valeur par défaut est 3. Ce paramètre limite la simultanéité pour un seul appel, et non la simultanéité globale.
leave_parts_on_error	bool	Indique s'il faut conserver les parties chargées en cas d'échec du chargement. Par défaut, les parties ne sont pas conservées.
enable_checkpoint	bool	Indique s'il faut activer le chargement avec reprise. Par défaut, cette fonctionnalité est désactivée. Remarque Le paramètre enable_checkpoint n'est valide que pour la méthode upload_file. La méthode upload_from ne prend pas en charge ce paramètre.
checkpoint_dir	str	Spécifie le chemin où le fichier d'enregistrement est sauvegardé, par exemple /local/dir/. Ce paramètre n'est valide que lorsque enable_checkpoint est défini sur true.

Pour obtenir les définitions complètes des méthodes du gestionnaire de chargement de fichiers, consultez Uploader.

Exemple de code

Utilisez le code suivant pour charger un fichier local vers un bucket à l'aide du gestionnaire de chargement.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files.
    uploader = client.uploader()

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Scénarios

Utiliser le gestionnaire de chargement pour activer le chargement avec reprise

Utilisez le code suivant pour activer le chargement avec reprise.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files, enable resumable upload, and specify the path to save the breakpoint record file.
    uploader = client.uploader(enable_checkpoint=True, checkpoint_dir="/Users/yourLocalPath/checkpoint/")

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Utiliser le gestionnaire de chargement pour charger un flux de fichier local

Utilisez le code suivant pour charger un flux de fichier local à l'aide du gestionnaire de chargement.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload from file sample
parser = argparse.ArgumentParser(description="upload from sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files.
    uploader = client.uploader()

    # Open the local file for reading in binary mode.
    with open(file=args.file_path, mode='rb') as f:
        # Call the method to perform the file upload operation.
        result = uploader.upload_from(
            oss.PutObjectRequest(
                bucket=args.bucket,  # Specify the destination bucket.
                key=args.key,        # Specify the name of the file in OSS.
            ),
            reader=f  # Pass in the file reader.
        )

        # Print information about the upload result, including the status code, request ID, and Content-MD5.
        print(f'status code: {result.status_code},'
              f' request id: {result.request_id},'
              f' content md5: {result.headers.get("Content-MD5")},'
              f' etag: {result.etag},'
              f' hash crc64: {result.hash_crc64},'
              f' version id: {result.version_id},'
              f' server time: {result.headers.get("x-oss-server-time")},'
              )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Utiliser le gestionnaire de chargement pour définir la taille des parties et la simultanéité

Utilisez le code suivant pour spécifier la taille des parties et la simultanéité.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files and set the part size and concurrency.
    uploader = client.uploader(
        part_size=100 * 1024,  # Set the part size to 100 KB.
        parallel_num=5,        # Set the concurrency to 5.
        leave_parts_on_error=True  # Retain the uploaded parts in case of an error.
    )

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Utiliser le gestionnaire de chargement pour configurer un callback de chargement

Si vous souhaitez notifier un serveur d'application après le chargement d'un fichier, utilisez l'exemple de code suivant.

import argparse
import base64
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter.
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider

    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an uploader object for uploading files.
    uploader = client.uploader()

    # Define the webhook address.
    call_back_url = "http://www.example.com/callback"
    # Construct the callback parameter (callback): specify the webhook address and the request body for the callback, and encode them in Base64.
    callback=base64.b64encode(str('{\"callbackUrl\":\"' + call_back_url + '\",\"callbackBody\":\"bucket=${bucket}&object=${object}&my_var_1=${x:var1}&my_var_2=${x:var2}\"}').encode()).decode()
    # Construct the custom variable (callback-var) and encode it in Base64.
    callback_var=base64.b64encode('{\"x:var1\":\"value1\",\"x:var2\":\"value2\"}'.encode()).decode()

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
            callback=callback,
            callback_var=callback_var,
        ),
        filepath=args.file_path,  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Références

Pour plus d'informations sur le gestionnaire de chargement, consultez le Guide du développeur.
Pour des exemples complets du gestionnaire de chargement, consultez upload_file.py et upload_from.py.