Object Storage Service:Téléchargement multipart (Python SDK V1)

Notes

• Cette rubrique utilise le point de terminaison public de la région Chine (Hangzhou). Si vous souhaitez accéder à OSS depuis d'autres services Alibaba Cloud situés dans la même région qu'OSS, utilisez un point de terminaison interne. Pour plus d'informations sur les régions et les points de terminaison OSS, consultez Régions et points de terminaison.

• Dans cette rubrique, les identifiants d'accès sont obtenus à partir des variables d'environnement. Pour plus d'informations sur la configuration des identifiants d'accès, consultez Configurer les identifiants d'accès à l'aide d'OSS SDK for Python 1.0.

• Cette rubrique montre comment créer une instance OSSClient avec un point de terminaison OSS. Pour d'autres configurations, telles que l'utilisation d'un domaine personnalisé ou l'authentification avec des identifiants provenant du Security Token Service (STS), consultez Initialisation.

• Le processus de téléchargement multipart (InitiateMultipartUpload, UploadPart et CompleteMultipartUpload) nécessite l'autorisation oss:PutObject. Accorder des autorisations personnalisées à un utilisateur RAM.

Processus de téléchargement multipart

Un téléchargement multipart comporte trois étapes :

1. Initialisez un événement de téléchargement multipart.

   Appelez bucket.init_multipart_upload pour obtenir un uploadId globalement unique.

2. Téléchargez les parties.

   Appelez bucket.upload_part pour télécharger chaque partie.

   Remarque

   • Pour un uploadId donné, le numéro de partie identifie la position d'une partie dans le fichier. Le téléchargement avec le même numéro de partie écrase les données existantes.

   • OSS renvoie le hachage MD5 des données de partie reçues dans l'en-tête de réponse ETag.

   • OSS compare le hachage MD5 des données téléchargées avec le hachage MD5 calculé par le SDK. En cas de non-concordance, le code d'erreur InvalidDigest est renvoyé.

3. Finalisez le téléchargement multipart.

   Une fois toutes les parties téléchargées, appelez bucket.complete_multipart_upload pour les combiner en un objet complet.

Exemples complets de téléchargement multipart

Vous pouvez combiner les parties téléchargées en un objet complet de deux manières :

• Combinez les parties en transmettant les informations de partie dans le corps de la requête

  # -*- coding: utf-8 -*-
  import os
  from oss2 import SizedFileAdapter, determine_part_size
  from oss2.models import PartInfo
  import oss2
  from oss2.credentials import EnvironmentVariableCredentialsProvider
  
  # Obtain access credentials from environment variables. Before running this code, make sure you have set the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.
  auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
  
  # Set Endpoint to the endpoint of the region where the bucket is located. For example, for a bucket in the China (Hangzhou) region, set Endpoint to https://oss-cn-hangzhou.aliyuncs.com.
  endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
  
  # Set region to the region ID that corresponds to the endpoint, for example, cn-hangzhou. Note: This parameter is required for SignatureV4.
  region = "cn-hangzhou"
  
  # Set yourBucketName to the name of your bucket.
  bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)
  
  # Set key to the full path of the object. The path cannot include the bucket name. Example: exampledir/exampleobject.txt.
  key = 'exampledir/exampleobject.txt'
  # Set filename to the full path of the local file. Example: D:\\localpath\\examplefile.txt.
  filename = 'D:\\localpath\\examplefile.txt'
  
  total_size = os.path.getsize(filename)
  # The determine_part_size method determines the part size. The minimum part size is 100 KB, and the maximum is 5 GB. The last part can be smaller than 100 KB. This example sets the part size to 1 MB.
  part_size = determine_part_size(total_size, preferred_size=1 * 1024 * 1024)
  
  # Initialize the multipart upload.
  # To set headers when you initialize the multipart upload, set the relevant headers in init_multipart_upload as shown below.
  # headers = dict()
  # Specify the web page caching behavior for the object.
  # headers['Cache-Control'] = 'no-cache'
  # Specify the name of the object when it is downloaded.
  # headers['Content-Disposition'] = 'oss_MultipartUpload.txt'
  # Specify the expiration time in milliseconds.
  # headers['Expires'] = '1000'
  # Specify whether to overwrite an object that has the same name when you initialize the multipart upload. Here, it is set to true, which prohibits overwriting.
  # headers['x-oss-forbid-overwrite'] = 'true'
  # Specify the server-side encryption method for each part of the object.
  # headers[OSS_SERVER_SIDE_ENCRYPTION] = SERVER_SIDE_ENCRYPTION_KMS
  # Specify the encryption algorithm for the object. If this is not specified, AES256 is used.
  # headers[OSS_SERVER_SIDE_DATA_ENCRYPTION] = SERVER_SIDE_ENCRYPTION_KMS
  # The customer master key (CMK) managed by KMS.
  # headers[OSS_SERVER_SIDE_ENCRYPTION_KEY_ID] = '9468da86-3509-4f8d-a61e-6eab1eac****'
  # Specify the storage class of the object.
  # headers['x-oss-storage-class'] = oss2.BUCKET_STORAGE_CLASS_STANDARD
  # Specify object tags. You can set multiple tags.
  # headers[OSS_OBJECT_TAGGING] = 'k1=v1&k2=v2&k3=v3'
  # upload_id = bucket.init_multipart_upload(key, headers=headers).upload_id
  upload_id = bucket.init_multipart_upload(key).upload_id
  # Use the upload_id to cancel the multipart upload event or list uploaded parts.
  # To cancel a multipart upload event by uploadId, get the uploadId after you call InitiateMultipartUpload.
  # To list uploaded parts by uploadId, get the uploadId after you call InitiateMultipartUpload and before you call CompleteMultipartUpload.
  # print("UploadID:", upload_id)
  parts = []
  
  # Upload parts one by one.
  with open(filename, 'rb') as fileobj:
      part_number = 1
      offset = 0
      while offset < total_size:
          num_to_upload = min(part_size, total_size - offset)
          # The SizedFileAdapter(fileobj, size) method generates a new file object and recalculates the starting position for appending.
          result = bucket.upload_part(key, upload_id, part_number,
                                      SizedFileAdapter(fileobj, num_to_upload))
          parts.append(PartInfo(part_number, result.etag))
  
          offset += num_to_upload
          part_number += 1
  
  # Complete the multipart upload.
  # To set headers when you complete the multipart upload, see the following sample code.
  headers = dict()
  # Set the access control list (ACL) for the file. Here, it is set to OBJECT_ACL_PRIVATE, which means private.
  # headers["x-oss-object-acl"] = oss2.OBJECT_ACL_PRIVATE
  bucket.complete_multipart_upload(key, upload_id, parts, headers=headers)
  # bucket.complete_multipart_upload(key, upload_id, parts)

  Important

  Si les conditions réseau sont bonnes, augmentez la taille des parties. Sinon, réduisez la taille des parties.

• Combinez les parties en répertoriant les données de partie depuis le serveur

  Remarque

  Avant d'utiliser cette méthode, assurez-vous que plusieurs parties ont été téléchargées avec l'upload_id spécifié dans le code suivant.

  # -*- coding: utf-8 -*-
  import oss2
  from oss2.credentials import EnvironmentVariableCredentialsProvider
  # Obtain access credentials from environment variables. Before running this code, make sure you have set the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.
  auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
  
  # Set Endpoint to the endpoint of the region where the bucket is located. For example, for a bucket in the China (Hangzhou) region, set Endpoint to https://oss-cn-hangzhou.aliyuncs.com.
  endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
  
  # Set region to the region ID that corresponds to the endpoint, for example, cn-hangzhou. Note: This parameter is required for SignatureV4.
  region = "cn-hangzhou"
  
  # Set yourBucketName to the name of your bucket.
  bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)
  
  # Set key to the full path of the object. The path cannot include the bucket name. Example: exampledir/exampleobject.txt.
  key = 'exampledir/exampleobject.txt'
  # Set filename to the full path of the local file. Example: D:\\localpath\\examplefile.txt.
  filename = 'D:\\localpath\\examplefile.txt'
  # Set upload_id. Get the upload_id after you call InitiateMultipartUpload and before you call CompleteMultipartUpload.
  upload_id = '0004B9894A22E5B1888A1E29F823****'
  
  # Complete the multipart upload.
  # To set the file ACL when you complete the multipart upload, set the relevant headers in the complete_multipart_upload function as shown below.
  headers = dict()
  # headers["x-oss-object-acl"] = oss2.OBJECT_ACL_PRIVATE
  # If you set x-oss-complete-all to yes, OSS lists all parts that have been uploaded with the current uploadId, sorts them by part number, and then runs the CompleteMultipartUpload operation.
  # If you set x-oss-complete-all to yes, you cannot specify a body. Otherwise, an error is returned.
  headers["x-oss-complete-all"] = 'yes'
  bucket.complete_multipart_upload(key, upload_id, None, headers=headers)

Annuler un événement de téléchargement multipart

Appelez bucket.abort_multipart_upload pour annuler un téléchargement multipart. Après l'annulation, l'uploadId devient invalide et les parties téléchargées sont supprimées.

# -*- coding: utf-8 -*-
import os
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

# Obtain access credentials from environment variables. Before running this code, make sure you have set the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Set Endpoint to the endpoint of the region where the bucket is located. For example, for a bucket in the China (Hangzhou) region, set Endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Set region to the region ID that corresponds to the endpoint, for example, cn-hangzhou. Note: This parameter is required for SignatureV4.
region = "cn-hangzhou"

# Set yourBucketName to the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# Set key to the full path of the object. The path cannot include the bucket name. Example: exampledir/exampleobject.txt.
key = 'exampledir/exampleobject.txt'
# Set upload_id. The upload_id is returned after you call InitiateMultipartUpload.
upload_id = 'yourUploadId'

# Cancel the multipart upload event for the specified upload_id. The uploaded parts will be deleted.
bucket.abort_multipart_upload(key, upload_id)

Répertorier les parties téléchargées

Le code suivant répertorie les parties téléchargées :

# -*- coding: utf-8 -*-
import os
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider

# Obtain access credentials from environment variables. Before running this code, make sure you have set the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Set Endpoint to the endpoint of the region where the bucket is located. For example, for a bucket in the China (Hangzhou) region, set Endpoint to https://oss-cn-hangzhou.aliyuncs.com.
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"

# Set region to the region ID that corresponds to the endpoint, for example, cn-hangzhou. Note: This parameter is required for SignatureV4.
region = "cn-hangzhou"

# Set yourBucketName to the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# Set key to the full path of the object. The path cannot include the bucket name. Example: exampledir/exampleobject.txt.
key = 'exampledir/exampleobject.txt'
# Set upload_id. Get the upload_id after you call InitiateMultipartUpload and before you call CompleteMultipartUpload.
upload_id = 'yourUploadId'

# List information about the parts uploaded with the specified upload_id.
for part_info in oss2.PartIterator(bucket, key, upload_id):
    print('part_number:', part_info.part_number)
    print('etag:', part_info.etag)
    print('size:', part_info.size)

Répertorier les événements de téléchargement multipart

• Répertorier les événements de téléchargement multipart pour un objet spécifique

  Le code suivant répertorie les événements de téléchargement multipart pour un objet spécifique :

  # -*- coding: utf-8 -*-
  import os
  import oss2
  from oss2.credentials import EnvironmentVariableCredentialsProvider
  
  # Obtain access credentials from environment variables. Before running this code, make sure you have set the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.
  auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
  
  # Set Endpoint to the endpoint of the region where the bucket is located. For example, for a bucket in the China (Hangzhou) region, set Endpoint to https://oss-cn-hangzhou.aliyuncs.com.
  endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
  
  # Set region to the region ID that corresponds to the endpoint, for example, cn-hangzhou. Note: This parameter is required for SignatureV4.
  region = "cn-hangzhou"
  
  # Set yourBucketName to the name of your bucket.
  bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)
  
  # Set key to the full path of the object. The path cannot include the bucket name. Example: exampledir/exampleobject.txt.
  key = 'exampledir/exampleobject.txt'
  
  # List all multipart upload events for the object. Each call to init_multipart_upload for the same object returns a different upload_id.
  # Each upload_id corresponds to one multipart upload event.
  for upload_info in oss2.ObjectUploadIterator(bucket, key):
      print('key:', upload_info.key)
      print('upload_id:', upload_info.upload_id)

• Répertorier tous les événements de téléchargement multipart dans un bucket

  Le code suivant répertorie tous les événements de téléchargement multipart dans un bucket :

  # -*- coding: utf-8 -*-
  import os
  import oss2
  from oss2.credentials import EnvironmentVariableCredentialsProvider
  
  # An AccessKey pair of an Alibaba Cloud account has permissions on all API operations. This poses a high security risk. We strongly recommend that you create and use a RAM user for API access or routine O&M. To create a RAM user, log on to the RAM console.
  auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
  
  # Set Endpoint to the endpoint of the region where the bucket is located. For example, for a bucket in the China (Hangzhou) region, set Endpoint to https://oss-cn-hangzhou.aliyuncs.com.
  endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
  
  # Set region to the region ID that corresponds to the endpoint, for example, cn-hangzhou. Note: This parameter is required for SignatureV4.
  region = "cn-hangzhou"
  
  # Set yourBucketName to the name of your bucket.
  bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)
  
  # List all multipart upload events in the bucket.
  for upload_info in oss2.MultipartUploadIterator(bucket):
      print('key:', upload_info.key)
      print('upload_id:', upload_info.upload_id)

• Répertorier les événements de téléchargement multipart par préfixe

  Le code suivant répertorie les événements de téléchargement multipart pour les objets ayant un préfixe spécifique :

  # -*- coding: utf-8 -*-
  import os
  import oss2
  from oss2.credentials import EnvironmentVariableCredentialsProvider
  # An AccessKey pair of an Alibaba Cloud account has permissions on all API operations. This poses a high security risk. We strongly recommend that you create and use a RAM user for API access or routine O&M. To create a RAM user, log on to the RAM console.
  auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
  
  # Set Endpoint to the endpoint of the region where the bucket is located. For example, for a bucket in the China (Hangzhou) region, set Endpoint to https://oss-cn-hangzhou.aliyuncs.com.
  endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
  
  # Set region to the region ID that corresponds to the endpoint, for example, cn-hangzhou. Note: This parameter is required for SignatureV4.
  region = "cn-hangzhou"
  
  # Set yourBucketName to the name of your bucket.
  bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)
  
  # List multipart upload events for objects in the bucket that have the 'test' prefix.
  for upload_info in oss2.MultipartUploadIterator(bucket, prefix='test'):
      print('key:', upload_info.key)
      print('upload_id:', upload_info.upload_id)

FAQ

Références

• Un téléchargement multipart implique trois opérations API :

  • Initialiser un téléchargement multipart : InitiateMultipartUpload.

  • Télécharger une partie : UploadPart.

  • Finaliser un téléchargement multipart : CompleteMultipartUpload.

• Annuler un téléchargement multipart : AbortMultipartUpload.

• Répertorier les parties téléchargées : ListParts.

• Répertorier les téléchargements multipart en cours : ListMultipartUploads.