Administrador de carga de archivos (Python SDK V2) - Object Storage Service

Este tema describe cómo utilizar el nuevo módulo Uploader en Python SDK V2 para cargar archivos.

Notas

El código de ejemplo de este tema utiliza el endpoint público de la región China (Hangzhou). El ID de región es cn-hangzhou. Si desea acceder a OSS desde otros servicios de Alibaba Cloud en la misma región, debe usar un endpoint interno. Para obtener más información sobre las regiones y endpoints que admite OSS, consulte OSS regions and endpoints.
Para realizar una carga, debe contar con el permiso oss:PutObject. Para obtener más información, consulte Grant custom permissions to a RAM user.

Definición del método

Introducción al administrador de carga

El nuevo módulo Uploader en Python SDK V2 proporciona un método de carga unificado que abstrae los detalles de implementación subyacentes para simplificar la carga de archivos.

El Uploader emplea el método de carga multiparte para dividir un archivo o flujo en varias partes y luego las carga de forma concurrente. Este proceso mejora el rendimiento de la carga.
El Uploader también ofrece una función de carga reanudable. Durante el proceso, el Uploader registra el estado de las partes completadas. Si la carga se interrumpe por problemas como un error de red o una salida inesperada del programa, puede reanudarla desde los puntos de interrupción registrados.

A continuación, la tabla describe los métodos comunes del Uploader.

class Uploader:
  ...

def uploader(self, **kwargs) -> Uploader:
  ...

def upload_file(self, request: models.PutObjectRequest, filepath: str, **kwargs: Any) -> UploadResult:
  ...
  
def upload_from(self, request: models.PutObjectRequest, reader: IO[bytes], **kwargs: Any) -> UploadResult:
  ...

Parámetros de solicitud

Parámetro	Tipo	Descripción
request	PutObjectRequest	Los parámetros de solicitud para cargar un objeto. Estos parámetros son los mismos que los del método PutObject. Para obtener más información, consulte PutObjectRequest
reader	IO[bytes]	El flujo de datos que se va a cargar
filepath	str	La ruta del archivo local
**kwargs	Any	(Opcional) Cualquier parámetro. El tipo es diccionario.

Parámetros de respuesta

Tipo	Descripción
UploadResult	Los parámetros de respuesta para cargar un objeto. Para obtener más información, consulte UploadResult

Cuando utiliza client.uploader para inicializar una instancia del administrador de carga, puede especificar opciones de configuración para personalizar el comportamiento de carga. También puede definir estas opciones en cada llamada a la API de carga para ajustar el comportamiento de un objeto específico. Por ejemplo, puede establecer el tamaño de parte.

Establezca los parámetros de configuración para el cargador
```
uploader = client.uploader(part_size=10  * 1024 * 1024)
```

Establezca los parámetros de configuración para cada solicitud de carga

result = uploader.upload_file(oss.PutObjectRequest(
        bucket="example_bucket",
        key="example_key",
    ),
    filepath="/local/dir/example",
    part_size=10 * 1024 * 1024,
)

A continuación, la tabla describe las opciones de configuración comunes.

Parámetro	Tipo	Descripción
part_size	int	Especifica el tamaño de parte. El valor predeterminado es 6 MiB.
parallel_num	int	Especifica el número de tareas de carga concurrentes. El valor predeterminado es 3. Este parámetro limita la concurrencia de una sola llamada, no la concurrencia global.
leave_parts_on_error	bool	Especifica si se conservan las partes cargadas cuando falla la carga. De forma predeterminada, las partes no se conservan.
enable_checkpoint	bool	Especifica si se habilita la carga reanudable. De forma predeterminada, esta función está deshabilitada. Nota El parámetro enable_checkpoint solo es válido para el método upload_file. El método upload_from no admite este parámetro.
checkpoint_dir	str	Especifica la ruta donde se guarda el archivo de registro, por ejemplo, /local/dir/. Este parámetro solo es válido cuando enable_checkpoint se establece en true.

Para consultar las definiciones completas de los métodos del administrador de carga de archivos, consulte Uploader.

Código de ejemplo

Puede utilizar el siguiente código para cargar un archivo local a un bucket mediante el administrador de carga.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files.
    uploader = client.uploader()

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Escenarios

Uso del administrador de carga para habilitar la carga reanudable

Puede utilizar el siguiente código para habilitar la carga reanudable.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files, enable resumable upload, and specify the path to save the breakpoint record file.
    uploader = client.uploader(enable_checkpoint=True, checkpoint_dir="/Users/yourLocalPath/checkpoint/")

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Uso del administrador de carga para cargar un flujo de archivo local

Puede utilizar el siguiente código para cargar un flujo de archivo local mediante el administrador de carga.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload from file sample
parser = argparse.ArgumentParser(description="upload from sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files.
    uploader = client.uploader()

    # Open the local file for reading in binary mode.
    with open(file=args.file_path, mode='rb') as f:
        # Call the method to perform the file upload operation.
        result = uploader.upload_from(
            oss.PutObjectRequest(
                bucket=args.bucket,  # Specify the destination bucket.
                key=args.key,        # Specify the name of the file in OSS.
            ),
            reader=f  # Pass in the file reader.
        )

        # Print information about the upload result, including the status code, request ID, and Content-MD5.
        print(f'status code: {result.status_code},'
              f' request id: {result.request_id},'
              f' content md5: {result.headers.get("Content-MD5")},'
              f' etag: {result.etag},'
              f' hash crc64: {result.hash_crc64},'
              f' version id: {result.version_id},'
              f' server time: {result.headers.get("x-oss-server-time")},'
              )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Uso del administrador de carga para establecer el tamaño de parte y la concurrencia

Puede utilizar el siguiente código para especificar el tamaño de parte y la concurrencia.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files and set the part size and concurrency.
    uploader = client.uploader(
        part_size=100 * 1024,  # Set the part size to 100 KB.
        parallel_num=5,        # Set the concurrency to 5.
        leave_parts_on_error=True  # Retain the uploaded parts in case of an error.
    )

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Uso del administrador de carga para configurar una devolución de llamada de carga

Si desea notificar a un servidor de aplicaciones después de cargar un archivo, utilice el siguiente código de ejemplo.

import argparse
import base64
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter.
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider

    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an uploader object for uploading files.
    uploader = client.uploader()

    # Define the webhook address.
    call_back_url = "http://www.example.com/callback"
    # Construct the callback parameter (callback): specify the webhook address and the request body for the callback, and encode them in Base64.
    callback=base64.b64encode(str('{\"callbackUrl\":\"' + call_back_url + '\",\"callbackBody\":\"bucket=${bucket}&object=${object}&my_var_1=${x:var1}&my_var_2=${x:var2}\"}').encode()).decode()
    # Construct the custom variable (callback-var) and encode it in Base64.
    callback_var=base64.b64encode('{\"x:var1\":\"value1\",\"x:var2\":\"value2\"}'.encode()).decode()

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
            callback=callback,
            callback_var=callback_var,
        ),
        filepath=args.file_path,  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Referencias

Para obtener más información sobre el administrador de carga, consulte la Guía del desarrollador.
Para ver ejemplos completos del administrador de carga, consulte upload_file.py y upload_from.py.