Administrador de descarga de archivos (Python SDK V2) - Object Storage Service

Use el módulo Downloader de Python SDK V2 para descargar archivos de OSS a un dispositivo local.

Notas de uso

El código de ejemplo de este tema usa la región China (Hangzhou) con el ID de región cn-hangzhou como referencia. De forma predeterminada, se usa un endpoint público. Si accede a OSS desde otros servicios de Alibaba Cloud en la misma región, use un endpoint interno. Para obtener más información sobre las regiones y endpoints que admite OSS, consulte Regions and endpoints.
Para descargar un archivo, debe contar con el permiso oss:GetObject. Para obtener más información, consulte Attach a custom policy to a RAM user.

Definición de métodos

Funciones de Downloader

El módulo Downloader de Python SDK V2 simplifica la descarga de archivos al abstraer los detalles de implementación subyacentes.

El módulo Downloader divide automáticamente un archivo en partes más pequeñas y las descarga en paralelo, lo que mejora el rendimiento de la descarga.
El módulo Downloader también ofrece la función de descarga reanudable. El progreso de la descarga se registra; por lo tanto, si se interrumpe debido a fallos de red o salidas inesperadas del programa, puede reanudarla desde el último punto de interrupción.

A continuación se muestran los métodos comunes del módulo Downloader:

class Downloader:
  ...

def downloader(self, **kwargs) -> Downloader:
  ...

def download_file(self, request: models.GetObjectRequest, filepath: str, **kwargs: Any) -> DownloadResult:
  ...
  
def download_to(self, request: models.GetObjectRequest, writer: IO[bytes], **kwargs: Any) -> DownloadResult:
  ...

Parámetros de solicitud

Parámetro	Tipo	Descripción
request	GetObjectRequest	Los parámetros de solicitud para descargar un objeto. Son los mismos que los del método GetObject. Para obtener más información, consulte GetObjectRequest
filepath	str	La ruta del archivo local.
writer	IO[bytes]	El flujo de descarga.
**kwargs	Any	(Opcional) Parámetro arbitrario. Tipo: diccionario.

Parámetros de respuesta

Tipo	Descripción
DownloadResult	Los parámetros de respuesta para descargar un objeto. Para obtener más información, consulte DownloadResult

Puede especificar opciones de configuración al inicializar una instancia del descargador o para cada llamada de descarga. Por ejemplo, especifique el tamaño de parte de la siguiente manera:

Establezca los parámetros de configuración del descargador.
```
downloader = client.downloader(part_size=1024 * 1024)
```

Establezca los parámetros de configuración para cada solicitud de descarga.

result = downloader.download_file(oss.GetObjectRequest(
    bucket="example_bucket",
    key="example_key",
),
    filepath="/local/dir/example",
    part_size=10 * 1024 * 1024,
)

La siguiente tabla describe las opciones de configuración comunes.

Parámetro	Tipo	Descripción
part_size	int	Especifica el tamaño de parte. El valor predeterminado es 6 MiB.
parallel_num	int	El número de tareas de descarga simultáneas. Valor predeterminado: 3. Este es el límite de concurrencia para una sola llamada, no el límite global.
enable_checkpoint	bool	Especifica si se habilita la función de descarga reanudable. De forma predeterminada, esta función está deshabilitada.
checkpoint_dir	str	Especifica la ruta para guardar el archivo de registro. Ejemplo: /local/dir/. Este parámetro solo es válido cuando enable_checkpoint se establece en True.
verify_data	bool	Especifica si se comprueba el valor CRC-64 de los datos descargados al reanudar la descarga. De forma predeterminada, no se comprueba el valor. Este parámetro solo es válido cuando enable_checkpoint se establece en True.
use_temp_file	bool	Especifica si se utiliza un archivo temporal durante la descarga. Habilitado de forma predeterminada. Los datos se escriben primero en un archivo temporal y luego se cambia el nombre al archivo destino después de que la descarga finaliza correctamente.

Para obtener más información sobre la definición de métodos del administrador de descarga de archivos, consulte Downloader.

Código de ejemplo

Use el siguiente código para descargar un archivo de un bucket a un dispositivo local.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the purpose of the script: download file sample.
parser = argparse.ArgumentParser(description="download file sample")

# Add the command-line argument --region, which indicates the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the command-line argument --bucket, which indicates the name of the bucket from which you want to download the file. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the command-line argument --endpoint, which indicates the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the command-line argument --key, which indicates the key of the object (file) in OSS. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the command-line argument --file_path, which indicates the local path to save the downloaded file. This argument is required. For example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path to save the downloaded file.', required=True)

def main():
    # Parse the command-line arguments to obtain the values entered by the user.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configurations of the SDK to create a configuration object and set the authentication provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property of the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSS client to interact with OSS.
    client = oss.Client(cfg)

    # Create an object for downloading files.
    downloader = client.downloader()

    # Call the method to perform the file download operation.
    result = downloader.download_file(
        oss.GetObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the local path to save the downloaded file.
    )

    # Print information about the download result, including the number of bytes written.
    print(f'written: {result.written}')

# When this script is directly executed, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script, where the program flow starts.

Escenarios comunes

Uso del descargador para configurar el tamaño de parte y la concurrencia

Use el siguiente código para configurar el descargador y establecer el tamaño de parte y la concurrencia.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the purpose of the script: download file sample.
parser = argparse.ArgumentParser(description="download file sample")

# Add the command-line argument --region, which indicates the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the command-line argument --bucket, which indicates the name of the bucket from which you want to download the file. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the command-line argument --endpoint, which indicates the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the command-line argument --key, which indicates the key of the object (file) in OSS. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the command-line argument --file_path, which indicates the local path to save the downloaded file. This argument is required. For example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path to save the downloaded file.', required=True)

def main():
    # Parse the command-line arguments to obtain the values entered by the user.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configurations of the SDK to create a configuration object and set the authentication provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property of the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSS client to interact with OSS.
    client = oss.Client(cfg)

    # Create an object for downloading files and set advanced options.
    downloader = client.downloader(
        part_size=1024 * 1024,  # Set the size of each part to 1 MB.
        parallel_num=5,         # Set the number of concurrent download threads to 5.
        block_size=1024 * 1024  # Set the size of the data block read each time to 1 MB.
    )

    # Call the method to perform the file download operation.
    result = downloader.download_file(
        oss.GetObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the local path to save the downloaded file.
    )

    # Print information about the download result, including the number of bytes written.
    print(f'written: {result.written}')

# When this script is directly executed, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script, where the program flow starts.

Uso del descargador para habilitar la descarga reanudable

Use el siguiente código para configurar el descargador y habilitar la función de descarga reanudable.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the purpose of the script: download file sample.
parser = argparse.ArgumentParser(description="download file sample")

# Add the command-line argument --region, which indicates the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the command-line argument --bucket, which indicates the name of the bucket from which you want to download the file. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the command-line argument --endpoint, which indicates the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the command-line argument --key, which indicates the key of the object (file) in OSS. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the command-line argument --file_path, which indicates the local path to save the downloaded file. This argument is required. For example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path to save the downloaded file.', required=True)

def main():
    # Parse the command-line arguments to obtain the values entered by the user.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configurations of the SDK to create a configuration object and set the authentication provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property of the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSS client to interact with OSS.
    client = oss.Client(cfg)

    # Create an object for downloading files and set advanced options.
    downloader = client.downloader(
        use_temp_file=True,            # Use a temporary file.
        enable_checkpoint=True,        # Enable resumable download.
        checkpoint_dir=args.file_path, # The directory to save the resumable download record file.
        verify_data=True               # Specifies whether to verify data.
    )

    # Call the method to perform the file download operation.
    result = downloader.download_file(
        oss.GetObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the local path to save the downloaded file.
    )

    # Print information about the download result, including the number of bytes written.
    print(f'written: {result.written}')

# When this script is directly executed, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script, where the program flow starts.

Uso del descargador con visualización de barra de progreso

El siguiente código de ejemplo muestra cómo visualizar una barra de progreso al descargar un archivo con el descargador.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the purpose of the script: download file sample.
parser = argparse.ArgumentParser(description="download file sample")

# Add the command-line argument --region, which indicates the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the command-line argument --bucket, which indicates the name of the bucket from which you want to download the file. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the command-line argument --endpoint, which indicates the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the command-line argument --key, which indicates the key of the object (file) in OSS. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the command-line argument --file_path, which indicates the local path to save the downloaded file. This argument is required. For example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path to save the downloaded file.', required=True)

def main():
    # Parse the command-line arguments to obtain the values entered by the user.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configurations of the SDK to create a configuration object and set the authentication provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider

    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property of the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSS client to interact with OSS.
    client = oss.Client(cfg)

    # Create an object for downloading files.
    downloader = client.downloader()

    # Define a dictionary variable progress_state to save the download progress status. The initial value is 0.
    progress_state = {'saved': 0}

    # Define the progress callback function _progress_fn.
    def _progress_fn(n, written, total):
        # Use a dictionary to store the cumulative number of bytes written.
        progress_state['saved'] += n

        # Calculate the current download percentage. The value is obtained by dividing the number of written bytes by the total number of bytes and rounding down to the nearest integer.
        rate = int(100 * (float(written) / float(total)))

        # Print the current download progress. \r indicates returning to the beginning of the line to implement real-time refresh in the command line.
        # end='' indicates that no line break is added, so that the next print overwrites the current line.
        print(f'\rDownload progress: {rate}% ', end='')

    # Call the method to perform the file download operation.
    result = downloader.download_file(
        oss.GetObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
            progress_fn=_progress_fn,  # Set the progress callback function.
        ),
        filepath=args.file_path  # Specify the local path to save the downloaded file.
    )

    # Print information about the download result, including the number of bytes written.
    print(f'written: {result.written}')

# When this script is directly executed, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script, where the program flow starts.

Referencias

Para obtener más información sobre el administrador de descargas, consulte la Guía del desarrollador.
Para consultar el código de ejemplo completo del administrador de descargas, revise download_file.py.