How to download files by using OSS Python SDK V2 - Object Storage Service: Central de Documentação do Alibaba Cloud

Observações de uso

Os códigos de exemplo deste tópico usam a região China (Hangzhou), com o ID de região cn-hangzhou, como referência. Por padrão, o sistema usa um endpoint público. Se você acessar o OSS a partir de outros serviços da Alibaba Cloud na mesma região, use um endpoint interno. Para mais informações sobre as regiões e os endpoints compatíveis com o OSS, consulte Regiões e endpoints.
Para baixar um arquivo, você precisa da permissão oss:GetObject. Para mais detalhes, consulte Anexar uma política personalizada a um usuário RAM.

Definição do método

Recursos do Downloader

O módulo Downloader do Python SDK V2 simplifica o download de arquivos ao abstrair os detalhes de implementação subjacentes.

O módulo Downloader divide automaticamente um arquivo em partes menores e baixa-as em paralelo, melhorando o desempenho da transferência.
Além disso, o Downloader oferece o recurso de download retomável. O progresso é registrado para que, em caso de interrupção por falhas de rede ou encerramento inesperado do programa, seja possível retomar a partir do último ponto salvo.

O código a seguir apresenta os métodos comuns do módulo Downloader:

class Downloader:
  ...

def downloader(self, **kwargs) -> Downloader:
  ...

def download_file(self, request: models.GetObjectRequest, filepath: str, **kwargs: Any) -> DownloadResult:
  ...
  
def download_to(self, request: models.GetObjectRequest, writer: IO[bytes], **kwargs: Any) -> DownloadResult:
  ...

Parâmetros da solicitação

Parâmetro	Tipo	Descrição
request	GetObjectRequest	Parâmetros da solicitação para baixar um objeto. São idênticos aos do método GetObject. Para mais informações, consulte GetObjectRequest
filepath	str	Caminho do arquivo local.
writer	IO[bytes]	Fluxo de download.
**kwargs	Any	(Opcional) Parâmetro arbitrário. Tipo: dicionário

Parâmetros de resposta

Tipo	Descrição
DownloadResult	Parâmetros de resposta do download de um objeto. Para mais informações, consulte DownloadResult

Você pode especificar opções de configuração ao inicializar uma instância do gerenciador de download ou em cada chamada individual. Por exemplo, defina o tamanho da parte conforme abaixo:

Configure os parâmetros do gerenciador de download.

downloader = client.downloader(part_size=1024 * 1024)

Defina os parâmetros de configuração para cada solicitação de download.

result = downloader.download_file(oss.GetObjectRequest(
    bucket="example_bucket",
    key="example_key",
),
    filepath="/local/dir/example",
    part_size=10 * 1024 * 1024,
)

A tabela a seguir descreve as opções de configuração mais utilizadas.

Parâmetro	Tipo	Descrição
part_size	int	Define o tamanho da parte. O valor padrão é 6 MiB.
parallel_num	int	Quantidade de tarefas de download simultâneas. Valor padrão: 3. Este limite aplica-se apenas a uma única chamada, não sendo um limite global de concorrência.
enable_checkpoint	bool	Indica se o recurso de download retomável deve ser ativado. Por padrão, esse recurso permanece desativado.
checkpoint_dir	str	Especifica o caminho para salvar o arquivo de registro. Exemplo: /local/dir/. Este parâmetro só é válido quando enable_checkpoint está definido como True.
verify_data	bool	Determina se o valor CRC-64 dos dados baixados deve ser verificado ao retomar o download. Por padrão, a verificação não ocorre. Aplica-se apenas quando enable_checkpoint está definido como True.
use_temp_file	bool	Controla o uso de um arquivo temporário durante o download. Ativado por padrão. Os dados são gravados primeiro em um arquivo temporário e renomeados para o arquivo de destino após a conclusão bem-sucedida.

Para obter mais detalhes sobre a definição de métodos do gerenciador de download de arquivos, consulte Downloader.

Código de exemplo

Use o código abaixo para baixar um arquivo de um bucket para um dispositivo local.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the purpose of the script: download file sample.
parser = argparse.ArgumentParser(description="download file sample")

# Add the command-line argument --region, which indicates the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the command-line argument --bucket, which indicates the name of the bucket from which you want to download the file. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the command-line argument --endpoint, which indicates the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the command-line argument --key, which indicates the key of the object (file) in OSS. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the command-line argument --file_path, which indicates the local path to save the downloaded file. This argument is required. For example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path to save the downloaded file.', required=True)

def main():
    # Parse the command-line arguments to obtain the values entered by the user.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configurations of the SDK to create a configuration object and set the authentication provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property of the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSS client to interact with OSS.
    client = oss.Client(cfg)

    # Create an object for downloading files.
    downloader = client.downloader()

    # Call the method to perform the file download operation.
    result = downloader.download_file(
        oss.GetObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the local path to save the downloaded file.
    )

    # Print information about the download result, including the number of bytes written.
    print(f'written: {result.written}')

# When this script is directly executed, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script, where the program flow starts.

Cenários comuns

Usar o gerenciador de download para definir o tamanho da parte e a concorrência

Use o código a seguir para configurar o gerenciador de download, definindo o tamanho da parte e a concorrência.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the purpose of the script: download file sample.
parser = argparse.ArgumentParser(description="download file sample")

# Add the command-line argument --region, which indicates the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the command-line argument --bucket, which indicates the name of the bucket from which you want to download the file. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the command-line argument --endpoint, which indicates the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the command-line argument --key, which indicates the key of the object (file) in OSS. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the command-line argument --file_path, which indicates the local path to save the downloaded file. This argument is required. For example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path to save the downloaded file.', required=True)

def main():
    # Parse the command-line arguments to obtain the values entered by the user.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configurations of the SDK to create a configuration object and set the authentication provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property of the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSS client to interact with OSS.
    client = oss.Client(cfg)

    # Create an object for downloading files and set advanced options.
    downloader = client.downloader(
        part_size=1024 * 1024,  # Set the size of each part to 1 MB.
        parallel_num=5,         # Set the number of concurrent download threads to 5.
        block_size=1024 * 1024  # Set the size of the data block read each time to 1 MB.
    )

    # Call the method to perform the file download operation.
    result = downloader.download_file(
        oss.GetObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the local path to save the downloaded file.
    )

    # Print information about the download result, including the number of bytes written.
    print(f'written: {result.written}')

# When this script is directly executed, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script, where the program flow starts.

Usar o gerenciador de download para ativar o download retomável

O código abaixo demonstra como configurar o gerenciador de download para habilitar o recurso de download retomável.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the purpose of the script: download file sample.
parser = argparse.ArgumentParser(description="download file sample")

# Add the command-line argument --region, which indicates the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the command-line argument --bucket, which indicates the name of the bucket from which you want to download the file. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the command-line argument --endpoint, which indicates the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the command-line argument --key, which indicates the key of the object (file) in OSS. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the command-line argument --file_path, which indicates the local path to save the downloaded file. This argument is required. For example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path to save the downloaded file.', required=True)

def main():
    # Parse the command-line arguments to obtain the values entered by the user.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configurations of the SDK to create a configuration object and set the authentication provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property of the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSS client to interact with OSS.
    client = oss.Client(cfg)

    # Create an object for downloading files and set advanced options.
    downloader = client.downloader(
        use_temp_file=True,            # Use a temporary file.
        enable_checkpoint=True,        # Enable resumable download.
        checkpoint_dir=args.file_path, # The directory to save the resumable download record file.
        verify_data=True               # Specifies whether to verify data.
    )

    # Call the method to perform the file download operation.
    result = downloader.download_file(
        oss.GetObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the local path to save the downloaded file.
    )

    # Print information about the download result, including the number of bytes written.
    print(f'written: {result.written}')

# When this script is directly executed, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script, where the program flow starts.

Usar o gerenciador de download e exibir uma barra de progresso

Este exemplo ilustra como exibir uma barra de progresso durante o download de um arquivo com o gerenciador.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the purpose of the script: download file sample.
parser = argparse.ArgumentParser(description="download file sample")

# Add the command-line argument --region, which indicates the region where the bucket is located. This argument is required.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the command-line argument --bucket, which indicates the name of the bucket from which you want to download the file. This argument is required.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the command-line argument --endpoint, which indicates the domain name that other services can use to access OSS. This argument is optional.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the command-line argument --key, which indicates the key of the object (file) in OSS. This argument is required.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the command-line argument --file_path, which indicates the local path to save the downloaded file. This argument is required. For example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path to save the downloaded file.', required=True)

def main():
    # Parse the command-line arguments to obtain the values entered by the user.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the default configurations of the SDK to create a configuration object and set the authentication provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider

    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property of the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Use the preceding configurations to initialize the OSS client to interact with OSS.
    client = oss.Client(cfg)

    # Create an object for downloading files.
    downloader = client.downloader()

    # Define a dictionary variable progress_state to save the download progress status. The initial value is 0.
    progress_state = {'saved': 0}

    # Define the progress callback function _progress_fn.
    def _progress_fn(n, written, total):
        # Use a dictionary to store the cumulative number of bytes written.
        progress_state['saved'] += n

        # Calculate the current download percentage. The value is obtained by dividing the number of written bytes by the total number of bytes and rounding down to the nearest integer.
        rate = int(100 * (float(written) / float(total)))

        # Print the current download progress. \r indicates returning to the beginning of the line to implement real-time refresh in the command line.
        # end='' indicates that no line break is added, so that the next print overwrites the current line.
        print(f'\rDownload progress: {rate}% ', end='')

    # Call the method to perform the file download operation.
    result = downloader.download_file(
        oss.GetObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
            progress_fn=_progress_fn,  # Set the progress callback function.
        ),
        filepath=args.file_path  # Specify the local path to save the downloaded file.
    )

    # Print information about the download result, including the number of bytes written.
    print(f'written: {result.written}')

# When this script is directly executed, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script, where the program flow starts.

Referências

Para mais informações sobre o gerenciador de download, consulte o Guia do Desenvolvedor.
Para acessar o código de exemplo completo do gerenciador de download, veja o arquivo download_file.py.