how to use oss python sdk v2 to upload files - Object Storage Service: Central de Documentação do Alibaba Cloud

Este tópico descreve como usar o novo módulo Uploader no Python SDK V2 para fazer upload de arquivos.

Observações

O código de exemplo neste tópico usa o endpoint público da região China (Hangzhou), com ID cn-hangzhou. Para acessar o OSS a partir de outros serviços da Alibaba Cloud na mesma região, use um endpoint interno. Para mais informações sobre as regiões e endpoints compatíveis com o OSS, consulte Regiões e endpoints do OSS.
Para fazer upload, você precisa da permissão oss:PutObject. Para mais detalhes, consulte Conceder permissões personalizadas a um usuário RAM.

Definição dos métodos

Introdução ao gerenciador de upload

O novo módulo Uploader do Python SDK V2 oferece um método unificado de upload que abstrai os detalhes de implementação subjacentes para simplificar o envio de arquivos.

O Uploader usa o método de upload multipart para dividir um arquivo ou stream em várias partes e enviá-las simultaneamente. Esse processo melhora o desempenho do upload.
O Uploader também fornece o recurso de upload retomável. Durante a transferência, o Uploader registra o status das partes concluídas. Se o upload for interrompido por problemas como falha de rede ou encerramento inesperado do programa, retome a operação a partir dos pontos de interrupção registrados.

A tabela a seguir descreve os métodos comuns do Uploader.

class Uploader:
  ...

def uploader(self, **kwargs) -> Uploader:
  ...

def upload_file(self, request: models.PutObjectRequest, filepath: str, **kwargs: Any) -> UploadResult:
  ...
  
def upload_from(self, request: models.PutObjectRequest, reader: IO[bytes], **kwargs: Any) -> UploadResult:
  ...

Parâmetros da solicitação

Parâmetro	Tipo	Descrição
request	PutObjectRequest	Os parâmetros da solicitação para fazer upload de um objeto. São idênticos aos do método PutObject. Para mais informações, consulte PutObjectRequest
reader	IO[bytes]	O stream de dados a ser enviado
filepath	str	O caminho do arquivo local
**kwargs	Any	(Opcional) Qualquer parâmetro. O tipo é dicionário.

Parâmetros de resposta

Tipo	Descrição
UploadResult	Os parâmetros de resposta do upload de um objeto. Para mais detalhes, consulte UploadResult

Ao usar client.uploader para inicializar uma instância do gerenciador de upload, especifique opções de configuração para personalizar o comportamento do upload. Também é possível definir essas opções em cada chamada de API de upload para ajustar o comportamento de um objeto específico. Por exemplo, defina o tamanho da parte.

Defina os parâmetros de configuração do uploader

uploader = client.uploader(part_size=10  * 1024 * 1024)

Configure os parâmetros para cada solicitação de upload

result = uploader.upload_file(oss.PutObjectRequest(
        bucket="example_bucket",
        key="example_key",
    ),
    filepath="/local/dir/example",
    part_size=10 * 1024 * 1024,
)

A tabela abaixo lista as opções de configuração mais utilizadas.

Parâmetro	Tipo	Descrição
part_size	int	Define o tamanho da parte. O valor padrão é 6 MiB.
parallel_num	int	Indica o número de tarefas de upload simultâneas. O valor padrão é 3. Este parâmetro limita a concorrência de uma única chamada, não a concorrência global.
leave_parts_on_error	bool	Determina se as partes já enviadas devem ser mantidas quando o upload falhar. Por padrão, as partes não são retidas.
enable_checkpoint	bool	Ativa ou desativa o upload retomável. Por padrão, esse recurso está desativado. Nota O parâmetro enable_checkpoint é válido apenas para o método upload_file. O método upload_from não oferece suporte a este parâmetro.
checkpoint_dir	str	Especifica o caminho onde o arquivo de registro será salvo, por exemplo, /local/dir/. Este parâmetro só é válido quando enable_checkpoint está definido como true.

Para consultar as definições completas dos métodos do gerenciador de upload de arquivos, acesse Uploader.

Código de exemplo

Use o código abaixo para enviar um arquivo local para um bucket por meio do gerenciador de upload.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files.
    uploader = client.uploader()

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Cenários

Usar o gerenciador de upload para ativar o upload retomável

Execute o código a seguir para habilitar o upload retomável.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files, enable resumable upload, and specify the path to save the breakpoint record file.
    uploader = client.uploader(enable_checkpoint=True, checkpoint_dir="/Users/yourLocalPath/checkpoint/")

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Usar o gerenciador de upload para enviar um stream de arquivo local

O exemplo abaixo demonstra como fazer upload de um stream de arquivo local usando o gerenciador de upload.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload from file sample
parser = argparse.ArgumentParser(description="upload from sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files.
    uploader = client.uploader()

    # Open the local file for reading in binary mode.
    with open(file=args.file_path, mode='rb') as f:
        # Call the method to perform the file upload operation.
        result = uploader.upload_from(
            oss.PutObjectRequest(
                bucket=args.bucket,  # Specify the destination bucket.
                key=args.key,        # Specify the name of the file in OSS.
            ),
            reader=f  # Pass in the file reader.
        )

        # Print information about the upload result, including the status code, request ID, and Content-MD5.
        print(f'status code: {result.status_code},'
              f' request id: {result.request_id},'
              f' content md5: {result.headers.get("Content-MD5")},'
              f' etag: {result.etag},'
              f' hash crc64: {result.hash_crc64},'
              f' version id: {result.version_id},'
              f' server time: {result.headers.get("x-oss-server-time")},'
              )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Usar o gerenciador de upload para definir o tamanho da parte e a concorrência

No código seguinte, configure o tamanho da parte e o nível de concorrência desejados.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files and set the part size and concurrency.
    uploader = client.uploader(
        part_size=100 * 1024,  # Set the part size to 100 KB.
        parallel_num=5,        # Set the concurrency to 5.
        leave_parts_on_error=True  # Retain the uploaded parts in case of an error.
    )

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Usar o gerenciador de upload para configurar um callback de upload

Para notificar um servidor de aplicação após o upload de um arquivo, use o exemplo de código abaixo.

import argparse
import base64
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter.
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider

    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an uploader object for uploading files.
    uploader = client.uploader()

    # Define the webhook address.
    call_back_url = "http://www.example.com/callback"
    # Construct the callback parameter (callback): specify the webhook address and the request body for the callback, and encode them in Base64.
    callback=base64.b64encode(str('{\"callbackUrl\":\"' + call_back_url + '\",\"callbackBody\":\"bucket=${bucket}&object=${object}&my_var_1=${x:var1}&my_var_2=${x:var2}\"}').encode()).decode()
    # Construct the custom variable (callback-var) and encode it in Base64.
    callback_var=base64.b64encode('{\"x:var1\":\"value1\",\"x:var2\":\"value2\"}'.encode()).decode()

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
            callback=callback,
            callback_var=callback_var,
        ),
        filepath=args.file_path,  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Referências

Para saber mais sobre o gerenciador de upload, consulte o Guia do Desenvolvedor.
Os exemplos completos do gerenciador de upload estão disponíveis em upload_file.py e upload_from.py.