how to use oss python sdk v2 to upload files - Object Storage Service: Central de Documentação do Alibaba Cloud

Este tópico descreve como usar o novo módulo Uploader no Python SDK V2 para fazer upload de arquivos.

Observações

O código de exemplo neste tópico usa o endpoint público da região China (Hangzhou). O ID da região é cn-hangzhou. Se você quiser acessar o OSS a partir de outros services da Alibaba Cloud na mesma região, use um endpoint interno. Para obter mais informações sobre as regiões e os endpoints compatíveis com o OSS, consulte OSS regions and endpoints.
Para realizar um upload, você precisa ter a permissão oss:PutObject. Para obter mais informações, consulte Grant custom permissions to a RAM user.

Definição do método

Introdução ao gerenciador de upload

O novo módulo Uploader no Python SDK V2 oferece um método de upload unificado que abstrai os detalhes de implementação subjacentes para simplificar o upload de arquivos.

O Uploader utiliza o método de upload multipart para dividir um arquivo ou stream em várias partes e, em seguida, faz o upload dessas partes simultaneamente. Esse processo melhora o desempenho do upload.
O Uploader também oferece o recurso de upload retomável. Durante o upload, o Uploader registra o status das partes concluídas. Se o upload for interrompido por um problema, como um erro de rede ou o encerramento inesperado do programa, você poderá retomar o upload a partir dos pontos de interrupção registrados.

A tabela a seguir descreve os métodos comuns do Uploader.

class Uploader:
  ...

def uploader(self, **kwargs) -> Uploader:
  ...

def upload_file(self, request: models.PutObjectRequest, filepath: str, **kwargs: Any) -> UploadResult:
  ...
  
def upload_from(self, request: models.PutObjectRequest, reader: IO[bytes], **kwargs: Any) -> UploadResult:
  ...

Parâmetros da requisição

Parameter	Type	Description
request	PutObjectRequest	Os parâmetros da requisição para fazer upload de um objeto. Esses parâmetros são os mesmos do método PutObject. Para obter mais informações, consulte PutObjectRequest
reader	IO[bytes]	O stream de dados a ser enviado
filepath	str	O caminho do arquivo local
**kwargs	Any	(Opcional) Qualquer parâmetro. O tipo é dicionário.

Parâmetros de resposta

Type	Description
UploadResult	Os parâmetros de resposta do upload de um objeto. Para obter mais informações, consulte UploadResult

Ao usar client.uploader para inicializar uma instância do gerenciador de upload, especifique opções de configuração para personalizar o comportamento do upload. Defina essas opções também para cada chamada de API de upload a fim de personalizar o comportamento para o upload de um objeto específico. Por exemplo, especifique o tamanho da parte.

Defina os parâmetros de configuração do uploader

uploader = client.uploader(part_size=10  * 1024 * 1024)

Defina os parâmetros de configuração para cada requisição de upload

result = uploader.upload_file(oss.PutObjectRequest(
        bucket="example_bucket",
        key="example_key",
    ),
    filepath="/local/dir/example",
    part_size=10 * 1024 * 1024,
)

A tabela a seguir descreve as opções de configuração comuns.

Parâmetro	Tipo	Descrição
part_size	int	Especifique o tamanho da parte. O valor padrão é 6 MiB.
parallel_num	int	Especifique o número de tarefas de upload simultâneas. O valor padrão é 3. Este parâmetro limita a simultaneidade para uma única chamada, não a simultaneidade global.
leave_parts_on_error	bool	Especifique se as partes enviadas devem ser retidas quando o upload falhar. Por padrão, as partes não são retidas.
enable_checkpoint	bool	ative o upload retomável. Por padrão, esse recurso está desativado. Nota O parâmetro enable_checkpoint é válido apenas para o método upload_file. O método upload_from não oferece suporte a este parâmetro.
checkpoint_dir	str	Especifique o caminho e salve o arquivo de registro, por exemplo, em /local/dir/. Este parâmetro é válido apenas quando enable_checkpoint é definido como true.

Para obter as definições completas dos métodos do gerenciador de upload de arquivos, consulte Uploader.

Código de exemplo

Use o código a seguir para fazer upload de um arquivo local para um bucket usando o gerenciador de upload.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files.
    uploader = client.uploader()

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Cenários

Ative o upload retomável usando o gerenciador de upload

Use o código a seguir e ative o upload retomável.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files, enable resumable upload, and specify the path to save the breakpoint record file.
    uploader = client.uploader(enable_checkpoint=True, checkpoint_dir="/Users/yourLocalPath/checkpoint/")

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Faça upload de um stream de arquivo local usando o gerenciador de upload

Use o código a seguir para fazer upload de um stream de arquivo local com o gerenciador de upload.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload from file sample
parser = argparse.ArgumentParser(description="upload from sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files.
    uploader = client.uploader()

    # Open the local file for reading in binary mode.
    with open(file=args.file_path, mode='rb') as f:
        # Call the method to perform the file upload operation.
        result = uploader.upload_from(
            oss.PutObjectRequest(
                bucket=args.bucket,  # Specify the destination bucket.
                key=args.key,        # Specify the name of the file in OSS.
            ),
            reader=f  # Pass in the file reader.
        )

        # Print information about the upload result, including the status code, request ID, and Content-MD5.
        print(f'status code: {result.status_code},'
              f' request id: {result.request_id},'
              f' content md5: {result.headers.get("Content-MD5")},'
              f' etag: {result.etag},'
              f' hash crc64: {result.hash_crc64},'
              f' version id: {result.version_id},'
              f' server time: {result.headers.get("x-oss-server-time")},'
              )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Defina o tamanho da parte e a simultaneidade usando o gerenciador de upload

Especifique o tamanho da parte e a simultaneidade com o código a seguir.

import argparse
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter, for example, "/Users/yourLocalPath/yourFileName".
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    
    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an object for uploading files and set the part size and concurrency.
    uploader = client.uploader(
        part_size=100 * 1024,  # Set the part size to 100 KB.
        parallel_num=5,        # Set the concurrency to 5.
        leave_parts_on_error=True  # Retain the uploaded parts in case of an error.
    )

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
        ),
        filepath=args.file_path  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Configure um callback de upload usando o gerenciador de upload

Para notificar um servidor de aplicativo após o upload de um arquivo, use o código de exemplo a seguir.

import argparse
import base64
import alibabacloud_oss_v2 as oss

# Create a command-line argument parser and describe the script's purpose: upload file sample
parser = argparse.ArgumentParser(description="upload file sample")

# Add the --region command-line argument, which specifies the region where the bucket is located. This is a required parameter.
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
# Add the --bucket command-line argument, which specifies the name of the bucket to which the file is uploaded. This is a required parameter.
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
# Add the --endpoint command-line argument, which specifies the domain name that other services can use to access OSS. This is an optional parameter.
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS')
# Add the --key command-line argument, which specifies the key of the object (file) in OSS. This is a required parameter.
parser.add_argument('--key', help='The name of the object.', required=True)
# Add the --file_path command-line argument, which specifies the path of the local file to be uploaded. This is a required parameter.
parser.add_argument('--file_path', help='The path of Upload file.', required=True)

def main():
    # Parse the command-line arguments to obtain the user-provided values.
    args = parser.parse_args()

    # Load the authentication information required to access OSS from environment variables for identity verification.
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Create a configuration object using the default configurations of the SDK and set the credentials provider.
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider

    # Set the region property of the configuration object based on the command-line arguments provided by the user.
    cfg.region = args.region

    # If a custom endpoint is provided, update the endpoint property in the configuration object.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    # Initialize the OSS client using the preceding configurations to prepare for interaction with OSS.
    client = oss.Client(cfg)

    # Create an uploader object for uploading files.
    uploader = client.uploader()

    # Define the webhook address.
    call_back_url = "http://www.example.com/callback"
    # Construct the callback parameter (callback): specify the webhook address and the request body for the callback, and encode them in Base64.
    callback=base64.b64encode(str('{\"callbackUrl\":\"' + call_back_url + '\",\"callbackBody\":\"bucket=${bucket}&object=${object}&my_var_1=${x:var1}&my_var_2=${x:var2}\"}').encode()).decode()
    # Construct the custom variable (callback-var) and encode it in Base64.
    callback_var=base64.b64encode('{\"x:var1\":\"value1\",\"x:var2\":\"value2\"}'.encode()).decode()

    # Call the method to perform the file upload operation.
    result = uploader.upload_file(
        oss.PutObjectRequest(
            bucket=args.bucket,  # Specify the destination bucket.
            key=args.key,        # Specify the name of the file in OSS.
            callback=callback,
            callback_var=callback_var,
        ),
        filepath=args.file_path,  # Specify the location of the local file.
    )

    # Print information about the upload result, including the status code, request ID, and Content-MD5.
    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
          f' content md5: {result.headers.get("Content-MD5")},'
          f' etag: {result.etag},'
          f' hash crc64: {result.hash_crc64},'
          f' version id: {result.version_id},'
          f' server time: {result.headers.get("x-oss-server-time")},'
          )

# When this script is executed directly, call the main function to start the processing logic.
if __name__ == "__main__":
    main()  # The entry point of the script. The program flow starts from here.

Referências

Para obter mais informações sobre o gerenciador de upload, consulte Guia do Desenvolvedor.
Para ver exemplos completos do gerenciador de upload, consulte upload_file.py e upload_from.py.