Object Storage Service:Multipart upload (C++ SDK)

Observações

• Neste tópico, o endpoint público da região China (Hangzhou) é utilizado como exemplo. Para acessar o OSS a partir de outros services da Alibaba Cloud na mesma região, use um endpoint interno. Para mais informações sobre regiões e endpoints do OSS, consulte Regions and endpoints.

• Este tópico demonstra a criação de uma instância OSSClient com um endpoint do OSS. Para configurações alternativas — como uso de domínio personalizado ou autenticação com credenciais do Security Token Service (STS) — consulte Create an OssClient instance.

• Para executar o processo completo de upload multipart — que inclui as operações InitiateMultipartUpload, UploadPart e CompleteMultipartUpload — é necessário ter a permissão oss:PutObject. Para mais informações, consulte Grant custom permissions to a RAM user.

Procedimento

O upload multipart envolve três etapas:

1. Inicie o upload multipart.

   Chame a operação InitiateMultipartUpload para obter um ID de upload globalmente único junto ao OSS.

2. Faça o upload das partes.

   Chame a operação UploadPart para enviar os dados de cada parte.

   Nota

   • O número da parte identifica sua posição dentro do objeto. Enviar dados com o mesmo número de parte sobrescreve o conteúdo existente.

   • O OSS retorna o hash MD5 dos dados da parte recebida no cabeçalho ETag da resposta.

   • O OSS calcula o hash MD5 dos dados enviados e o compara com o hash calculado pelo SDK. Em caso de divergência, o código de erro InvalidDigest é retornado.

3. Conclua o upload multipart.

   Após o envio de todas as partes, chame a operação CompleteMultipartUpload para combiná-las em um objeto completo.

Código de exemplo para upload multipart

O exemplo a seguir ilustra o processo completo de upload multipart:

#include <alibabacloud/oss/OssClient.h>
#include <fstream>

int64_t getFileSize(const std::string& file)
{
    std::fstream f(file, std::ios::in | std::ios::binary);
    f.seekg(0, f.end);
    int64_t size = f.tellg();
    f.close();
    return size;
}

using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize the OSS account information. */
    
    /* Set Endpoint to the endpoint of the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Set Region to the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set Region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the bucket name. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. The full path cannot contain the bucket name. Example: exampledir/exampleobject.txt. */
    std::string ObjectName = "exampledir/exampleobject.txt";

    /* Initialize network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);
  
    InitiateMultipartUploadRequest initUploadRequest(BucketName, ObjectName);
    /* (Optional) Set the storage class as shown in the following example. */
    //initUploadRequest.MetaData().addHeader("x-oss-storage-class", "Standard");

    /* Initiate the multipart upload event. */
    auto uploadIdResult = client.InitiateMultipartUpload(initUploadRequest);
    /* You can cancel the multipart upload event or list uploaded parts based on the upload ID. */
    /* To cancel a multipart upload event based on the upload ID, obtain the upload ID after you call InitiateMultipartUpload to initiate the event. */
    /* To list uploaded parts based on the upload ID, obtain the upload ID after you call InitiateMultipartUpload but before you call CompleteMultipartUpload. */ 
    auto uploadId = uploadIdResult.result().UploadId();
    std::string fileToUpload = "yourLocalFilename";
    /* The minimum part size is 100 KB and the maximum part size is 5 GB. The size of the last part can be less than 100 KB. */
    int64_t partSize = 100 * 1024;
    PartList partETagList;
    auto fileSize = getFileSize(fileToUpload);
    int partCount = static_cast<int>(fileSize / partSize);
    /* Calculate the number of parts. */
    if (fileSize % partSize != 0) {
        partCount++;
    }

    /* Upload each part. */
    for (int i = 1; i <= partCount; i++) {
        auto skipBytes = partSize * (i - 1);
        auto size = (partSize < fileSize - skipBytes) ? partSize : (fileSize - skipBytes);
        std::shared_ptr<std::iostream> content = std::make_shared<std::fstream>(fileToUpload, std::ios::in|std::ios::binary);
        content->seekg(skipBytes, std::ios::beg);

        UploadPartRequest uploadPartRequest(BucketName, ObjectName, content);
        uploadPartRequest.setContentLength(size);
        uploadPartRequest.setUploadId(uploadId);
        uploadPartRequest.setPartNumber(i);
        auto uploadPartOutcome = client.UploadPart(uploadPartRequest);
        if (uploadPartOutcome.isSuccess()) {
            Part part(i, uploadPartOutcome.result().ETag());
            partETagList.push_back(part);
        }
        else {
            std::cout << "uploadPart fail" <<
            ",code:" << uploadPartOutcome.error().Code() <<
            ",message:" << uploadPartOutcome.error().Message() <<
            ",requestId:" << uploadPartOutcome.error().RequestId() << std::endl;
        }

    }

    /* Complete the multipart upload. */
    /* When you complete the multipart upload, you must provide all valid partETags. After OSS receives the partETags, OSS verifies each part. After all parts are verified, OSS combines them into a complete object. */
    CompleteMultipartUploadRequest request(BucketName, ObjectName);
    request.setUploadId(uploadId);
    request.setPartList(partETagList);
    /* (Optional) Set the access control list (ACL) as shown in the following example. */
    //request.setAcl(CannedAccessControlList::Private);

    auto outcome = client.CompleteMultipartUpload(request);

    if (!outcome.isSuccess()) {
        /* Handle the exception. */
        std::cout << "CompleteMultipartUpload fail" <<
        ",code:" << outcome.error().Code() <<
        ",message:" << outcome.error().Message() <<
        ",requestId:" << outcome.error().RequestId() << std::endl;
        return -1;
    }

    /* Release network resources. */
    ShutdownSdk();
    return 0;
}

Ao chamar a operação CompleteMultipartUpload, forneça o valor ETag de cada parte. Esse valor pode ser obtido de duas formas:

• Durante o upload de uma parte, a resposta inclui o valor ETag correspondente. Salve e utilize esse valor conforme necessário. Esse método é adotado no código de exemplo anterior.

• Chame a operação ListParts para consultar os valores ETag de todas as partes já enviadas.

Listar partes enviadas

Chame a operação ListParts para listar todas as partes enviadas com sucesso para um ID de upload específico.

#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize the OSS account information. */
    
    /* Set Endpoint to the endpoint of the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Set Region to the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set Region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the bucket name. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. The full path cannot contain the bucket name. Example: exampledir/exampleobject.txt. */
    std::string ObjectName = "exampledir/exampleobject.txt";
    /* Specify the upload ID. The upload ID is obtained from the response to the InitiateMultipartUpload operation. You must obtain the upload ID before you call the CompleteMultipartUpload operation. */
    std::string UploadId = "yourUploadId";

    /* Initialize network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);

    /* List uploaded parts. By default, a maximum of 1,000 parts are listed. */
    ListPartsRequest listuploadrequest(BucketName, ObjectName);
    listuploadrequest.setUploadId(UploadId);

    bool IsTruncated = false;

    do {
        auto listUploadResult = client.ListParts(listuploadrequest);
        if (!listUploadResult.isSuccess()) {
            /* Handle the exception. */
            std::cout << "ListParts fail" <<
            ",code:" << listUploadResult.error().Code() <<
            ",message:" << listUploadResult.error().Message() <<
            ",requestId:" << listUploadResult.error().RequestId() << std::endl;
            break;
        }
        else {
            for (const auto& part : listUploadResult.result().PartList()) {
                std::cout << "part"<<
                ",name:" << part.PartNumber() <<
                ",size:" << part.Size() <<
                ",etag:" << part.ETag() <<
                ",lastmodify time:" << part.LastModified() << std::endl;
            }
        }
        listuploadrequest.setPartNumberMarker(listUploadResult.result().NextPartNumberMarker());
        IsTruncated = listUploadResult.result().IsTruncated();
    } while (IsTruncated);

    /* Release network resources. */
    ShutdownSdk();
    return 0;
}

#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize the OSS account information. */
    
    /* Set Endpoint to the endpoint of the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Set Region to the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set Region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the bucket name. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. The full path cannot contain the bucket name. Example: exampledir/exampleobject.txt. */
    std::string ObjectName = "exampledir/exampleobject.txt";
    /* Specify the upload ID. The upload ID is obtained from the response to the InitiateMultipartUpload operation. You must obtain the upload ID before you call the CompleteMultipartUpload operation. */
    std::string UploadId = "yourUploadId";

    /* Initialize network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);

    /* List all uploaded parts by page. */
    /* Set the maximum number of parts to return on each page. */
    ListPartsRequest listuploadrequest(BucketName, ObjectName);
    listuploadrequest.setMaxParts(50);
    listuploadrequest.setUploadId(UploadId);
    bool IsTruncated = false;
    do {
        auto listuploadresult = client.ListParts(listuploadrequest);
        if (!listuploadresult.isSuccess()) {
            /* Handle the exception. */
            std::cout << "ListParts fail" <<
            ",code:" << listuploadresult.error().Code() <<
            ",message:" << listuploadresult.error().Message() <<
            ",requestId:" << listuploadresult.error().RequestId() << std::endl;
            break;
        }
        else {
            for (const auto& part : listuploadresult.result().PartList()) {
                std::cout << "part"<<
                ",name:" << part.PartNumber() <<
                ",size:" << part.Size() <<
                ",etag:" << part.ETag() <<
                ",lastmodify time:" << part.LastModified() << std::endl;
            }
        }  
        listuploadrequest.setPartNumberMarker(listuploadresult.result().NextPartNumberMarker());
        IsTruncated = listuploadresult.result().IsTruncated();
    } while (IsTruncated);

    /* Release network resources. */
    ShutdownSdk();
    return 0;
}

Listar eventos de upload multipart

Chame a operação ListMultipartUploads para listar todos os eventos de upload multipart em andamento. Um evento em andamento é aquele que foi iniciado, mas ainda não foi concluído nem cancelado.

#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize the OSS account information. */
    
    /* Set Endpoint to the endpoint of the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Set Region to the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set Region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the bucket name. Example: examplebucket. */
    std::string BucketName = "examplebucket";    

    /* Initialize network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);

    /* List upload events. By default, a maximum of 1,000 events are listed. */
    ListMultipartUploadsRequest listmultiuploadrequest(BucketName);
	bool IsTruncated = false;
    do {
        auto listresult = client.ListMultipartUploads(listmultiuploadrequest);
        if (!listresult.isSuccess()) {
            /* Handle the exception. */
            std::cout << "ListMultipartUploads fail" <<
            ",code:" << listresult.error().Code() <<
            ",message:" << listresult.error().Message() <<
            ",requestId:" << listresult.error().RequestId() << std::endl;
            break;
        }
        else {
            for (const auto& part : listresult.result().MultipartUploadList()) {
                std::cout << "part"<<
                ",name:" << part.Key <<
                ",uploadid:" << part.UploadId <<
                ",initiated time:" << part.Initiated << std::endl;
            }
        }
        listmultiuploadrequest.setKeyMarker(listresult.result().NextKeyMarker()); 
        listmultiuploadrequest.setUploadIdMarker(listresult.result().NextUploadIdMarker());
        IsTruncated = listresult.result().IsTruncated();
    } while (IsTruncated);

    /* Release network resources. */
    ShutdownSdk();
    return 0;
}

#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize the OSS account information. */
    
    /* Set Endpoint to the endpoint of the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Set Region to the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set Region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the bucket name. Example: examplebucket. */
    std::string BucketName = "examplebucket"; 

    /* Initialize network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);

    /* List all upload events by page. */
    /* Set the maximum number of upload events to return on each page. */
    ListMultipartUploadsRequest  listmultiuploadrequest(BucketName);
    listmultiuploadrequest.setMaxUploads(50);
    bool IsTruncated = false;
    do {
        auto listresult = client.ListMultipartUploads(listmultiuploadrequest);
        if (!listresult.isSuccess()) {
            /* Handle the exception. */
            std::cout << "ListMultipartUploads fail" <<
            ",code:" << listresult.error().Code() <<
            ",message:" << listresult.error().Message() <<
            ",requestId:" << listresult.error().RequestId() << std::endl;
            break;
        }
        else {
            for (const auto& part : listresult.result().MultipartUploadList()) {
                std::cout << "part"<<
                ",name:" << part.Key <<
                ",uploadid:" << part.UploadId <<
                ",initiated time:" << part.Initiated << std::endl;
            }
        }  
        listmultiuploadrequest.setKeyMarker(listresult.result().NextKeyMarker()); 
        listmultiuploadrequest.setUploadIdMarker(listresult.result().NextUploadIdMarker());
        IsTruncated = listresult.result().IsTruncated();
    } while (IsTruncated);

    /* Release network resources. */
    ShutdownSdk();
    return 0;
}

Cancelar um evento de upload multipart

Chame a operação client.AbortMultipartUpload para cancele um evento de upload multipart. Após o cancelamento, o ID de upload não pode mais ser utilizado em nenhuma operação, e as partes já enviadas são excluídas.

#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize the OSS account information. */
    
    /* Set Endpoint to the endpoint of the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Set Region to the region in which the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set Region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Specify the bucket name. Example: examplebucket. */
    std::string BucketName = "examplebucket";
    /* Specify the full path of the object. The full path cannot contain the bucket name. Example: exampledir/exampleobject.txt. */
    std::string ObjectName = "exampledir/exampleobject.txt";
    /* Specify the upload ID. The upload ID is obtained from the response to the InitiateMultipartUpload operation. */
    std::string UploadId = "yourUploadId";

    /* Initialize network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region);
  
    InitiateMultipartUploadRequest initUploadRequest(BucketName, ObjectName);

    /* Initiate the multipart upload event. */
    auto uploadIdResult = client.InitiateMultipartUpload(initUploadRequest);
    auto uploadId = uploadIdResult.result().UploadId();

    /* Cancel the multipart upload event. */
    AbortMultipartUploadRequest  abortUploadRequest(BucketName, ObjectName, uploadId);
    auto outcome = client.AbortMultipartUpload(abortUploadRequest);

    if (!outcome.isSuccess()) {
        /* Handle the exception. */
        std::cout << "AbortMultipartUpload fail" <<
        ",code:" << outcome.error().Code() <<
        ",message:" << outcome.error().Message() <<
        ",requestId:" << outcome.error().RequestId() << std::endl;
        return -1;
    }

    /* Release network resources. */
    ShutdownSdk();
    return 0;
}

Referências

• Para acessar o código de exemplo completo do upload multipart, consulte o exemplo no GitHub.

• O upload multipart envolve três operações de API:

  • Iniciar um upload multipart: InitiateMultipartUpload.

  • Enviar uma parte: UploadPart.

  • Concluir um upload multipart: CompleteMultipartUpload.

• Para mais informações sobre a operação de API usada para cancelar um evento de upload multipart, consulte AbortMultipartUpload.

• Para mais informações sobre a operação de API usada para listar as partes enviadas, consulte ListUploadedParts.

• Para mais informações sobre a operação de API usada para listar todos os eventos de upload multipart em andamento (iniciados, mas ainda não concluídos nem cancelados), consulte ListMultipartUploads.