O Object Storage Service (OSS) oferece o recurso de multipart upload para objetos grandes. Esse processo divide o objeto em partes menores, envia cada uma independentemente e chama a API CompleteMultipartUpload para combinar as partes em um único objeto, permitindo retomar uploads interrompidos.
Observações
Este tópico utiliza o endpoint público da região China (Hangzhou). Para acessar o OSS a partir de outros serviços da Alibaba Cloud na mesma região, use um endpoint interno. Para obter mais informações sobre regiões e endpoints do OSS, consulte Regiões e endpoints.
Este tópico demonstra a criação de uma instância OSSClient com um endpoint do OSS. Para configurações alternativas, como uso de domínio personalizado ou autenticação com credenciais do Security Token Service (STS), consulte Inicialização (C SDK).
O multipart upload utiliza as operações InitiateMultipartUpload, UploadPart e CompleteMultipartUpload. É necessária a permissão
oss:PutObject. Consulte Conceder políticas de acesso personalizadas a um usuário RAM.
Processo de multipart upload
Para enviar um objeto usando multipart upload, execute as etapas a seguir:
-
Inicialize um evento de multipart upload.
Chame o método oss_init_multipart_upload. Esse método retorna um ID de upload globalmente exclusivo criado pelo OSS.
-
Envie as partes.
Chame oss_upload_part_from_file para enviar as partes.
NotaO número da parte identifica sua posição no objeto. Enviar dados com o mesmo número de parte substitui a parte existente.
O OSS retorna o hash MD5 dos dados da parte recebida no cabeçalho ETag da resposta.
O OSS calcula o hash MD5 dos dados enviados e o compara com o hash calculado pelo SDK. Em caso de incompatibilidade, o sistema retorna o código de erro InvalidDigest.
-
Conclua o multipart upload.
Após enviar todas as partes, chame o método oss_complete_multipart_upload para reuni-las em um arquivo completo.
Exemplo completo de multipart upload
O exemplo a seguir mostra o processo completo de multipart upload:
#include "oss_api.h"
#include "aos_http_io.h"
#include <sys/stat.h>
/* Set yourEndpoint to the Endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
const char *endpoint = "yourEndpoint";
/* Enter the bucket name, for example, examplebucket. */
const char *bucket_name = "examplebucket";
/* Enter the full path of the object. The path cannot contain the bucket name. For example, enter exampledir/exampleobject.txt. */
const char *object_name = "exampledir/exampleobject.txt";
/* Enter the full path of the local file. */
const char *local_filename = "yourLocalFilename";
/* Set yourRegion to the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the Region to cn-hangzhou. */
const char *region = "yourRegion";
void init_options(oss_request_options_t *options)
{
options->config = oss_config_create(options->pool);
/* Initialize the aos_string_t type with a char* string. */
aos_str_set(&options->config->endpoint, endpoint);
/* Get access credentials from environment variables. Before running this code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set. */
aos_str_set(&options->config->access_key_id, getenv("OSS_ACCESS_KEY_ID"));
aos_str_set(&options->config->access_key_secret, getenv("OSS_ACCESS_KEY_SECRET"));
// Configure the following two parameters.
aos_str_set(&options->config->region, region);
options->config->signature_version = 4;
/* Specify whether to use a CNAME to access OSS. A value of 0 means a CNAME is not used. */
options->config->is_cname = 0;
/* Set network parameters, such as the timeout period. */
options->ctl = aos_http_controller_create(options->pool, 0);
}
int64_t get_file_size(const char *file_path)
{
int64_t filesize = -1;
struct stat statbuff;
if(stat(file_path, &statbuff) < 0){
return filesize;
} else {
filesize = statbuff.st_size;
}
return filesize;
}
int main(int argc, char *argv[])
{
/* At the program entry point, call the aos_http_io_initialize method to initialize global resources, such as the network and memory. */
if (aos_http_io_initialize(NULL, 0) != AOSE_OK) {
exit(1);
}
/* A memory pool (pool) for memory management. It is equivalent to apr_pool_t. The implementation code is in the apr library. */
aos_pool_t *pool;
/* Create a new memory pool. The second parameter is NULL, which indicates that the pool does not inherit from other memory pools. */
aos_pool_create(&pool, NULL);
/* Create and initialize options. This parameter includes global configurations such as endpoint, access_key_id, access_key_secret, is_cname, and curl. */
oss_request_options_t *oss_client_options;
/* Allocate memory for options in the memory pool. */
oss_client_options = oss_request_options_create(pool);
/* Initialize the client options, oss_client_options. */
init_options(oss_client_options);
/* Initialize parameters. */
aos_string_t bucket;
aos_string_t object;
oss_upload_file_t *upload_file = NULL;
aos_string_t upload_id;
aos_table_t *headers = NULL;
aos_table_t *complete_headers = NULL;
aos_table_t *resp_headers = NULL;
aos_status_t *resp_status = NULL;
aos_str_set(&bucket, bucket_name);
aos_str_set(&object, object_name);
aos_str_null(&upload_id);
headers = aos_table_make(pool, 1);
complete_headers = aos_table_make(pool, 1);
int part_num = 1;
/* Initialize the multipart upload and obtain an upload ID. */
resp_status = oss_init_multipart_upload(oss_client_options, &bucket, &object, &upload_id, headers, &resp_headers);
/* Check whether the multipart upload is successfully initialized. */
if (aos_status_is_ok(resp_status)) {
printf("Init multipart upload succeeded, upload_id:%.*s\n",
upload_id.len, upload_id.data);
} else {
printf("Init multipart upload failed, upload_id:%.*s\n",
upload_id.len, upload_id.data);
}
/* Upload parts. */
int64_t file_length = 0;
int64_t pos = 0;
aos_list_t complete_part_list;
oss_complete_part_content_t* complete_content = NULL;
char* part_num_str = NULL;
char* etag = NULL;
aos_list_init(&complete_part_list);
file_length = get_file_size(local_filename);
while(pos < file_length) {
upload_file = oss_create_upload_file(pool);
aos_str_set(&upload_file->filename, local_filename);
upload_file->file_pos = pos;
pos += 100 * 1024;
upload_file->file_last = pos < file_length ? pos : file_length;
resp_status = oss_upload_part_from_file(oss_client_options, &bucket, &object, &upload_id, part_num++, upload_file, &resp_headers);
/* Save the part number and ETag. */
complete_content = oss_create_complete_part_content(pool);
part_num_str = apr_psprintf(pool, "%d", part_num-1);
aos_str_set(&complete_content->part_number, part_num_str);
etag = apr_pstrdup(pool,
(char*)apr_table_get(resp_headers, "ETag"));
aos_str_set(&complete_content->etag, etag);
aos_list_add_tail(&complete_content->node, &complete_part_list);
if (aos_status_is_ok(resp_status)) {
printf("Multipart upload part from file succeeded\n");
} else {
printf("Multipart upload part from file failed\n");
}
}
/* Complete the multipart upload. */
resp_status = oss_complete_multipart_upload(oss_client_options, &bucket, &object, &upload_id,
&complete_part_list, complete_headers, &resp_headers);
/* Check whether the multipart upload is complete. */
if (aos_status_is_ok(resp_status)) {
printf("Complete multipart upload from file succeeded, upload_id:%.*s\n",
upload_id.len, upload_id.data);
} else {
printf("Complete multipart upload from file failed\n");
}
/* Release the memory pool. This releases the memory allocated for each resource during the request. */
aos_pool_destroy(pool);
/* Release the previously allocated global resources. */
aos_http_io_deinitialize();
return 0;
}
Ao chamar a operação oss_complete_multipart_upload, forneça o valor ETag de cada parte. Obtenha esses valores de duas maneiras:
Durante o envio de cada parte, a resposta contém o valor ETag correspondente. Salve esse valor para uso posterior. O exemplo anterior utiliza este método.
Chame a operação oss_list_upload_part para recuperar os valores ETag das partes enviadas anteriormente.
Cancelar um evento de multipart upload
O código a seguir mostra como cancelar um evento de multipart upload.
#include "oss_api.h"
#include "aos_http_io.h"
/* Set yourEndpoint to the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
const char *endpoint = "yourEndpoint";
/* Specify the bucket name. For example, examplebucket. */
const char *bucket_name = "examplebucket";
/* Specify the full path of the object. The full path cannot contain the bucket name. For example, exampledir/exampleobject.txt. */
const char *object_name = "exampledir/exampleobject.txt";
/* Set yourRegion to the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the region to cn-hangzhou. */
const char *region = "yourRegion";
void init_options(oss_request_options_t *options)
{
options->config = oss_config_create(options->pool);
/* Initialize the aos_string_t type with a char* string. */
aos_str_set(&options->config->endpoint, endpoint);
/* Obtain access credentials from environment variables. Before you run this code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set. */
aos_str_set(&options->config->access_key_id, getenv("OSS_ACCESS_KEY_ID"));
aos_str_set(&options->config->access_key_secret, getenv("OSS_ACCESS_KEY_SECRET"));
// Configure the following two parameters.
aos_str_set(&options->config->region, region);
options->config->signature_version = 4;
/* Specifies whether to use a CNAME to access OSS. 0 indicates that a CNAME is not used. */
options->config->is_cname = 0;
/* Set network parameters, such as the timeout period. */
options->ctl = aos_http_controller_create(options->pool, 0);
}
int main(int argc, char *argv[])
{
/* Call the aos_http_io_initialize method at the program entry to initialize global resources, such as the network and memory. */
if (aos_http_io_initialize(NULL, 0) != AOSE_OK) {
exit(1);
}
/* The memory pool (pool) for memory management is equivalent to apr_pool_t. Its implementation code is in the apr library. */
aos_pool_t *pool;
/* Create a new memory pool. The second parameter is NULL, which indicates that the new memory pool does not inherit from another memory pool. */
aos_pool_create(&pool, NULL);
/* Create and initialize options. This parameter contains global configuration information, such as endpoint, access_key_id, access_key_secret, is_cname, and curl. */
oss_request_options_t *oss_client_options;
/* Allocate memory to options in the memory pool. */
oss_client_options = oss_request_options_create(pool);
/* Initialize the client options oss_client_options. */
init_options(oss_client_options);
/* Initialize parameters. */
aos_string_t bucket;
aos_string_t object;
aos_string_t upload_id;
aos_table_t *headers = NULL;
aos_table_t *resp_headers = NULL;
aos_status_t *resp_status = NULL;
aos_str_set(&bucket, bucket_name);
aos_str_set(&object, object_name);
aos_str_null(&upload_id);
/* Initialize the multipart upload to obtain an upload ID (upload_id). */
resp_status = oss_init_multipart_upload(oss_client_options, &bucket, &object, &upload_id, headers, &resp_headers);
/* Check whether the multipart upload is successfully initialized. */
if (aos_status_is_ok(resp_status)) {
printf("Init multipart upload succeeded, upload_id:%.*s\n",
upload_id.len, upload_id.data);
} else {
printf("Init multipart upload failed, upload_id:%.*s\n",
upload_id.len, upload_id.data);
}
/* Abort the multipart upload. */
resp_status = oss_abort_multipart_upload(oss_client_options, &bucket, &object, &upload_id, &resp_headers);
/* Check whether the multipart upload is successfully aborted. */
if (aos_status_is_ok(resp_status)) {
printf("Abort multipart upload succeeded, upload_id::%.*s\n",
upload_id.len, upload_id.data);
} else {
printf("Abort multipart upload failed\n");
}
/* Release the memory pool. This releases the memory allocated to resources during the request. */
aos_pool_destroy(pool);
/* Release the previously allocated global resources. */
aos_http_io_deinitialize();
return 0;
}
Após o cancelamento de um evento de multipart upload, o ID de upload torna-se inutilizável para qualquer outra operação. As partes já enviadas também são excluídas.
Referências
Para um exemplo de código completo de multipart upload, consulte o exemplo no GitHub.
-
O multipart upload envolve três operações de API:
Inicializar um multipart upload: InitiateMultipartUpload.
Enviar uma parte: UploadPart.
Concluir um multipart upload: CompleteMultipartUpload.
Para obter mais informações sobre como cancelar um evento de multipart upload, consulte AbortMultipartUpload.