O Object Storage Service (OSS) oferece um recurso de multipart upload para objetos grandes. Esse processo divide o objeto em partes menores, envia cada uma independentemente e chama a API CompleteMultipartUpload para combiná-las em um único objeto, permitindo uploads retomáveis.
Observações
Este tópico utiliza o endpoint público da região China (Hangzhou). Para acessar o OSS a partir de outros serviços da Alibaba Cloud na mesma região, use um endpoint interno. Para mais informações sobre regiões e endpoints do OSS, consulte Regiões e endpoints.
Este tópico demonstra como criar uma instância OSSClient com um endpoint do OSS. Para configurações alternativas, como uso de domínio personalizado ou autenticação com credenciais do Security Token Service (STS), consulte Inicialização (C# SDK V1).
A permissão
oss:PutObjecté necessária para executar o multipart upload. Para mais informações, consulte Exemplos comuns de políticas do RAM.
Procedimento
Para enviar um objeto usando multipart upload, execute as etapas a seguir:
-
Inicie uma tarefa de multipart upload.
Chame o método InitiateMultipartUploadRequest para obter um ID de upload exclusivo no OSS.
-
Envie as partes.
Chame o método UploadPartRequest para enviar as partes.
NotaO número da parte identifica sua posição dentro do objeto. Enviar dados com o mesmo número de parte substitui a parte existente.
O OSS retorna o hash MD5 dos dados da parte recebida no cabeçalho ETag da resposta.
O OSS calcula o hash MD5 dos dados enviados e o compara com o hash calculado pelo SDK. Em caso de incompatibilidade, o sistema retorna o código de erro InvalidDigest.
-
Conclua a tarefa de multipart upload.
Após enviar todas as partes, chame o método CompleteMultipartUploadRequest para combiná-las em um objeto completo.
Código de exemplo de multipart upload
O exemplo a seguir mostra o processo completo de multipart upload:
using Aliyun.OSS;
using Aliyun.OSS.Common;
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
var endpoint = "yourEndpoint";
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
var accessKeyId = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_ID");
var accessKeySecret = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_SECRET");
// Specify the name of the bucket.
var bucketName = "examplebucket";
// Specify the full path of the object. Do not include the bucket name in the full path.
var objectName = "exampleobject.txt";
// Specify the full path of the local file that you want to upload. By default, if you do not specify the full path of a local file, the local file is uploaded from the path of the project to which the sample program belongs.
var localFilename = "D:\\localpath\\examplefile.txt";
// Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou.
const string region = "cn-hangzhou";
// Create a ClientConfiguration instance and modify the default parameters based on your requirements.
var conf = new ClientConfiguration();
// Use the signature algorithm V4.
conf.SignatureVersion = SignatureVersion.V4;
// Create an OSSClient instance.
var client = new OssClient(endpoint, accessKeyId, accessKeySecret, conf);
client.SetRegion(region);
// Initiate the multipart upload task and obtain the upload ID in the response.
var uploadId = "";
try
{
// Specify the name of the uploaded object and the bucket for the object. You can configure object metadata in InitiateMultipartUploadRequest. However, you do not need to specify ContentLength.
var request = new InitiateMultipartUploadRequest(bucketName, objectName);
var result = client.InitiateMultipartUpload(request);
uploadId = result.UploadId;
// Display the upload ID.
Console.WriteLine("Init multi part upload succeeded");
// Cancel the multipart upload task or list uploaded parts based on the upload ID.
// If you want to cancel a multipart upload task based on the upload ID, obtain the upload ID after you call the InitiateMultipartUpload operation to initiate the multipart upload task.
// If you want to list the uploaded parts in a multipart upload task based on the upload ID, obtain the upload ID after you call the InitiateMultipartUpload operation to initiate the multipart upload task but before you call the CompleteMultipartUpload operation to complete the multipart upload task.
Console.WriteLine("Upload Id:{0}", result.UploadId);
}
catch (Exception ex)
{
Console.WriteLine("Init multi part upload failed, {0}", ex.Message);
Environment.Exit(1);
}
// Calculate the total number of parts.
// Each part can be 100 KB to 5 GB in size. The size of the last part can be smaller than 100 KB.
var partSize = 100 * 1024;
var fi = new FileInfo(localFilename);
var fileSize = fi.Length;
var partCount = fileSize / partSize;
if (fileSize % partSize != 0)
{
partCount++;
}
// Initialize parts and start the multipart upload task. partETags is a list of PartETags. After OSS receives the partETags, OSS verifies all parts one by one. After all parts are verified, OSS combines these parts into a complete object.
var partETags = new List<PartETag>();
try
{
using (var fs = File.Open(localFilename, FileMode.Open))
{
for (var i = 0; i < partCount; i++)
{
var skipBytes = (long)partSize * i;
// Find the start position of the current upload task.
fs.Seek(skipBytes, 0);
// Calculate the part size in this upload. The size of the last part is the size of the remainder after the object is split by the calculated part size.
var size = (partSize < fileSize - skipBytes) ? partSize : (fileSize - skipBytes);
var request = new UploadPartRequest(bucketName, objectName, uploadId)
{
InputStream = fs,
PartSize = size,
PartNumber = i + 1
};
// Call UploadPart to upload parts. The returned results contain the ETag values of parts.
var result = client.UploadPart(request);
partETags.Add(result.PartETag);
Console.WriteLine("finish {0}/{1}", partETags.Count, partCount);
}
Console.WriteLine("Put multi part upload succeeded");
}
}
catch (Exception ex)
{
Console.WriteLine("Put multi part upload failed, {0}", ex.Message);
Environment.Exit(1);
}
// Combine the parts after the parts are uploaded.
try
{
var completeMultipartUploadRequest = new CompleteMultipartUploadRequest(bucketName, objectName, uploadId);
foreach (var partETag in partETags)
{
completeMultipartUploadRequest.PartETags.Add(partETag);
}
var result = client.CompleteMultipartUpload(completeMultipartUploadRequest);
Console.WriteLine("complete multi part succeeded");
}
catch (Exception ex)
{
Console.WriteLine("complete multi part failed, {0}", ex.Message);
Environment.Exit(1);
}
Cancelar uma tarefa de multipart upload
Chame o método client.AbortMultipartUpload para cancelar uma tarefa de multipart upload. Após o cancelamento, não é possível usar o ID de upload para enviar novas partes, e as partes já enviadas são excluídas.
Veja abaixo um exemplo de código para listar as partes enviadas:
using Aliyun.OSS;
using Aliyun.OSS.Common;
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
var endpoint = "yourEndpoint";
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
var accessKeyId = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_ID");
var accessKeySecret = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_SECRET");
// Specify the name of the bucket.
var bucketName = "examplebucket";
// Specify the full path of the object. Do not include the bucket name in the full path.
var objectName = "exampleobject.txt";
// Specify the upload ID. You can obtain the upload ID from the response to the InitiateMultipartUpload operation.
var uploadId = "yourUploadId";
// Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou.
const string region = "cn-hangzhou";
// Create a ClientConfiguration instance and modify the default parameters based on your requirements.
var conf = new ClientConfiguration();
// Use the signature algorithm V4.
conf.SignatureVersion = SignatureVersion.V4;
// Create an OSSClient instance.
var client = new OssClient(endpoint, accessKeyId, accessKeySecret, conf);
client.SetRegion(region);
// Initiate the multipart upload task.
try
{
var request = new InitiateMultipartUploadRequest(bucketName, objectName);
var result = client.InitiateMultipartUpload(request);
uploadId = result.UploadId;
// Display the upload ID.
Console.WriteLine("Init multi part upload succeeded");
Console.WriteLine("Upload Id:{0}", result.UploadId);
}
catch (Exception ex)
{
Console.WriteLine("Init multi part upload failed, {0}", ex.Message);
}
// Cancel the multipart upload task.
try
{
var request = new AbortMultipartUploadRequest(bucketName, objectName, uploadId);
client.AbortMultipartUpload(request);
Console.WriteLine("Abort multi part succeeded, {0}", uploadId);
}
catch (Exception ex)
{
Console.WriteLine("Abort multi part failed, {0}", ex.Message);
}
Listar partes enviadas
O código a seguir exemplifica como listar as partes enviadas:
using Aliyun.OSS;
using Aliyun.OSS.Common;
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
var endpoint = "yourEndpoint";
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
var accessKeyId = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_ID");
var accessKeySecret = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_SECRET");
// Specify the name of the bucket.
var bucketName = "examplebucket";
// Specify the full path of the object. Do not include the bucket name in the full path.
var objectName = "exampleobject.txt";
// Specify the upload ID. You can obtain the upload ID from the response to the InitiateMultipartUpload operation. You must obtain the upload ID before you call the CompleteMultipartUpload operation to complete the multipart upload task.
var uploadId = "yourUploadId";
// Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou.
const string region = "cn-hangzhou";
// Create a ClientConfiguration instance and modify the default parameters based on your requirements.
var conf = new ClientConfiguration();
// Use the signature algorithm V4.
conf.SignatureVersion = SignatureVersion.V4;
// Create an OSSClient instance.
var client = new OssClient(endpoint, accessKeyId, accessKeySecret, conf);
client.SetRegion(region);
try
{
PartListing listPartsResult = null;
var nextMarker = 0;
do
{
var listPartsRequest = new ListPartsRequest(bucketName, objectName, uploadId)
{
PartNumberMarker = nextMarker,
};
// List the uploaded parts.
listPartsResult = client.ListParts(listPartsRequest);
Console.WriteLine("List parts succeeded");
// Traverse all the uploaded parts.
var parts = listPartsResult.Parts;
foreach (var part in parts)
{
Console.WriteLine("partNumber: {0}, ETag: {1}, Size: {2}", part.PartNumber, part.ETag, part.Size);
}
nextMarker = listPartsResult.NextPartNumberMarker;
} while (listPartsResult.IsTruncated);
}
catch (Exception ex)
{
Console.WriteLine("List parts failed, {0}", ex.Message);
}
Listar tarefas de multipart upload
Chame o método ossClient.listMultipartUploads para listar todas as tarefas de multipart upload em andamento (iniciadas, mas não concluídas ou canceladas). A tabela a seguir descreve os parâmetros configuráveis nessa listagem.
|
Parâmetro |
Descrição |
Método |
|
prefix |
Prefixo obrigatório nos nomes dos objetos retornados. Se usado na consulta, os nomes dos objetos incluirão esse prefixo. |
ListMultipartUploadsRequest.setPrefix(String prefix) |
|
delimiter |
Caractere usado para agrupar nomes de objetos. Objetos cujos nomes contêm uma substring entre o prefixo especificado e a primeira ocorrência do delimitador são retornados como um único elemento. |
ListMultipartUploadsRequest.setDelimiter(String delimiter) |
|
maxUploads |
Número máximo de tarefas de multipart upload a retornar nesta solicitação. Valor padrão: 1000. Valor máximo: 1000. |
ListMultipartUploadsRequest.setMaxUploads(Integer maxUploads) |
|
keyMarker |
Posição inicial da listagem. Lista todas as tarefas de multipart upload em andamento iniciadas para objetos cujos nomes são alfabeticamente posteriores ao valor de keyMarker. Use este parâmetro com uploadIdMarker para definir o ponto de início da listagem. |
ListMultipartUploadsRequest.setKeyMarker(String keyMarker) |
|
uploadIdMarker |
Posição inicial da listagem. Este parâmetro deve ser usado com keyMarker. Se keyMarker não estiver configurado, uploadIdMarker será ignorado. Caso keyMarker esteja definido, os resultados da consulta incluem:
|
ListMultipartUploadsRequest.setUploadIdMarker(String uploadIdMarker) |
Liste eventos de multipart upload:
using Aliyun.OSS;
using Aliyun.OSS.Common;
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
var endpoint = "yourEndpoint";
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
var accessKeyId = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_ID");
var accessKeySecret = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_SECRET");
// Specify the name of the bucket.
var bucketName = "examplebucket";
// Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou.
const string region = "cn-hangzhou";
// Create a ClientConfiguration instance and modify the default parameters based on your requirements.
var conf = new ClientConfiguration();
// Use the signature algorithm V4.
conf.SignatureVersion = SignatureVersion.V4;
// Create an OSSClient instance.
var client = new OssClient(endpoint, accessKeyId, accessKeySecret, conf);
client.SetRegion(region);
try
{
MultipartUploadListing multipartUploadListing = null;
var nextMarker = string.Empty;
do
{
// List multipart upload tasks.
var request = new ListMultipartUploadsRequest(bucketName)
{
KeyMarker = nextMarker,
};
multipartUploadListing = client.ListMultipartUploads(request);
Console.WriteLine("List multi part succeeded");
// List information about each multipart upload task.
foreach (var mu in multipartUploadListing.MultipartUploads)
{
Console.WriteLine("Key: {0}, UploadId: {1}", mu.Key, mu.UploadId);
}
// If the value of the isTruncated field in the returned result is false, values of nextKeyMarker and nextUploadIdMarker are returned and used as the start point for the next read.
nextMarker = multipartUploadListing.NextKeyMarker;
} while (multipartUploadListing.IsTruncated);
}
catch (Exception ex)
{
Console.WriteLine("List multi part uploads failed, {0}", ex.Message);
}
Listar tarefas de multipart upload especificando um prefixo e o número máximo de registros
O código a seguir exemplifica como listar tarefas de multipart upload definindo um prefixo e o limite de registros a retornar:
using Aliyun.OSS;
using Aliyun.OSS.Common;
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com.
var endpoint = "yourEndpoint";
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
var accessKeyId = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_ID");
var accessKeySecret = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_SECRET");
// Specify the name of the bucket.
var bucketName = "examplebucket";
// Specify the prefix.
var prefix = "yourObjectPrefix";
// Specify the maximum number of records to return as 100.
var maxUploads = 100;
// Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou.
const string region = "cn-hangzhou";
// Create a ClientConfiguration instance and modify the default parameters based on your requirements.
var conf = new ClientConfiguration();
// Use the signature algorithm V4.
conf.SignatureVersion = SignatureVersion.V4;
// Create an OSSClient instance.
var client = new OssClient(endpoint, accessKeyId, accessKeySecret, conf);
client.SetRegion(region);
try
{
MultipartUploadListing multipartUploadListing = null;
var nextMarker = string.Empty;
do
{
// List multipart upload tasks. By default, 1,000 multipart upload tasks are listed.
var request = new ListMultipartUploadsRequest(bucketName)
{
KeyMarker = nextMarker,
// Specify the prefix that is contained in the names of the objects that you want to list.
Prefix = prefix,
// Specify the maximum number of records to return.
MaxUploads = maxUploads,
};
multipartUploadListing = client.ListMultipartUploads(request);
Console.WriteLine("List multi part succeeded");
// List information about each multipart upload task.
foreach (var mu in multipartUploadListing.MultipartUploads)
{
Console.WriteLine("Key: {0}, UploadId: {1}", mu.Key, mu.UploadId);
}
nextMarker = multipartUploadListing.NextKeyMarker;
} while (multipartUploadListing.IsTruncated);
}
catch (Exception ex)
{
Console.WriteLine("List multi part uploads failed, {0}", ex.Message);
}
Referências
Para obter o código de exemplo completo de multipart upload, acesse o GitHub.
-
O multipart upload envolve três operações de API:
Inicializar um multipart upload: InitiateMultipartUpload.
Enviar uma parte: UploadPart.
Concluir um multipart upload: CompleteMultipartUpload.
Para mais informações sobre a operação de API usada para abortar um evento de multipart upload, consulte AbortMultipartUpload.
Para mais detalhes sobre a operação de API usada para listar partes enviadas, consulte ListUploadedParts.
Para saber mais sobre a operação de API usada para listar todos os eventos de multipart upload em andamento (iniciados, mas ainda não concluídos ou abortados), consulte ListMultipartUploads.