Multipart upload splits a large object into multiple parts and uploads them independently. After all parts are uploaded, call CompleteMultipartUpload to assemble them into a complete object.
Prerequisites
Before you begin, make sure you have:
An OSS bucket
The
oss:PutObjectpermission. For details, see Attach a custom policy to a RAM userAccess credentials configured as environment variables. For details, see Configure access credentials
How it works
A multipart upload consists of three steps:
Initiate — Call
Client.InitiateMultipartUploadto get a unique upload ID from OSS.Upload parts — Call
Client.UploadPartto upload each part using that upload ID.Complete — Call
Client.CompleteMultipartUploadto assemble all uploaded parts into a final object.
Part numbering behavior:
Part numbers define the order of parts in the final object. Uploading a new part with an existing part number overwrites the original.
OSS returns the MD5 hash of each uploaded part in the
ETagresponse header, then validates it against the hash computed by OSS SDK for Go. If they differ, OSS returnsInvalidDigest. For more information, see Can I use ETag values as OSS MD5 hashes to check data consistency.
The sample code uses region cn-hangzhou and the public endpoint. To access OSS from another Alibaba Cloud service in the same region, switch to the internal endpoint. For available regions and endpoints, see Regions and endpoints.Upload a local file
The following example reads a local file into memory, splits it into 3 parts, uploads the parts concurrently using goroutines, and then assembles them.
package main
import (
"bufio"
"bytes"
"context"
"flag"
"io"
"log"
"os"
"sync"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)
var (
region string
bucketName string
objectName string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&bucketName, "bucket", "", "The name of the bucket.")
flag.StringVar(&objectName, "object", "", "The name of the object.")
}
func main() {
flag.Parse()
var uploadId string
if len(bucketName) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, bucket name required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
if len(objectName) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, object name required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := oss.NewClient(cfg)
// Initiate the multipart upload and get an upload ID.
initRequest := &oss.InitiateMultipartUploadRequest{
Bucket: oss.Ptr(bucketName),
Key: oss.Ptr(objectName),
}
initResult, err := client.InitiateMultipartUpload(context.TODO(), initRequest)
if err != nil {
log.Fatalf("failed to initiate multipart upload: %v", err)
}
log.Printf("upload ID: %s\n", *initResult.UploadId)
uploadId = *initResult.UploadId
var wg sync.WaitGroup
var parts []oss.UploadPart
var mu sync.Mutex
count := 3
// Read the local file into memory. Replace "yourLocalFile" with the actual file path.
file, err := os.Open("yourLocalFile")
if err != nil {
log.Fatalf("failed to open file: %v", err)
}
defer file.Close()
bufReader := bufio.NewReader(file)
content, err := io.ReadAll(bufReader)
if err != nil {
log.Fatalf("failed to read file: %v", err)
}
log.Printf("file size: %d bytes\n", len(content))
// Split the file into parts and upload them concurrently.
chunkSize := len(content) / count
if chunkSize == 0 {
chunkSize = 1
}
for i := 0; i < count; i++ {
start := i * chunkSize
end := start + chunkSize
if i == count-1 {
end = len(content)
}
wg.Add(1)
go func(partNumber int, start, end int) {
defer wg.Done()
partRequest := &oss.UploadPartRequest{
Bucket: oss.Ptr(bucketName),
Key: oss.Ptr(objectName),
PartNumber: int32(partNumber),
UploadId: oss.Ptr(uploadId),
Body: bytes.NewReader(content[start:end]),
}
partResult, err := client.UploadPart(context.TODO(), partRequest)
if err != nil {
log.Fatalf("failed to upload part %d: %v", partNumber, err)
}
mu.Lock()
parts = append(parts, oss.UploadPart{
PartNumber: partRequest.PartNumber,
ETag: partResult.ETag,
})
mu.Unlock()
}(i+1, start, end)
}
wg.Wait()
// Assemble all uploaded parts into the final object.
request := &oss.CompleteMultipartUploadRequest{
Bucket: oss.Ptr(bucketName),
Key: oss.Ptr(objectName),
UploadId: oss.Ptr(uploadId),
CompleteMultipartUpload: &oss.CompleteMultipartUpload{
Parts: parts,
},
}
result, err := client.CompleteMultipartUpload(context.TODO(), request)
if err != nil {
log.Fatalf("failed to complete multipart upload: %v", err)
}
log.Printf("multipart upload complete: %#v\n", result)
}More examples
API reference
| Operation | Description |
|---|---|
| InitiateMultipartUpload | Initiates a multipart upload task and returns an upload ID |
| UploadPart | Uploads a single part |
| CompleteMultipartUpload | Assembles uploaded parts into a complete object |
| AbortMultipartUpload | Cancels a multipart upload task |
| NewListPartsPaginator | Lists parts uploaded in a specific task |
| NewListMultipartUploadsPaginator | Lists all ongoing multipart upload tasks in a bucket. Ongoing tasks are those that have been initiated but not yet completed or aborted. |
For the complete sample code, see GitHub.