All Products
Search
Document Center

Object Storage Service:Resumable upload

Last Updated:Sep 06, 2023

When you upload an object to Object Storage Service (OSS) by using resumable upload, you can specify a directory for the checkpoint file that stores resumable upload records. If an object fails to be uploaded because of a network exception or program error, the upload task is resumed from the position recorded in the checkpoint file.

Usage notes

  • In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS by using other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about the regions and endpoints supported by OSS, see Regions and endpoints.

  • In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.

  • In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Initialization.

  • To use resumable upload, you must have oss:PutObject permission. For more information, see Common examples of RAM policies.

  • During resumable upload, the upload progress is recorded in a checkpoint file. If a part fails to be uploaded, the next upload starts from the position that is recorded in the checkpoint file. After the object is uploaded, the checkpoint file is deleted.

  • When you use OSS SDK for Go to perform resumable upload, the upload progress is recorded in the checkpoint file. Make sure that your program is granted write permissions on the checkpoint file.

  • You cannot modify the checksum that is contained in the checkpoint file. If the checkpoint file is damaged, you must upload all parts of the object again.

  • If the local file is modified during the upload, you must upload all parts of the object again.

Implementation method

You can use the Bucket.UploadFile method to perform resumable upload. The following table describes the parameters that you can configure.

Parameter

Description

objectKey

The name of the OSS object. This parameter is equivalent to objectName.

filePath

The path of the local file that you want to upload to OSS.

partSize

The size of a part. Valid values: 100 KB to 5 GB. Default value: 100 KB.

options

The options for the upload. Valid values:

  • Routines: specifies the number of concurrent upload threads. The default value is 1. The value 1 indicates that concurrent upload is not used.

  • Checkpoint: specifies whether resumable upload is enabled and the checkpoint file. By default, resumable upload is disabled.

    For example, you can specify oss.Checkpoint(true, "") to enable the resumable upload feature and the file.cp file in the same directory as the local file as the checkpoint file. file indicates the name of the local file. You can also use oss.Checkpoint(true, "your-cp-file.cp") to specify the checkpoint file.

Note

For more information about metadata, see Manage object metadata.

Sample code

The following sample code provides an example on how to perform resumable upload:

package main

import (
    "fmt"
    "os"
    "github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
    /// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
    provider, err := oss.NewEnvironmentVariableCredentialsProvider()
    if err != nil {
        fmt.Println("Error:", err)
        os.Exit(-1)
    }

    // Create an OSSClient instance. 
    // Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
    client, err := oss.New("yourEndpoint", "", "", oss.SetCredentialsProvider(&provider))
    if err != nil {
        fmt.Println("Error:", err)
        os.Exit(-1)
    }

    // Specify the name of the bucket. Example: examplebucket. 
    bucket, err := client.Bucket("examplebucket")
    if err != nil {
        fmt.Println("Error:", err)
        os.Exit(-1)
    }

    // Set the part size to 100 KB (100 × 1024), set the number of concurrent upload threads to 3, and then enable resumable upload. 
    // Specify the full path of the object. Do not include the bucket name in the full path. Example: exampledir/exampleobject.txt. 
    // Specify the full path of the local file. Example: D:\\localpath\\examplefile.txt. By default, if you do not specify the full path of the local file, the local file is uploaded from the path of the project to which the sample program belongs. 
    err = bucket.UploadFile("exampledir/exampleobject.txt", "D:\\localpath\\examplefile.txt", 100*1024, oss.Routines(3), oss.Checkpoint(true, ""))
    if err != nil {
        fmt.Println("Error:", err)
        os.Exit(-1)
    }
}            

References

For the complete sample code that is used to perform resumable upload, visit GitHub.