This topic describes how to copy large objects using the new Copier module of Object Storage Service (OSS) SDK for Go V2.
Usage notes
The sample code in this topic uses the region ID
cn-hangzhou, which specifies the China (Hangzhou) region. By default, a public endpoint is used to access resources in a bucket. If you want to access resources in the bucket from other Alibaba Cloud services in the same region as the bucket, use an internal endpoint. For more information about OSS regions and endpoints, see Regions and endpoints.In this topic, access credentials are obtained from environment variables. For more information about how to configure the access credentials, see Configure access credentials.
To copy an object, you must have the read permissions on the source object and read and write permissions on the destination bucket.
The source bucket and destination bucket must be located in the same region. For example, objects in a bucket located in the China (Hangzhou) region cannot be copied to a bucket located in the China (Qingdao) region.
Make sure that no retention policies are configured for the source bucket and the destination bucket. Otherwise, the following error message is returned: The object you specified is immutable.
Methods
Introduction to Copy Manager
If you want to copy an object from a bucket to another bucket or modify the attributes of an object, you can call the CopyObject operation or the UploadPartCopy operation. The two operations are suitable for different scenarios:
The CopyObject operation is suitable only for copying an object smaller than 5 GiB.
The UploadPartCopy operation is suitable for copying an object that is equal to or greater than 5 GiB in size. Take note that the operation does not support the x-oss-metadata-directive and x-oss-tagging-directive directives. To configure metadata and tagging during a multipart copy operation, you need additional implementation.
The Copier module hides the operation differences and implementation details, providing a universal solution for copying objects. The Copier automatically selects an appropriate operation to copy objects according to the specified request parameters. The following lines provide common methods of the Copier module:
type Copier struct {
...
}
// Create a Copier.
func (c *Client) NewCopier(optFns ...func(*CopierOptions)) *Copier
// // Copy objects.
func (c *Copier) Copy(ctx context.Context, request *CopyObjectRequest, optFns ...func(*CopierOptions)) (*CopyResult, error)Request parameters
Parameter | Type | Description |
ctx | context.Context | The context of the request, which can be used to specify the total duration of the request. |
request | *CopyObjectRequest | The parameters of a specific API operation. For more information, see CopyObjectRequest. |
optFns | ...func(*CopierOptions) | Optional parameters. For more information, see CopierOptions. |
Common parameters of CopyObjectRequest
Parameter | Type | Description |
Bucket | *string | The name of the destination bucket. |
Key | *string | The name of the destination object. |
SourceBucket | *string | The name of the source bucket. |
SourceKey | *string | The name of the source object. |
ForbidOverwrite | *string | Specifies whether the CopyObject operation disables the overwriting of an object with the same name. |
Tagging | *string | The tags of the destination object. You can configure multiple tags for the destination object. Example: TagA=A&TagB=B. |
TaggingDirective | *string | The method that is used to configure tags for the destination object. Values:
|
Common parameters of CopierOptions
Parameter | Type | Description |
PartSize | int64 | The part size. The default part size is 64 MiB. |
ParallelNum | int | The number of upload tasks in parallel. The default value is 3. The setting is specific to this call only and does not apply globally. |
MultipartCopyThreshold | int64 | The minimum object size for calling the multipart copy operation. The default size is 200 MiB. |
LeavePartsOnError | bool | Specifies whether to retain the copied parts in case of a copy failure. By default, copied parts are not retained. |
DisableShallowCopy | bool | Specifies whether to disable shallow copy. By default, shallow copy is enabled. |
Response parameters
Parameter | Type | Description |
result | *CopyResult | The response to the operation. This parameter is available when the value of err is nil. For more information, see CopyResult. |
err | error | The status of the request. If the request fails, the value of err is not nil. |
Sample code
The following sample code provides an example on how to copy an object from the source bucket to a destination bucket and modify the object attributes:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)
// Define the global variables.
var (
region string // The region.
srcBucketName string // The name of the source bucket.
srcObjectName string // The name of the source object.
destBucketName string // The name of the destination bucket.
destObjectName string // The name of the destination object.
)
// Use the init function to initialize parameters.
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&srcBucketName, "src-bucket", "", "The name of the source bucket.")
flag.StringVar(&srcObjectName, "src-object", "", "The name of the source object.")
flag.StringVar(&destBucketName, "dest-bucket", "", "The name of the destination bucket.")
flag.StringVar(&destObjectName, "dest-object", "", "The name of the destination object.")
}
func main() {
// Parse parameters.
flag.Parse()
// Check whether the source bucket name is empty.
if len(srcBucketName) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, bucket name required")
}
// Check whether the region is empty.
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
// If the destination bucket name is not specified, the source bucket name is used.
if len(destBucketName) == 0 {
destBucketName = srcBucketName
}
// Check whether the source object name is empty.
if len(srcObjectName) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, src object name required")
}
// Check whether the destination object name is empty.
if len(destObjectName) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, destination object name required")
}
// Create an OSS client configuration.
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
// Create an OSS client.
client := oss.NewClient(cfg)
// Create a Copier.
c := client.NewCopier()
// Create a request to copy the object.
request := &oss.CopyObjectRequest{
Bucket: oss.Ptr(destBucketName), // The name of the destination bucket.
Key: oss.Ptr(destObjectName), // The name of the destination object.
SourceKey: oss.Ptr(srcObjectName), // The name of the source object.
SourceBucket: oss.Ptr(srcBucketName), // The name of the source bucket.
StorageClass: oss.StorageClassStandard, // Set the storage class of the destination object to Standard.
MetadataDirective: oss.Ptr("Replace"), // Ignore the metadata of the source object.
TaggingDirective: oss.Ptr("Replace"), // Ignore the tags of the source object.
}
// Copy the object.
result, err := c.Copy(context.TODO(), request)
if err != nil {
log.Fatalf("failed to copy object %v", err) // In case of an error, log the error and exit the operation.
}
// Display the result of the operation.
log.Printf("copy object result:%#v\n", result)
}
References
For the API operation of the copy manager, see Copy.