how to upload files by using go sdk v2 - Object Storage Service

This topic describes how to quickly upload local files to Object Storage Service (OSS) using simple upload. This method is simple and suitable for scenarios that require fast local file uploads.

Notes

The sample code in this topic uses the cn-hangzhou region ID for China (Hangzhou). By default, the public endpoint is used to access resources in a bucket. If you want to access resources in the bucket using other Alibaba Cloud services in the same region as the bucket, use the internal endpoint. For more information about OSS regions and endpoints, see Regions and endpoints.
In this topic, access credentials are read from environment variables. For more information about how to configure access credentials, see Configure access credentials.

Permissions

By default, an Alibaba Cloud account has full permissions. RAM users or RAM roles under an Alibaba Cloud account do not have any permissions by default. The Alibaba Cloud account or account administrator must grant operation permissions through RAM Policy or Bucket Policy.

API	Action	Definition
PutObject	`oss:PutObject`	Uploads an object.
	`oss:PutObjectTagging`	When uploading an object, if you specify object tags through x-oss-tagging, this permission is required.
	`kms:GenerateDataKey`	When uploading an object, if the object metadata contains X-Oss-Server-Side-Encryption: KMS, these two permissions are required.
	`kms:Decrypt`

Method definition

func (c *Client) PutObject(ctx context.Context, request *PutObjectRequest, optFns ...func(*Options)) (*PutObjectResult, error)

func (c *Client) PutObjectFromFile(ctx context.Context, request *PutObjectRequest, filePath string, optFns ...func(*Options)) (*PutObjectResult, error)

API operation

Description

Client.PutObject

Performs simple upload. The maximum size of an object that can be uploaded is 5 GiB.

CRC-64 data validation is supported and enabled by default.

Progress bar charts are supported.

The type of the request body is io.Reader. If the type is io.Seeker, retransmission is supported if the upload fails.

Client.PutObjectFromFile

This operation provides the same capabilities as the Client.PutObject operation.

The request body data is from the file path.

Request parameters

Parameter	Type	Description
ctx	context.Context	The context of the request. You can use this parameter to set the total timeout period of a request.
request	*PutObjectRequest	The parameters of a specific API operation. For example, you can set the access control method (Acl), prevent overwrites (ForbidOverwrite), and specify custom metadata (Metadata). For more information, see PutObjectRequest.
optFns	...func(*Options)	(Optional) The operation-level configuration parameters. For more information, see Options.

Return values

Return value	Type	Description
result	*PutObjectResult	The return value of the API operation. This parameter is valid when err is nil. For more information, see PutObjectResult.
err	error	The status of the request. If the request fails, err is not nil.

Sample code

You can use the following code to upload a local file to a destination bucket.

package main

import (
	"context"
	"flag"
	"log"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)

// Define global variables.
var (
	region     string // The region.
	bucketName string // The bucket name.
	objectName string // The object name.
)

// The init function is used to initialize command-line parameters.
func init() {
	flag.StringVar(&region, "region", "", "The region in which the bucket is located.")
	flag.StringVar(&bucketName, "bucket", "", "The name of the bucket.")
	flag.StringVar(&objectName, "object", "", "The name of the object.")
}

func main() {
	// Parse command-line parameters.
	flag.Parse()

	// Check whether the bucket name is empty.
	if len(bucketName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, bucket name required")
	}

	// Check whether the region is empty.
	if len(region) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, region required")
	}

	// Check whether the object name is empty.
	if len(objectName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, object name required")
	}

	// Load the default configurations and set the credential provider and region.
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	// Create an OSS client.
	client := oss.NewClient(cfg)

	// Specify the path and name of the local file to upload, for example, /Users/localpath/exampleobject.txt.
	localFile := "/Users/localpath/exampleobject.txt"

	// Create a request to upload the object.
	putRequest := &oss.PutObjectRequest{
		Bucket:       oss.Ptr(bucketName),      // The bucket name.
		Key:          oss.Ptr(objectName),      // The object name.
		StorageClass: oss.StorageClassStandard, // Specify the storage class of the object as Standard.
		Acl:          oss.ObjectACLPrivate,     // Specify the access permissions of the object as private.
		Metadata: map[string]string{
			"yourMetadataKey1": "yourMetadataValue1", // Set the metadata of the object.
		},
	}

	// Execute the request to upload the object.
	result, err := client.PutObjectFromFile(context.TODO(), putRequest, localFile)
	if err != nil {
		log.Fatalf("failed to put object from file %v", err)
	}

	// Print the result of the object upload.
	log.Printf("put object from file result:%#v\n", result)
}

Common scenarios

Upload a string

You can use the following code to upload a string to a destination bucket.

package main

import (
	"context"
	"flag"
	"log"
	"strings"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)

// Define global variables.
var (
	region     string // The region.
	bucketName string // The bucket name.
	objectName string // The object name.
)

// The init function is used to initialize command-line parameters.
func init() {
	flag.StringVar(&region, "region", "", "The region in which the bucket is located.")
	flag.StringVar(&bucketName, "bucket", "", "The name of the bucket.")
	flag.StringVar(&objectName, "object", "", "The name of the object.")
}

func main() {
	// Parse command-line parameters.
	flag.Parse()

	// Check whether the bucket name is empty.
	if len(bucketName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, bucket name required")
	}

	// Check whether the region is empty.
	if len(region) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, region required")
	}

	// Check whether the object name is empty.
	if len(objectName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, object name required")
	}

	// Define the string content to upload.
	body := strings.NewReader("hi oss")

	// Load the default configurations and set the credential provider and region.
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	// Create an OSS client.
	client := oss.NewClient(cfg)

	// Create a request to upload the object.
	request := &oss.PutObjectRequest{
		Bucket: oss.Ptr(bucketName), // The bucket name.
		Key:    oss.Ptr(objectName), // The object name.
		Body:   body,                // The string content to upload.
	}

	// Send the request to upload the object.
	result, err := client.PutObject(context.TODO(), request)
	if err != nil {
		log.Fatalf("failed to put object %v", err)
	}

	// Print the result of the object upload.
	log.Printf("put object result:%#v\n", result)
}

Upload a byte array

You can use the following code to upload a byte array to a destination bucket.

package main

import (
	"bytes"
	"context"
	"flag"
	"log"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)

// Define global variables.
var (
	region     string // The region.
	bucketName string // The bucket name.
	objectName string // The object name.
)

// The init function is used to initialize command-line parameters.
func init() {
	flag.StringVar(&region, "region", "", "The region in which the bucket is located.")
	flag.StringVar(&bucketName, "bucket", "", "The name of the bucket.")
	flag.StringVar(&objectName, "object", "", "The name of the object.")
}

func main() {
	// Parse command-line parameters.
	flag.Parse()

	// Check whether the bucket name is empty.
	if len(bucketName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, bucket name required")
	}

	// Check whether the region is empty.
	if len(region) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, region required")
	}

	// Check whether the object name is empty.
	if len(objectName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, object name required")
	}

	// Define the byte array content to upload.
	body := bytes.NewReader([]byte("yourObjectValueByteArray"))

	// Load the default configurations and set the credential provider and region.
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	// Create an OSS client.
	client := oss.NewClient(cfg)

	// Create a request to upload the object.
	request := &oss.PutObjectRequest{
		Bucket: oss.Ptr(bucketName), // The bucket name.
		Key:    oss.Ptr(objectName), // The object name.
		Body:   body,                // The byte array content to upload.
	}

	// Send the request to upload the object.
	result, err := client.PutObject(context.TODO(), request)
	if err != nil {
		log.Fatalf("failed to put object %v", err)
	}

	// Print the result of the object upload.
	log.Printf("put object result:%#v\n", result)
}

Upload a network stream

You can use the following code to upload a network stream to a destination bucket.

package main

import (
	"context"
	"flag"
	"io"
	"log"
	"net/http"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)

// Define global variables.
var (
	region     string // The region.
	bucketName string // The bucket name.
	objectName string // The object name.
)

// The init function is used to initialize command-line parameters.
func init() {
	flag.StringVar(&region, "region", "", "The region in which the bucket is located.")
	flag.StringVar(&bucketName, "bucket", "", "The name of the bucket.")
	flag.StringVar(&objectName, "object", "", "The name of the object.")
}

func main() {
	// Parse command-line parameters.
	flag.Parse()

	// Check whether the bucket name is empty.
	if len(bucketName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, bucket name required")
	}

	// Check whether the region is empty.
	if len(region) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, region required")
	}

	// Check whether the object name is empty.
	if len(objectName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, object name required")
	}

	// Load the default configurations and set the credential provider and region.
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	// Create an OSS client.
	client := oss.NewClient(cfg)

	// Specify the network stream to upload.
	resp, err := http.Get("https://www.aliyun.com/")
	if err != nil {
		log.Fatalf("Failed to fetch URL: %v", err)
	}
	defer resp.Body.Close()

	// Create a request to upload the object.
	request := &oss.PutObjectRequest{
		Bucket: oss.Ptr(bucketName),  // The bucket name.
		Key:    oss.Ptr(objectName),  // The object name.
		Body:   io.Reader(resp.Body), // The network stream content to upload.
	}

	// Send the request to upload the object.
	result, err := client.PutObject(context.TODO(), request)
	if err != nil {
		log.Fatalf("failed to put object %v", err)
	}

	// Print the result of the object upload.
	log.Printf("put object result:%#v\n", result)
}

Display the progress during upload

When you upload a file, you can use a progress bar chart to view the upload progress in real time. This helps you monitor the upload and confirm that the task is not stuck during a long wait time.

You can use the following code to view the upload progress of a file using a progress bar chart.

package main

import (
	"context"
	"flag"
	"fmt"
	"log"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)

// Define global variables.
var (
	region     string // The region.
	bucketName string // The bucket name.
	objectName string // The object name.
)

// The init function is used to initialize command-line parameters.
func init() {
	flag.StringVar(&region, "region", "", "The region in which the bucket is located.")
	flag.StringVar(&bucketName, "bucket", "", "The name of the bucket.")
	flag.StringVar(&objectName, "object", "", "The name of the object.")
}

func main() {
	// Parse command-line parameters.
	flag.Parse()

	// Check whether the bucket name is empty.
	if len(bucketName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, bucket name required")
	}

	// Check whether the region is empty.
	if len(region) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, region required")
	}

	// Check whether the object name is empty.
	if len(objectName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, object name required")
	}

	// Load the default configurations and set the credential provider and region.
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	// Create an OSS client.
	client := oss.NewClient(cfg)

	// Specify the path and name of the local file to upload, for example, /Users/localpath/exampleobject.txt.
	localFile := "/Users/localpath/exampleobject.txt"

	// Create a request to upload the object.
	putRequest := &oss.PutObjectRequest{
		Bucket: oss.Ptr(bucketName), // The bucket name.
		Key:    oss.Ptr(objectName), // The object name.
		ProgressFn: func(increment, transferred, total int64) {
			fmt.Printf("increment:%v, transferred:%v, total:%v\n", increment, transferred, total)
		}, // The progress callback function, which is used to display the upload progress.
	}

	// Send the request to upload the object.
	result, err := client.PutObjectFromFile(context.TODO(), putRequest, localFile)
	if err != nil {
		log.Fatalf("failed to put object from file %v", err)
	}

	// Print the result of the object upload.
	log.Printf("put object from file result:%#v\n", result)
}

Set a callback function when you upload a file

OSS can provide callbacks to an application server when a simple upload (PutObject and PutObjectFromFile) is complete. You only need to include the corresponding callback parameters in the request sent to OSS to implement the callback.

You can use the following code to set a callback function when you upload a file.

package main

import (
	"context"
	"encoding/base64"
	"encoding/json"
	"flag"
	"log"
	"strings"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)

// Define global variables.
var (
	region     string // The region.
	bucketName string // The bucket name.
	objectName string // The object name.
)

// The init function is used to initialize command-line parameters.
func init() {
	flag.StringVar(&region, "region", "", "The region in which the bucket is located.")
	flag.StringVar(&bucketName, "bucket", "", "The name of the bucket.")
	flag.StringVar(&objectName, "object", "", "The name of the object.")
}

func main() {
	// Parse command-line parameters.
	flag.Parse()

	// Check whether the bucket name is empty.
	if len(bucketName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, bucket name required")
	}

	// Check whether the region is empty.
	if len(region) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, region required")
	}

	// Check whether the object name is empty.
	if len(objectName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, object name required")
	}

	// Load the default configurations and set the credential provider and region.
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	// Create an OSS client.
	client := oss.NewClient(cfg)

	// Define the callback parameters.
	callbackMap := map[string]string{
		"callbackUrl":      "http://example.com:23450",                                                        // Set the URL of the callback server, for example, https://example.com:23450.
		"callbackBody":     "bucket=${bucket}&object=${object}&size=${size}&my_var_1=${x:my_var1}&my_var_2=${x:my_var2}", // Set the callback request body.
		"callbackBodyType": "application/x-www-form-urlencoded",                                                          // Set the type of the callback request body.
	}

	// Convert the callback parameters to JSON and perform Base64 encoding to pass them as callback parameters.
	callbackStr, err := json.Marshal(callbackMap)
	if err != nil {
		log.Fatalf("failed to marshal callback map: %v", err)
	}
	callbackBase64 := base64.StdEncoding.EncodeToString(callbackStr)

	callbackVarMap := map[string]string{}
	callbackVarMap["x:my_var1"] = "thi is var 1"
	callbackVarMap["x:my_var2"] = "thi is var 2"
	callbackVarStr, err := json.Marshal(callbackVarMap)
	if err != nil {
		log.Fatalf("failed to marshal callback var: %v", err)
	}
	callbackVarBase64 := base64.StdEncoding.EncodeToString(callbackVarStr)
	// Define the string content to upload.
	body := strings.NewReader("Hello, OSS!") // Replace this with the string content to upload.

	// Create a request to upload the object.
	request := &oss.PutObjectRequest{
		Bucket:      oss.Ptr(bucketName),     // The bucket name.
		Key:         oss.Ptr(objectName),     // The object name.
		Body:        body,                    // The object content.
		Callback:    oss.Ptr(callbackBase64), // The callback parameters.
		CallbackVar: oss.Ptr(callbackVarBase64),
	}

	// Send the request to upload the object.
	result, err := client.PutObject(context.TODO(), request)
	if err != nil {
		log.Fatalf("failed to put object %v", err)
	}

	// Print the result of the object upload.
	log.Printf("put object result:%#v\n", result)
}

References

For the complete sample code for simple upload, see the put_object.go and put_object_from_file.go examples on GitHub.
For more information about the API operations for simple upload, see PutObject and PutObjectFromFile.