How to perform a streaming download in OSS - Object Storage Service

This topic describes how to use streaming download to process content in parts. This is useful when a file is large or a one-time download takes too long.

Usage notes

In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions and endpoints.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Configure OSSClient instances.
To use streaming download, you must have the oss:GetObject permission. For more information, see Attach a custom policy to a RAM user.

Download an object to a stream

The following code shows how to download a specified OSS object to a stream:

package main

import (
	"io"
	"log"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		log.Fatalf("Failed to create credentials provider: %v", err)
	}

	// Create an OSSClient instance.
	// Set yourEndpoint to the Endpoint of the bucket. For example, for a bucket in the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, set the Endpoint as needed.
	// Set yourRegion to the region where the bucket is located. For example, for a bucket in the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, set the region as needed.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("yourRegion"))
	// Set the signature version.
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("yourEndpoint", "", "", clientOptions...)
	if err != nil {
		log.Fatalf("Failed to create OSS client: %v", err)
	}

	// Set yourBucketName to the name of the bucket.
	bucket, err := client.Bucket("yourBucketName")
	if err != nil {
		log.Fatalf("Failed to get bucket: %v", err)
	}

	// Download the object to a stream.
	body, err := bucket.GetObject("yourObjectName")
	if err != nil {
		log.Fatalf("Failed to get object: %v", err)
	}
	// After the data is read, close the stream. Otherwise, a connection leak may occur. This can exhaust the connection pool and cause the program to stop working.
	defer body.Close()

	data, err := io.ReadAll(body)
	if err != nil {
		log.Fatalf("Failed to read all data from object: %v", err)
	}
	log.Println("Data:", string(data))
}

Download an object to a buffer

The following code shows how to download a specified OSS object to a buffer:

package main

import (
	"bytes"
	"io"
	"log"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		log.Fatalf("Failed to create credentials provider: %v", err)
	}

	// Create an OSSClient instance.
	// Set yourEndpoint to the Endpoint of the bucket. For example, for a bucket in the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, set the Endpoint as needed.
	// Set yourRegion to the region where the bucket is located. For example, for a bucket in the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, set the region as needed.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("yourRegion"))
	// Set the signature version.
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("yourEndpoint", "", "", clientOptions...)
	if err != nil {
		log.Fatalf("Failed to create OSS client: %v", err)
	}

	// Set yourBucketName to the name of the bucket.
	bucketName := "yourBucketName" // Replace this with the actual bucket name.
	bucket, err := client.Bucket(bucketName)
	if err != nil {
		log.Fatalf("Failed to get bucket: %v", err)
	}

	// Download the object to a buffer.
	// Set yourObjectName to the full path of the object. The full path does not include the bucket name.
	objectName := "yourObjectName" // Replace this with the actual object path.
	body, err := bucket.GetObject(objectName)
	if err != nil {
		log.Fatalf("Failed to get object: %v", err)
	}
	defer body.Close()

	buf := new(bytes.Buffer)
	_, err = io.Copy(buf, body)
	if err != nil {
		log.Fatalf("Failed to copy object to buffer: %v", err)
	}

	log.Println("Buffer content:", buf.String())
}

Download an object to a local file

The following code shows how to download a specified OSS object to a local file:

package main

import (
	"io"
	"log"
	"os"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		log.Fatalf("Failed to create credentials provider: %v", err)
	}

	// Create an OSSClient instance.
	// Set yourEndpoint to the Endpoint of the bucket. For example, for a bucket in the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, set the Endpoint as needed.
	// Set yourRegion to the region where the bucket is located. For example, for a bucket in the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, set the region as needed.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("yourRegion"))
	// Set the signature version.
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("yourEndpoint", "", "", clientOptions...)
	if err != nil {
		log.Fatalf("Failed to create OSS client: %v", err)
	}

	// Set yourBucketName to the name of the bucket.
	bucketName := "yourBucketName" // Replace this with the actual bucket name.
	bucket, err := client.Bucket(bucketName)
	if err != nil {
		log.Fatalf("Failed to get bucket: %v", err)
	}

	// Download the object to a local file.
	// Set yourObjectName to the full path of the object. The full path does not include the bucket name.
	objectName := "yourObjectName" // Replace this with the actual object path.
	body, err := bucket.GetObject(objectName)
	if err != nil {
		log.Fatalf("Failed to get object: %v", err)
	}
	defer body.Close()

	// Open or create a local file.
	localFilePath := "LocalFile" // Replace this with the actual local file path.
	fd, err := os.OpenFile(localFilePath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0660)
	if err != nil {
		log.Fatalf("Failed to open or create local file: %v", err)
	}
	defer fd.Close()

	// Copy the content of the OSS object to the local file.
	_, err = io.Copy(fd, body)
	if err != nil {
		log.Fatalf("Failed to copy object to local file: %v", err)
	}

	log.Println("File downloaded successfully to", localFilePath)
}

References

For the complete sample code for streaming downloads, see GitHub sample.
For more information about the API operation for streaming downloads, see GetObject.