How to use the lifecycle feature with Go SDK - Object Storage Service

Not all data uploaded to Object Storage Service (OSS) requires frequent access. For reasons such as data compliance or archiving, you may need to store some data in a cold storage class. You can configure a lifecycle rule based on the last modified time to batch delete data that you no longer need. To have OSS automatically monitor data access patterns, detect cold data, and transition the cold data to a different storage class, you can configure a lifecycle rule based on the last access time. This process implements Automatic Storage Tiering and reduces your storage costs.

Usage notes

Before you configure lifecycle rules based on the last modified time or last access time of objects, make sure that you familiarize yourself with this feature. For more information, see Lifecycle rules based on the last modified time and Lifecycle rules based on the last access time.

In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions and endpoints.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Configure OSSClient instances.
To configure lifecycle rules, you must have the oss:PutBucketLifecycle permission. To query lifecycle rules, you must have the oss:GetBucketLifecycle permission. To clear lifecycle rules, you must have the oss:DeleteBucketLifecycle permission. For more information, see Attach a custom policy to a RAM user.

Set lifecycle rules

The following code provides examples of how to set lifecycle rules based on the last modified time and the last access time. To modify one or more rules after you set them, see How do I modify one or more lifecycle rule configurations?.

Transition storage classes and delete files based on the last modified time

The following code provides an example of how to set a lifecycle rule for a bucket named examplebucket to transition storage classes and delete files based on the last modified time.

package main

import (
	"fmt"
	"os"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	// Create an OSSClient instance.
	// Set yourEndpoint to the Endpoint of the bucket. For example, for the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, use the actual Endpoint.
	// Set yourRegion to the region where the bucket is located. For example, for the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, use the actual region.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("yourRegion"))
	// Set the signature version.
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("yourEndpoint", "", "", clientOptions...)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	// Specify the bucket name.
	bucketName := "examplebucket"
	// Specify lifecycle rule 1. This rule specifies that files with the prefix "foo/" expire 3 days after they are last modified.
	rule1 := oss.BuildLifecycleRuleByDays("rule1", "foo/", true, 3)

	// Automatically delete delete markers for objects in a versioning-enabled bucket when the objects have only delete markers.
	deleteMark := true
	expiration := oss.LifecycleExpiration{
		ExpiredObjectDeleteMarker: &deleteMark,
	}

	// Delete noncurrent versions of objects 30 days after they become noncurrent.
	versionExpiration := oss.LifecycleVersionExpiration{
		NoncurrentDays: 30,
	}

	// Transition noncurrent versions of objects to the Infrequent Access (IA) storage class 10 days after they become noncurrent.
	versionTransition := oss.LifecycleVersionTransition{
		NoncurrentDays: 10,
		StorageClass:   "IA",
	}

	// Specify lifecycle rule 2.
	rule2 := oss.LifecycleRule{
		ID:                   "rule2",
		Prefix:               "yourObjectPrefix",
		Status:               "Enabled",
		Expiration:           &expiration,
		NonVersionExpiration: &versionExpiration,
		NonVersionTransitions: []oss.LifecycleVersionTransition{
			versionTransition,
		},
	}

	// Specify lifecycle rule 3. This rule specifies that files with the tag key "tag1" and tag value "value1" expire 3 days after they are last modified.
	rule3 := oss.LifecycleRule{
		ID:     "rule3",
		Prefix: "",
		Status: "Enabled",
		Tags: []oss.Tag{
			{
				Key:   "tag1",
				Value: "value1",
			},
		},
		Expiration: &oss.LifecycleExpiration{Days: 3},
	}

	// Set the lifecycle rules.
	rules := []oss.LifecycleRule{rule1, rule2, rule3}
	err = client.SetBucketLifecycle(bucketName, rules)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
}

Transition the storage class for files, excluding those with a specific prefix and tag, based on the last modified time

The following code uses the Not element under the filter node to transition all files in examplebucket to the Infrequent Access storage class 30 days after their last modification time, excluding files with the prefix log, a tag with the key key1 and value value1, and a specified size.

package main

import (
	"fmt"
	"os"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Set yourBucketName to the name of your bucket.
	bucketName := "yourBucketName"

	// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Create an OSSClient instance.
	// Set yourEndpoint to the Endpoint of the bucket. For example, for the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, use the actual Endpoint.
	// Set yourRegion to the region where the bucket is located. For example, for the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, use the actual region.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("yourRegion"))
	// Set the signature version.
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("yourEndpoint", "", "", clientOptions...)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	tag := oss.Tag{
		Key:   "key1",
		Value: "value1",
	}
	// Specify the minimum size of objects to which the lifecycle rule applies.
	size := int64(500)
	// Specify the maximum size of objects to which the lifecycle rule applies.
	smallSize := int64(64500)
	filter := oss.LifecycleFilter{
		ObjectSizeGreaterThan: &size,
		ObjectSizeLessThan:    &smallSize,
		Not: []oss.LifecycleFilterNot{
			{
				Prefix: "logs/log",
				Tag:    &tag,
			},
		},
	}
	rule1 := oss.LifecycleRule{
		ID:     "mtime transition1",
		Prefix: "logs",
		Status: "Enabled",
		Transitions: []oss.LifecycleTransition{
			{
				Days:         30,
				StorageClass: oss.StorageIA,
			},
		},
		Filter: &filter,
	}
	var rules = []oss.LifecycleRule{rule1}
	err = client.SetBucketLifecycle(bucketName, rules)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	fmt.Printf("%s\n", "set lifecycle succeeded")
}

Transition storage classes based on the last access time

The following code provides an example of how to set a lifecycle rule for a bucket named `yourBucketName` to transition storage classes based on the last access time.

package main

import (
	"fmt"
	"os"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Set yourBucketName to the name of your bucket.
	bucketName := "yourBucketName"

	// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Create an OSSClient instance.
	// Set yourEndpoint to the Endpoint of the bucket. For example, for the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, use the actual Endpoint.
	// Set yourRegion to the region where the bucket is located. For example, for the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, use the actual region.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("yourRegion"))
	// Set the signature version.
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("yourEndpoint", "", "", clientOptions...)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	isTrue := true
	isFalse := false
	// Specify lifecycle rule 1. This rule transitions all objects that have the prefix "logs" and are 64 KB or smaller to the Infrequent Access storage class 30 days after they are last accessed. When these objects are accessed again, they remain in the Infrequent Access storage class.
	rule1 := oss.LifecycleRule{
		ID:     "rule1",
		Prefix: "logs",
		Status: "Enabled",
		Transitions: []oss.LifecycleTransition{
			{
				Days:                 30,
				StorageClass:         oss.StorageIA,
				IsAccessTime:         &isTrue,
				ReturnToStdWhenVisit: &isFalse,
				AllowSmallFile:       &isTrue,
			},
		},
	}
	// Specify lifecycle rule 2. This rule transitions all noncurrent versions of objects that have the prefix "dir" and are larger than 64 KB to the Infrequent Access storage class 10 days after they are last accessed. When these objects are accessed again, they are transitioned back to the Standard storage class.
	rule2 := oss.LifecycleRule{
		ID:     "rule2",
		Prefix: "dir",
		Status: "Enabled",
		NonVersionTransitions: []oss.LifecycleVersionTransition{
			{
				NoncurrentDays:       10,
				StorageClass:         oss.StorageIA,
				IsAccessTime:         &isTrue,
				ReturnToStdWhenVisit: &isTrue,
				AllowSmallFile:       &isFalse,
			},
		},
	}
	// Set the lifecycle rules.
	var rules = []oss.LifecycleRule{rule1, rule2}
	err = client.SetBucketLifecycle(bucketName, rules)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	fmt.Printf("%s\n", "set bucket lifecycle success")
}

View lifecycle rules

The following code provides an example on how to query the lifecycle rules configured for the bucket named examplebucket:

package main

import (
	"fmt"
	"os"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Set yourBucketName to the name of your bucket.
	bucketName := "yourBucketName"

	// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Create an OSSClient instance.
	// Set yourEndpoint to the Endpoint of the bucket. For example, for the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, use the actual Endpoint.
	// Set yourRegion to the region where the bucket is located. For example, for the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, use the actual region.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("yourRegion"))
	// Set the signature version.
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("yourEndpoint", "", "", clientOptions...)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	lcRes, err := client.GetBucketLifecycle(bucketName)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	for _, rule := range lcRes.Rules {
		fmt.Println("Lifecycle Rule Id:", rule.ID)
		fmt.Println("Lifecycle Rule Prefix:", rule.Prefix)
		fmt.Println("Lifecycle Rule Status:", rule.Status)
		if rule.Expiration != nil {
			fmt.Println("Lifecycle Rule Expiration Days:", rule.Expiration.Days)
			fmt.Println("Lifecycle Rule Expiration Date:", rule.Expiration.Date)
			fmt.Println("Lifecycle Rule Expiration Created Before Date:", rule.Expiration.CreatedBeforeDate)
			if rule.Expiration.ExpiredObjectDeleteMarker != nil {
				fmt.Println("Lifecycle Rule Expiration Expired Object DeleteMarker:", *rule.Expiration.ExpiredObjectDeleteMarker)
			}
		}

		for _, tag := range rule.Tags {
			fmt.Println("Lifecycle Rule Tag Key:", tag.Key)
			fmt.Println("Lifecycle Rule Tag Value:", tag.Value)
		}

		for _, transition := range rule.Transitions {
			fmt.Println("Lifecycle Rule Transition Days:", transition.Days)
			fmt.Println("Lifecycle Rule Transition Created Before Date:", transition.CreatedBeforeDate)
			fmt.Println("Lifecycle Rule Transition Storage Class:", transition.StorageClass)
			if transition.IsAccessTime != nil {
				fmt.Println("Lifecycle Rule Transition Is Access Time:", *transition.IsAccessTime)
			}
			if transition.ReturnToStdWhenVisit != nil {
				fmt.Println("Lifecycle Rule Transition Return To Std When Visit:", *transition.ReturnToStdWhenVisit)
			}

			if transition.AllowSmallFile != nil {
				fmt.Println("Lifecycle Rule Transition Allow Small File:", *transition.AllowSmallFile)
			}

		}
		if rule.AbortMultipartUpload != nil {
			fmt.Println("Lifecycle Rule Abort Multipart Upload Days:", rule.AbortMultipartUpload.Days)
			fmt.Println("Lifecycle Rule Abort Multipart Upload Created Before Date:", rule.AbortMultipartUpload.CreatedBeforeDate)
		}

		if rule.NonVersionExpiration != nil {
			fmt.Println("Lifecycle Non Version Expiration Non Current Days:", rule.NonVersionExpiration.NoncurrentDays)
		}

		for _, nonVersionTransition := range rule.NonVersionTransitions {
			fmt.Println("Lifecycle Rule Non Version Transitions Non current Days:", nonVersionTransition.NoncurrentDays)
			fmt.Println("Lifecycle Rule Non Version Transition Storage Class:", nonVersionTransition.StorageClass)
			if nonVersionTransition.IsAccessTime != nil {
				fmt.Println("Lifecycle Rule Non Version Transition Is Access Time:", *nonVersionTransition.IsAccessTime)
			}

			if nonVersionTransition.ReturnToStdWhenVisit != nil {
				fmt.Println("Lifecycle Rule Non Version Transition Return To Std When Visit:", *nonVersionTransition.ReturnToStdWhenVisit)
			}

			if nonVersionTransition.AllowSmallFile != nil {
				fmt.Println("Lifecycle Rule Non Version Allow Small File:", *nonVersionTransition.AllowSmallFile)
			}

			if rule.Filter != nil {
				if rule.Filter.ObjectSizeGreaterThan != nil {
					fmt.Println("Lifecycle Rule Filter Object Size Greater Than:", *rule.Filter.ObjectSizeGreaterThan)
				}
				if rule.Filter.ObjectSizeLessThan != nil {
					fmt.Println("Lifecycle Rule Filter Object Size Less Than:", *rule.Filter.ObjectSizeLessThan)
				}
				for _, filterNot := range rule.Filter.Not {
					fmt.Println("Lifecycle Rule Filter Not Prefix:", filterNot.Prefix)
					if filterNot.Tag != nil {
						fmt.Println("Lifecycle Rule Filter Not Tag Key:", filterNot.Tag.Key)
						fmt.Println("Lifecycle Rule Filter Not Tag Value:", filterNot.Tag.Value)
					}
				}
			}
		}
	}
}

Clear lifecycle rules

The following code provides an example on how to clear lifecycle rules in a bucket named examplebucket. If you want to delete one or more lifecycle rules, refer to How do I delete one or more lifecycle rules?.

package main

import (
	"fmt"
	"os"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Set yourBucketName to the name of your bucket.
	bucketName := "yourBucketName"

	// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Create an OSSClient instance.
	// Set yourEndpoint to the Endpoint of the bucket. For example, for the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, use the actual Endpoint.
	// Set yourRegion to the region where the bucket is located. For example, for the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, use the actual region.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("yourRegion"))
	// Set the signature version.
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("yourEndpoint", "", "", clientOptions...)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Delete all lifecycle rules.
	err = client.DeleteBucketLifecycle(bucketName)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
}

References

For the complete sample code for lifecycle rules, see the example on GitHub.
For more information about the API operation that you can call to set lifecycle rules, see PutBucketLifecycle.
For more information about the API operation that you can call to view lifecycle rules, see GetBucketLifecycle.
For more information about the API operation that you can call to delete all lifecycle rules, see DeleteBucketLifecycle.