Object Storage Service:Data validation (Go SDK V1)

Prerequisites

Before you run the examples in this document, make sure that you have:

• An OSS bucket in your Alibaba Cloud account

• Access credentials configured as environment variables (OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET). For setup instructions, see Configure access credentials.

• An OSSClient instance created using an OSS endpoint. To use a custom domain name or Security Token Service (STS), see Configure OSSClient instances.

The examples in this document use the public endpoint for the China (Hangzhou) region. To access OSS from other Alibaba Cloud services in the same region, use an internal endpoint. For more information, see Regions and endpoints.

Choose a validation method

OSS Go SDK supports two data validation methods:

Range downloads do not support CRC-64 validation.

MD5 validation

Set the Content-MD5 header when uploading an object. OSS calculates the MD5 hash of the received data and compares it with the value you provided. If the hashes do not match, the upload fails with an InvalidDigest error. Check whether the object was changed or corrupted in transit, then retry the upload.

The following example calculates an MD5 checksum and uploads an object with MD5 validation enabled.

package main

import (
	"crypto/md5"
	"encoding/base64"
	"log"
	"strings"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

// calculateMD5 returns the Base64-encoded MD5 checksum of the given content.
func calculateMD5(content string) string {
	h := md5.New()
	h.Write([]byte(content))
	return base64.StdEncoding.EncodeToString(h.Sum(nil))
}

func main() {
	// Load OSS access credentials from environment variables.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		log.Fatalf("Failed to create credentials provider: %v", err)
	}

	// Create an OSS client.
	// Replace oss-cn-hangzhou.aliyuncs.com with your bucket's endpoint.
	// Replace cn-hangzhou with your bucket's region ID.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("cn-hangzhou"))
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("https://oss-cn-hangzhou.aliyuncs.com", "", "", clientOptions...)
	if err != nil {
		log.Fatalf("Failed to create OSS client: %v", err)
	}

	// Replace examplebucket with your actual bucket name.
	bucket, err := client.Bucket("examplebucket")
	if err != nil {
		log.Fatalf("Failed to get bucket: %v", err)
	}

	content := "Hello, OSS!"            // Replace with the content to upload.
	objectName := "exampledir/example.txt" // Replace with the target object name.

	// Calculate the MD5 checksum and upload the object.
	contentMD5 := calculateMD5(content)
	err = bucket.PutObject(objectName, strings.NewReader(content), oss.ContentMD5(contentMD5))
	if err != nil {
		log.Fatalf("Failed to upload object %s: %v", objectName, err)
	}

	log.Printf("Object '%s' uploaded successfully.", objectName)
}

CRC-64 validation

CRC-64 validation is enabled by default for uploads, downloads, and copies. The SDK calculates a CRC-64 checksum on the client side and compares it with the value returned by the server. If the values do not match, the SDK returns an error.

For AppendObject operations, pass the CRC value from each append result to the next call using oss.InitCRC. This allows the SDK to chain CRC validation across multiple appends.

• First append: initialize with oss.InitCRC(0).

• Subsequent appends: initialize with oss.InitCRC(result.CRC), where result.CRC is the value returned by the previous append.

The following example appends data to an object twice and validates data integrity with CRC-64 across both operations.

package main

import (
	"log"
	"strings"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Load OSS access credentials from environment variables.
	// Before running this code, set the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		log.Fatalf("Failed to create credentials provider: %v", err)
	}

	// Create an OSS client.
	// Replace oss-cn-hangzhou.aliyuncs.com with your bucket's endpoint.
	// Replace cn-hangzhou with your bucket's region ID.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("cn-hangzhou"))
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("https://oss-cn-hangzhou.aliyuncs.com", "", "", clientOptions...)
	if err != nil {
		log.Fatalf("Failed to create OSS client: %v", err)
	}

	// Replace examplebucket with your actual bucket name.
	bucket, err := client.Bucket("examplebucket")
	if err != nil {
		log.Fatalf("Failed to get bucket: %v", err)
	}

	objectName := "exampledir/example.txt" // Replace with the target object name.

	// First append: position is 0, CRC is initialized to 0.
	request := &oss.AppendObjectRequest{
		ObjectKey: objectName,
		Reader:    strings.NewReader("Hello, "),
		Position:  0,
	}
	result, err := bucket.DoAppendObject(request, []oss.Option{oss.InitCRC(0)})
	if err != nil {
		log.Fatalf("First append failed: %v", err)
	}
	log.Printf("First append successful. Next position: %d", result.NextPosition)

	// Second append: position and CRC are taken from the previous result.
	request = &oss.AppendObjectRequest{
		ObjectKey: objectName,
		Reader:    strings.NewReader("OSS!"),
		Position:  result.NextPosition,
	}
	result, err = bucket.DoAppendObject(request, []oss.Option{oss.InitCRC(result.CRC)})
	if err != nil {
		log.Fatalf("Second append failed: %v", err)
	}
	log.Printf("Second append successful. Next position: %d", result.NextPosition)

	log.Printf("All appends completed successfully.")
}

What's next

• Multipart upload (Go SDK V1)

• Resumable upload (Go SDK V1)