The data indexing feature lets you quickly find objects in a bucket. You can search for objects that match specific conditions, such as object name, ETag, storage class, size, and last modified time. You can also specify filter conditions and sort or aggregate the query results, which improves the efficiency of finding target objects.
Usage notes
The data indexing feature is supported in Go SDK V2.2.5 and later.
In this topic, the public endpoint of the China (Hangzhou) region is used. If you want to access OSS from other Alibaba Cloud services in the same region as OSS, use an internal endpoint. For more information about OSS regions and endpoints, see Regions and endpoints.
In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials.
In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Configure OSSClient instances.
Enable the metadata management feature for a bucket
The following code enables the metadata management feature for a specified bucket. After this feature is enabled, OSS creates a metadata index library for the bucket and builds metadata indexes for all objects in it. After the metadata index library is created, OSS performs Near Real-Time incremental scans of new files in the bucket and builds metadata indexes for them.
package main
import (
"fmt"
"os"
"github.com/aliyun/aliyun-oss-go-sdk/oss"
)
func main() {
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
provider, err := oss.NewEnvironmentVariableCredentialsProvider()
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Create an OSSClient instance.
// Set yourEndpoint to the Endpoint of the bucket. For example, for the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, set the Endpoint as needed.
// Set yourRegion to the region where the bucket is located. For example, for the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, set the region as needed.
clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
clientOptions = append(clientOptions, oss.Region("yourRegion"))
// Set the signature version.
clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
client, err := oss.New("yourEndpoint", "", "", clientOptions...)
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Enable the metadata management feature.
err = client.OpenMetaQuery("examplebucket")
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
fmt.Println("Open data query success")
}
Query the metadata index library of a bucket
The following code provides an example on how to query the metadata index library of a bucket:
package main
import (
"fmt"
"os"
"github.com/aliyun/aliyun-oss-go-sdk/oss"
)
func main() {
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
provider, err := oss.NewEnvironmentVariableCredentialsProvider()
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Create an OSSClient instance.
// Set yourEndpoint to the Endpoint of the bucket. For example, for the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, set the Endpoint as needed.
// Set yourRegion to the region where the bucket is located. For example, for the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, set the region as needed.
clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
clientOptions = append(clientOptions, oss.Region("yourRegion"))
// Set the signature version.
clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
client, err := oss.New("yourEndpoint", "", "", clientOptions...)
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Get information about the metadata index library.
result, err := client.GetMetaQueryStatus("examplebucket")
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
fmt.Printf("State:%s\n", result.State)
fmt.Printf("Phase:%s\n", result.Phase)
fmt.Printf("CreateTime:%s\n", result.CreateTime)
fmt.Printf("UpdateTime:%s\n", result.UpdateTime)
}
Query objects that meet specific conditions
The following sample code provides an example on how to query objects that meet specific conditions and list the object information based on specific fields and sorting methods:
package main
import (
"fmt"
"os"
"github.com/aliyun/aliyun-oss-go-sdk/oss"
)
func main() {
// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
provider, err := oss.NewEnvironmentVariableCredentialsProvider()
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Create an OSSClient instance.
// Set yourEndpoint to the Endpoint of the bucket. For example, for the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, set the Endpoint as needed.
// Set yourRegion to the region where the bucket is located. For example, for the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, set the region as needed.
clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
clientOptions = append(clientOptions, oss.Region("yourRegion"))
// Set the signature version.
clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
client, err := oss.New("yourEndpoint", "", "", clientOptions...)
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Query for files larger than 30 bytes, return a maximum of 10 results, and sort the results in ascending order.
query := oss.MetaQuery{
NextToken: "",
MaxResults: 10,
Query: `{"Field": "Size","Value": "30","Operation": "gt"}`,
Sort: "Size",
Order: "asc",
}
// Query for objects that meet the specified conditions and list the object information based on the specified field and sorting method.
result, err := client.DoMetaQuery("examplebucket", query)
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
fmt.Printf("NextToken:%s\n", result.NextToken)
for _, file := range result.Files {
fmt.Printf("File name: %s\n", file.Filename)
fmt.Printf("size: %d\n", file.Size)
fmt.Printf("File Modified Time:%s\n", file.FileModifiedTime)
fmt.Printf("Oss Object Type:%s\n", file.OssObjectType)
fmt.Printf("Oss Storage Class:%s\n", file.OssStorageClass)
fmt.Printf("Object ACL:%s\n", file.ObjectACL)
fmt.Printf("ETag:%s\n", file.ETag)
fmt.Printf("Oss CRC64:%s\n", file.OssCRC64)
fmt.Printf("Oss Tagging Count:%d\n", file.OssTaggingCount)
for _, tagging := range file.OssTagging {
fmt.Printf("Oss Tagging Key:%s\n", tagging.Key)
fmt.Printf("Oss Tagging Value:%s\n", tagging.Value)
}
for _, userMeta := range file.OssUserMeta {
fmt.Printf("Oss User Meta Key:%s\n", userMeta.Key)
fmt.Printf("Oss User Meta Key Value:%s\n", userMeta.Value)
}
}
}
Disable the metadata management feature for a bucket
The following code disables the metadata management feature for a specified bucket.
package main
import (
"fmt"
"os"
"github.com/aliyun/aliyun-oss-go-sdk/oss"
)
func main() {
/// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
provider, err := oss.NewEnvironmentVariableCredentialsProvider()
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Create an OSSClient instance.
// Set yourEndpoint to the Endpoint of the bucket. For example, for the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. For other regions, set the Endpoint as needed.
// Set yourRegion to the region where the bucket is located. For example, for the China (Hangzhou) region, set the region to cn-hangzhou. For other regions, set the region as needed.
clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
clientOptions = append(clientOptions, oss.Region("yourRegion"))
// Set the signature version.
clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
client, err := oss.New("yourEndpoint", "", "", clientOptions...)
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
// Disable the metadata management feature.
err = client.CloseMetaQuery("examplebucket")
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
fmt.Println("Closed data query success")
}
References
For more information about the API operation that you can call to enable the metadata management feature, see OpenMetaQuery.
For more information about the API operation that you can call to query information about a metadata index library, see GetMetaQueryStatus.
For more information about the API operation that you can call to query objects that meet specific conditions and list the object information based on specific fields and sorting methods, see DoMetaQuery.
For more information about the API operation that you can call to disable the metadata management feature, see CloseMetaQuery.