Search OSS objects by metadata and semantic content - Object Storage Service

Why use data indexing

Traditional retrieval methods have limitations that data indexing addresses:

Traditional retrieval	OSS data indexing
Complex operations: Requires iterating objects with ListObjects and building a custom metadata database.	Easy to use: No data migration or custom search system needed. Query and analyze data directly through indexes that OSS builds automatically.
Low retrieval performance: Slow and inefficient at massive scale.	High-performance retrieval: Sub-second indexing and aggregation across tens of billions of objects.
Limited search capabilities: Only supports OSS metadata-based searches.	Multimodal support: Semantic search and object feature analysis across content types.

OSS supports two retrieval methods: MetaSearch and AISearch.

Item	MetaSearch	AISearch
Description	Queries objects by metadata attributes including OSS metadata, ETags, and object tags.	Converts documents, images, videos, and audio into vectors, then retrieves objects by semantic similarity.
Use cases	Object search and statistics.	Multimodal search and complex object retrieval.
Example query	Search for objects uploaded on September 14, 2024, with a private ACL and the Standard storage class. The OSS Metadata panel filters by storage class, ACL, Object Name, Upload Type, Last Modified Time, Object Size, and Version.	Search for images related to "apple". Enter "apple" in the Semantic Content field (AI tag) and select Image under Multimedia Metadata. Filter further by OSS Metadata: storage class, ACL, Object Name (wildcard supported), Upload Type, Last Modified Time, Object Size, and Object Version.
Example result	Returns a list of objects uploaded on September 14, 2024, with a private ACL and the Standard storage class. The query returns three objects: `demo.mov` (105.5 MB), `demo.pdf` (1.39 MB), and `demo.png` (230.16 KB).	Returns a list of image objects related to "apple". For example, the object `multimodal/apple.jpg` is returned.

All supported MetaSearch metadata fields are listed in Appendix: List of fields and operators for MetaSearch.
All supported AISearch metadata fields are listed in Appendix: List of AISearch fields and operators.

Cost optimization statistics

Use metadata such as timestamps to identify unused or cold data and optimize storage costs.

Recommended: MetaSearch.
Data validation

After data processing or cleaning, compare metrics like data volume and object size to verify results.

Recommended: MetaSearch.
Data auditing

Combine metadata and semantic content to audit object content for compliance.

Recommended: AISearch.
Multimodal search

Retrieve objects by multimedia data and semantic content—ideal for chat histories, media asset libraries, and semantic search.

Recommended: AISearch.

An application uploads objects—such as images, videos, documents, and audio—to an OSS bucket.
A RAM user with OSS management permissions enables data indexing and selects MetaSearch.
OSS automatically creates a data index using the default schema, containing OSS metadata, ETags, and object tags.
The application calls the DoMetaQuery API to query objects by metadata attributes.
OSS returns objects matching the query conditions.

An application uploads objects—such as images, videos, documents, and audio—to an OSS bucket.
A RAM user with OSS management permissions enables data indexing and selects AISearch.
OSS automatically creates a data index using the default schema and an embedding model, containing OSS metadata, ETags, object tags, user metadata, multimedia metadata, and semantic content.
The application calls the DoMetaQuery API to query objects by metadata attributes and semantic content.
OSS returns objects matching the query conditions.

Get started with MetaSearch and AISearch:

Tutorials for specific use cases:

Once enabled, data indexing builds and continuously updates the metadata index with dedicated query capacity (QPS).

MetaSearch and AISearch differ in index build time, update latency, and QPS limits: