Intelligent Search (PolarSearch) - PolarDB - Alibaba Cloud Documentation Center

PolarSearch is a high-performance, distributed data search and analytics engine developed by PolarDB and built on OpenSearch. It is compatible with the Elasticsearch and OpenSearch ecosystems. You do not need to manually synchronize data from PolarDB to other search platforms. Instead, you can use APIs or an SDK to perform millisecond-level full-text search, vector search, and intelligent analysis on multimodal data, such as text documents, image features, and logs.

With PolarSearch, you can:

Full-text search

curl -X GET "http://<endpoint>:<port>/articles/_search" -H "Content-Type:application/json" -d '
{
  "query": {
    "match": {
      "content": "PolarSearch"
    }
  }
}'

Vector search

curl -X GET "http://<endpoint>:<port>/my-vector-index/_search" -H "Content-Type:application/json" -d '
{
  "size": 2,
  "query": {
    "knn": {
      "vector_field": {
        "vector": [0.1, 0.5, -0.3, 0.8],
        "k": 2
      }
    }
  }
}'

Technical architecture

PolarSearch is built on PolarStore's distributed shared storage and a cloud-native storage-compute decoupled architecture. It deeply integrates a proprietary intelligent search engine with a distributed computing framework. It supports Elasticsearch DSL syntax and protocols. PolarSearch stores, analyzes, and performs real-time, multi-path fused search on massive, heterogeneous data at the petabyte scale. This lets you quickly build highly concurrent and highly available data search services and unlock the value of your data.

Product value

Improved efficiency: Eliminates the need to manually configure data synchronization links from or PostgreSQL to a retrieval engine. This reduces retrieval workload latency from minutes to milliseconds and shortens the development cycle by 50%.
Cost optimization: You can replace traditional multi-engine, multi-system architectures, such as those that use a database, file storage, and a compute engine, with PolarSearch. The multi-tier distributed shared storage of PolarFS reduces the total cost of ownership (TCO) by 40%.
Business innovation: You can use unstructured data storage and mining, AI-powered vector search, and other capabilities to build AI infrastructure, such as personalized recommendation systems, retrieval-augmented generation (RAG) knowledge bases, and agent memory stores.

Scenarios

E-commerce content platforms and SaaS services

Fuzzy search, semantic matching, and personalized recommendations for product titles and product pages.
Real-time keyword analysis and sentiment mining for user reviews and user-generated content (UGC).

Enterprise RAG knowledge bases and document management

Full-text search and document retrieval for PDFs, Word documents, and other files.
Vectorized storage of image features to support similar-image search.

Agent memory stores and intelligent agent data management

Short-term memory: includes current conversation context, session context, and temporary variables.
Long-term memory: stores user preferences, historical queries, and large language model (LLM) parameters—data from long-running interactions.

Log analysis and service monitoring

Real-time search, aggregation, and anomaly alerts for petabyte-scale log data.
Association analysis across multiple log fields and visualization report generation.

Internet of Things (IoT) and real-time IoT data streams

High-concurrency writes and fast retrieval of IoT device time series data.
Multi-condition filtering and dynamic aggregation of sensor data streams.

Core features

High availability and scalability

The distributed architecture provides automatic load balancing. If a node fails, failover is seamless. Service availability is 99.99%.
The service supports online, dynamic scaling. You can scale storage and compute resources on demand to handle data volumes in the hundreds of millions.

Intelligent search engine

PolarSearch builds inverted secondary indexes on InnoDB primary tables in PolarDB RW nodes and supports transaction-level visibility.
It uses an optimizer to identify full-text search requests on InnoDB primary tables and automatically forwards them to search nodes.
It supports hybrid indexing across multiple dimensions, such as text tokenization, semantic embeddings, and numeric ranges, to boost query performance by 10x or more.
Built-in Chinese NLP enhancement models support synonym expansion, pinyin correction, and intent recognition.

Multimodal data fusion

PolarSearch stores scalar forward indexes, full-text inverted indexes, vectors, and other data types in a single system and supports multi-path fused search.
It provides plug-ins to store, extract, and parse massive, heterogeneous unstructured data, such as images and documents.

Real-time search and aggregation and analysis

Data is searchable within hundreds of milliseconds of ingestion. It supports complex filtering, bucketing, and Top-K sorting.
Built-in functions include tumbling window calculations for time series data and geofence checks for location-based data.