The AISearch feature of PolarDB for MySQL integrates vector similarity search capabilities into the database kernel. This lets you perform highly efficient similarity searches on vectors generated from unstructured data—such as text, images, and audio—while simultaneously storing and processing your structured business data. You can build powerful AI-native applications, such as semantic search, Artificial Intelligence Recommendation, and search by image, within a PolarDB cluster without needing to build and maintain a separate vector database or complex data synchronization pipelines. PolarDB offers two distinct protocols for vector search to suit different technology stacks and business scenarios: a fully-compatible MySQL protocol and the OpenSearch protocol (PolarSearch), which is compatible with the mainstream search ecosystem.
Core concepts
Vector embedding: A technique that uses a pre-trained embedding model to convert unstructured data, such as text and images, into numerical arrays (vectors). These vectors capture the deep semantic information of the data, which allows machines to understand and compare its similarity.
Similarity search (k-NN): Also known as k-nearest neighbor search, its main goal is to find the k vectors in a massive dataset that are "closest" to a given query vector. The "distance" in this context is calculated using different mathematical formulas and represents the semantic similarity of the data.
Vector index: A data structure optimized for efficient queries. It is used to avoid a full, one-by-one comparison in a massive dataset by quickly narrowing the search scope based on the distribution of vectors. This reduces search latency from seconds to milliseconds while ensuring a high recall rate.
Comparison of vector index algorithms: PolarDB supports multiple mainstream vector index algorithms. HNSW and IVF are the two most common. They have different strengths in terms of performance, resource consumption, and applicable scenarios.
HNSW (Hierarchical Navigable Small World): A graph-based index that provides high performance and a high recall rate. However, it has a large memory overhead. It is suitable for scenarios that require extremely low query latency, high precision, and where the dataset size fits within the available memory.
IVF (Inverted File): A clustering-based inverted index. It has low memory usage and is suitable for scenarios that require processing very large datasets with limited memory. However, its search precision is typically slightly lower than that of HNSW.
Key advantages
The PolarDB vector engine combines the ease of use of a relational database with the high performance of a dedicated vector database. It offers significant advantages over other technology options.
Feature | Traditional relational database | Dedicated vector database | PolarDB vector engine |
SQL support | Supported | Not supported | Supported |
Vector search | Poor performance | High performance | High performance |
Learning curve | Low | High | Low |
Ecosystem compatibility | Rich | Limited | Dual ecosystem |
Scalability | Limited | Good | Excellent |
O&M complexity | Simple | Complex | Simple |
It also has the following core advantages:
All-in-one solution: You do not need to use a separate vector database. You can process both business data and vector data in PolarDB. This simplifies the system architecture and reduces O&M costs.
Enterprise-grade reliability: It fully supports atomicity, consistency, isolation, and durability (ACID) transactions to ensure data consistency. The distributed architecture supports high availability and automatic switchover to ensure business continuity.
Seamless integration with LLMs: It has built-in inference capabilities for the Qwen large language model (LLM), which simplifies the development of AI applications.
Core capabilities and performance metrics
Search performance
Latency: P99 (99% of requests) < 10 ms, P95 (95% of requests) < 5 ms.
Throughput: Single node > 10,000 queries per second (QPS).
Precision: Recall rate > 99%.
Scalability
Data scale: Supports petabytes of vector data and billions of vectors.
Concurrency: Supports tens of thousands of concurrent queries.
Cluster size: Supports dynamic scale-out to hundreds of nodes with intelligent data sharding strategies.
Resource efficiency
Storage compression: Vector data compression ratio > 50%.
Memory usage: Effectively supports terabyte-level graph indexes using layered caching technology.
CPU utilization: CPU utilization > 80% through multi-core parallel optimization.
Use cases
PolarDB offers a dual-protocol design. You can choose the appropriate protocol based on your team's technology stack, business requirements, and performance expectations.
Comparison | MySQL protocol | OpenSearch protocol |
Access method | Standard SQL | RESTful API (compatible with Elasticsearch/OpenSearch) |
Core advantages | Integration with business data: Supports adding vector columns to existing tables, supports ACID transactions, and has a low learning curve. | Hybrid search capabilities: Supports combined queries of vectors, full-text, and scalars, with a mature ecosystem. |
Underlying dependency | Depends on In-Memory Column Index (IMCI). Vector searches are executed on IMCI read-only nodes to isolate analytical and transactional workloads. | Runs on independent search nodes (PolarSearch) to provide a service similar to a search engine. |
Data synchronization | No synchronization required. After data is written to the primary database, it is automatically visible to IMCI read-only nodes. | No synchronization required. After data is written to the primary database, it is automatically visible to search nodes. |
Use case 1: AI chat and customer service bots
Pain point: Traditional keyword matching cannot understand the true intent of user questions, which leads to inaccurate answers.
Solution: Convert the "question-answer" pairs in the knowledge base into vectors and store them. When a user asks a question, the question is also converted into a vector. Then, you can use similarity search to find the most relevant knowledge points and provide a more accurate answer.
Recommended protocol:
MySQL protocol: If you only need to implement basic semantic matching and want to quickly integrate it into an existing MySQL application, this solution is more convenient.
OpenSearch protocol: To perform complex filtering with keywords, category labels, or other criteria, you can use its hybrid search capabilities.
Use case 2: Personalized recommendation systems
Pain point: How to recommend products or content that a user might be interested in based on their historical behavior (browsing, clicking, purchasing).
Solution: Represent both users and items (products, articles, videos) as vectors. By calculating the similarity between user vectors and item vectors, you can retrieve a candidate set of items that the user might be interested in. Then, use other strategies for fine-grained ranking.
Recommended protocol:
OpenSearch protocol: The retrieval layer of a recommendation system usually has a massive amount of data and is sensitive to performance and cost. The IVF index and Product Quantization (PQ) technology provided by the OpenSearch protocol are suitable for handling massive data and controlling memory costs.
Use case 3: Search by image and multi-modal retrieval
Pain point: How to find similar images by uploading an image, or find images using a text description.
Solution: Convert both images and text into vectors and store them in PolarDB. Whether the input is an image or text, convert it into a query vector and perform a similarity search in the database.
Recommended protocol:
MySQL protocol: For scenarios that require strong consistency management of image feature vectors with business data, such as product IDs and prices, the transactional capabilities of the MySQL protocol are essential.