The Openstore engine decouples storage and compute. Developed by Alibaba Cloud Elasticsearch, this retrieval engine uses a shared storage architecture to separate data storage from compute nodes. This design addresses the high storage costs and slow scaling common in traditional multi-replica architectures, where scaling is hindered by data migration. The engine provides rapid elastic scaling and reduces storage costs while maintaining high data availability. It is ideal for general-purpose or retrieval scenarios.
Background information
In retrieval scenarios, achieving high query performance and high availability often requires multiple replicas. This configuration multiplies write and storage overhead. Additionally, when service traffic fluctuates, you cannot quickly scale nodes because shard data migration is required. This forces you to reserve extra compute resources, which results in waste.
The Openstore engine for high-performance retrieval is a key feature of the Alibaba Cloud Elasticsearch V7.10 performance-enhanced edition. To use this feature, create a V7.10 performance-enhanced instance and enable the Openstore engine.
Benefits
Scalable mass storage: The shared storage architecture lets you quickly expand remote storage space to seamlessly meet growing business demands.
Ultra-low cost and high performance:
Supports real-time data writes and updates. You do not need to configure complex index lifecycle policies. Data is automatically moved to remote shared storage for ease of use.
In a single-replica scenario, storage costs are over 40% lower than enterprise SSDs (ESSDs), and write performance increases by 40%.
In multi-replica scenarios, the storage cost savings increase with the number of replicas.
Applicability
Regions and Activation Method: This service is currently available only to whitelisted users. To enable the service, you must contact technical support (). Supported regions include the following:
China (Hangzhou): Zones J and K
China (Hong Kong): Zones B, C, and D
China (Beijing): Zones K, F, and I
China (Shanghai): Zones E, G, and L
China (Shenzhen): Zones C, D, and E
Instance version: Only V7.10 performance-enhanced instances are supported.
Instance type: The minimum instance type is 2-core 8 GB. The minimum number of data nodes is 4.
Instance storage capacity: The minimum local cache storage space for a single data node is 50 GB. The minimum total logical shared storage space is 160 GB.
Billing Method: Subscription and pay-as-you-go are supported.
Number of Zones: Single-zone and dual-zone (high availability) deployments are supported. Three-zone deployments are not currently supported.
Data writes: By default, Openstore sets the write threshold to 95% of shared storage usage. When the index data on a primary shard reaches this threshold, the system automatically blocks new writes. This prevents storage overflow and cluster failures. You can use the following API to temporarily increase the write threshold to 98%. This gives you time to perform a storage expansion:
PUT _cluster/settings { "persistent": { "apack.openstore.quota.limit.ratio": "98%" } }
Performance metrics
Metric | Data |
Access latency (local cache hit) | 0.2 ms |
Access latency (local cache miss) | 1 ms |
Access throughput (local cache miss) | Up to 350 MB/s (ESSD PL1 performance, which is related to disk size. For more information, see enterprise SSD) |
Access throughput (local cache miss) | Related to the bandwidth of the actual instance type. For more information, see c8i, compute-optimized instance family |
Scenarios | Real-time reports, online computing, real-time search, and other low-latency data access scenarios. |