Hologres V2.0 introduces virtual warehouse instances as a new type of instances. You can use a virtual warehouse instance to implement read/write splitting. This topic describes the architecture of virtual warehouse instances.
Background
In Hologres V1.1, a high availability deployment architecture with read/write splitting and shared storage across multiple instances is implemented. A primary instance can be attached to multiple read-only secondary instances. These instances share storage but have isolated computing resources, which implements high availability deployment with read/write splitting. For more information, see Read/write splitting deployment of primary and secondary instances (shared storage). However, the current read-only secondary instance mode has the following limitations:
You need to create multiple instances, each with an independent endpoint. When you switch traffic, you need to change the endpoint.
Read-only secondary instances share metadata with the primary instance. You cannot configure parameters for each read-only secondary instance based on its specific role. For example, you cannot configure different numbers of replicas for each read-only secondary instance based on query high availability requirements.
To address these issues, Hologres V2.0 introduces virtual warehouse instances as a new type of instances. Computing resources are divided into different virtual warehouses, which effectively support multiple scenarios such as read/write splitting, resource isolation, and business isolation. This provides users with core capabilities such as resource isolation and elasticity. Compared with the read-only secondary instance mode, virtual warehouse instances have the following advantages:
Virtual warehouses have independent elastic scalability (elastic allocation and on-demand creation).
Virtual warehouses can share data and metadata.
You need only one endpoint to switch traffic without changing the endpoint.
Starting from Hologres V4.0, the architecture of virtual warehouse instances is upgraded. In addition to flexible read/write splitting and read/read splitting, it supports ultimate write/write splitting.
Notes
A single virtual warehouse instance can contain up to 10 virtual warehouses, each with 32 to 512 CU of resources.
Starting from Hologres V3.0.10, the maximum size of a virtual warehouse is adjusted to 1,024 CUs.
Starting from Hologres V3.0.27, the maximum size limit of virtual warehouses is removed.
In Hologres V2.0, virtual warehouse instances support data read/write tasks as follows:
Only the primary virtual warehouse can execute data writes.
All virtual warehouses can handle data reads.
For more information about granting permissions to virtual warehouses, see Grant data access permissions to virtual warehouses.
Starting from Hologres V4.0, virtual warehouse instances support data write tasks as follows:
Only the primary virtual warehouse can handle write tasks optimized by Fixed Plan, such as INSERT, UPDATE, and Fixed Copy. For more information, see Accelerate SQL execution with Fixed Plan.
All virtual warehouses can handle batch data import tasks, such as
insert into select.To handle data writes with virtual warehouse instances, enable the
hg_warehouse_enable_use_local_resourceGUC parameter. For more information, see GUC parameters.
Architecture
The following diagrams show the architectures of virtual warehouse instances.
Hologres V4.0
Hologres V4.0 supports ultimate write/write isolation for the same table group.
Hologres V2.0
Hologres V2.0 supports read/write isolation and read/read isolation.
In V2.0, to implement write/write isolation, you must split data into multiple table groups at the storage layer. For more information, see Basic concepts.
Core components related to a virtual warehouse instance:
Data storage: Hologres data storage is built on Apsara Distributed File System, providing core services such as high performance, high reliability, high availability, low cost, elastic storage space, and strong stability and security.
Virtual warehouse: A virtual warehouse provides independent, elastically scalable computing resources responsible for executing user query requests.
Cloud service components: Cloud service components include gateways, Meta Service, and Holo Master. These components provide capabilities such as metadata management, security authentication management, unified access management, and node management. Gateways are used to forward connections to different frontend (FE) nodes on virtual warehouses. For example, if you need to use the
read_warehousevirtual warehouse, the gateway forwards the connection to one of the FE nodes on theread_warehousevirtual warehouse. A gateway can forward up to 100 connections per second to FE nodes on virtual warehouses. In Hologres V2.2.22 and later, the number can be increased to 150 connections per second.