Analytical platforms have historically required two storage tiers. A columnar engine handles aggregation scans by exploiting projection pruning and achieving high compression ratios. A row-oriented store handles point lookups, returning records quickly because all columns sit contiguously on disk. The two layouts are not interchangeable, and bridging them has required scheduled ETL pipelines that move data on a schedule, with the unavoidable consequence that analytical queries see data that is minutes or hours stale.
The Hybrid Serving and Analytical Processing (HSAP) paradigm collapses this separation. A single engine handles both workload shapes against shared storage, with per-table layout decisions that let each dataset pick the orientation appropriate to its access pattern. Hologres is the managed HSAP engine on Alibaba Cloud, designed to ingest from Realtime Compute for Apache Flink and to query across MaxCompute without copying data.
Figure 1. Hologres reference architecture for HSAP workloads on Alibaba Cloud.
Hologres separates compute from storage architecturally. Frontend nodes handle SQL parsing and connection management; backend workers execute scans, joins, and aggregations against a distributed file system rather than against local disks. Compute scales without redistributing data, and multiple isolated compute groups can be configured against the same storage, the mechanism behind workload separation between dashboard and ad-hoc queries on identical underlying tables.
The frontend speaks the PostgreSQL wire protocol. Standard JDBC drivers and tools that connect to PostgreSQL work without modification, and the SQL dialect tracks PostgreSQL for the majority of analytical operations. This compatibility means existing dashboards and ORM mappings connect to Hologres without rebuilding the access layer.
Column orientation is the default and serves analytical scans. Columns are stored independently in compressed, sorted segments, so a query reads only the projected columns. Compression works well because values within a column are homogeneous in type, and vectorised execution exploits the locality. The penalty appears on point lookups, where retrieving a single row requires reading from every column file separately.
Row orientation inverts the tradeoff. All columns of a record sit contiguously, so primary-key lookups return at low latency, but analytical scans must read every column even when only a few are projected. Row orientation is appropriate where Hologres backs application-tier serving and is rarely the right choice for analytical tables.
Hybrid orientation maintains both layouts for the same data, increasing storage cost in exchange for serving both query shapes from one table. The hybrid choice is appropriate where the same dataset is genuinely accessed by both shapes a user-profile table that feeds cohort analysis, and also services per-user fetch requests, which is the canonical case, and is wasteful where the workload is one-sided.
The distribution_key controls how rows are sharded across worker nodes. A high-cardinality, evenly distributed column spreads load uniformly; a low-cardinality or skewed one concentrates traffic on a small set of workers and creates a horizontal scaling ceiling that cluster expansion alone cannot lift. The distribution_key also determines join locality: two tables distributed on the same key can be joined locally without a network shuffle, which reduces the cost of joins between tables that frequently appear together.
The segment_key orders data within each shard and is conventionally aligned with a time column. Range filters on the segment key prune entire segments from the scan plan without opening their files. Most analytical queries are time-bounded, so segmenting on event_time or ingestion_time aligns the physical layout with the predominant filter.
The clustering_key defines secondary sort order within segments and is the principal lever for filter and join performance on non-temporal predicates. Where the clustering key is chosen to match common filter columns, scan locality on those filters improves materially. Both distribution and clustering are set at table creation and require table recreation to change, so the choice rewards measurement from access logs rather than estimation.
Stream ingestion uses the Hologres Flink connector, which presents a Hologres table as a Flink SQL sink. The conceptual model is upserted into a primary-keyed table, with three configurable behaviours on collision: discard the incoming record, overwrite the columns present in the incoming record, or overwrite the entire row. For idempotent stream-to-table replication where late-arriving corrections should overwrite earlier values, the column-level overwrite is the standard configuration.
The connector takes Hologres’ fixed-plan write path for primary-key inserts. The fixed-plan optimisation bypasses the SQL parser and optimiser for writes whose shape is known in advance, eliminating per-row planning overhead. Throughput on this path exceeds that of equivalent SQL INSERT … ON CONFLICT statements, and the ceiling becomes a function of worker capacity on the target instance rather than of the ingestion path itself.
Hologres queries MaxCompute tables directly through a foreign-table interface. After a one-time schema import, MaxCompute tables appear inside the Hologres namespace and behave as queryable tables in joins and subqueries; only metadata is registered with Hologres, while the data remains in MaxCompute storage.
Joins between resident Hologres tables and MaxCompute foreign tables execute through the Hologres planner, which makes a cost-based choice between pushing filter predicates down to MaxCompute or pulling the foreign table into Hologres in full. Pushdown is preferred when filters are selective, and the source can evaluate them efficiently; pull-in becomes preferable when the foreign table is small or when the filter is too complex to push. The production pattern places real-time event data in resident Hologres tables and historical records in MaxCompute, joining across the boundary at query time. Where the federation cost becomes noticeable with a repeated-query pattern, the foreign table can be materialised as a resident Hologres table via scheduled refresh.
Dictionary encoding maps low-cardinality string columns to integer codes, reducing storage size and accelerating equality predicates and group-by aggregations. Comparisons work on the integer codes rather than the original strings, and the dictionary fits in memory for fast lookup. Dictionary encoding is most effective on bounded value sets such as status or region codes; on high-cardinality columns, the dictionary grows large enough to lose its advantage.
Bitmap indexes maintain a bitmap per distinct column value, with one bit per row. Filtered scans evaluate by AND-ing or OR-ing bitmaps for the filter values, returning a row set without examining the underlying data. They are effective on low-to-medium cardinality columns when filter selectivity is high, and become counterproductive on very high cardinality columns where the maintenance cost on writes exceeds the read benefit.
Query plans are inspected through EXPLAIN ANALYZE, which returns measured execution time per plan node rather than the planner’s estimates. The standard workflow identifies the plan node consuming the bulk of the measured time, then checks whether segment pruning, dictionary lookup, or bitmap index access is used on that node; if not, it
adds the missing index or revises the clustering key on the next table iteration.
Concurrency behaviour is governed by resource groups, which partition a Hologres instance into isolated CPU and memory quotas. The mechanism addresses a problem common to shared analytical systems: a single expensive ad-hoc query can consume enough resources to delay every other query on the cluster. Dashboard query traffic and ad-hoc analyst queries should be assigned to separate resource groups, so a long-running scan cannot starve latency-sensitive widgets.
Each frontend node maintains a finite connection ceiling, and connections, including idle ones, consume frontend resources. High-concurrency clients should connect through a pooler, sized to the expected concurrent active-query count rather than the total client population. A fleet of a thousand clients issuing a few queries per minute does not require a thousand connections, only enough to cover the queries running at any given instant.
The case for Hologres rests on whether a workload genuinely exhibits both analytical and serving shapes against the same data. Where the workload is purely analytical and batch-oriented, MaxCompute alone is sufficient and more economical for cold-storage retention. Where the workload is purely transactional, a dedicated OLTP database is the appropriate choice. Hologres is suited to workloads where the same dataset must answer aggregation queries quickly and point lookups at low latency, and where the analytical view must reflect events still being written.
For deployments that extend past the basic pipeline, Hologres Binlog exposes table-level change streams that downstream Flink jobs can consume; result caching reduces compute load on dashboard queries with stable parameters; and the jsonb type allows semi-structured ingest with indexed path access where the source schema is still evolving.
Disclaimer: The views expressed herein are for reference only and don’t necessarily represent the official views of Alibaba Cloud.
99 posts | 2 followers
FollowRupal_Click2Cloud - December 15, 2023
Alibaba Cloud MaxCompute - January 22, 2021
Alibaba Cloud Big Data and AI - April 15, 2026
Alibaba Cloud MaxCompute - August 15, 2022
Alibaba Clouder - October 29, 2020
Hologres - June 16, 2022
99 posts | 2 followers
Follow
Hologres
A real-time data warehouse for serving and analytics which is compatible with PostgreSQL.
Learn More
MaxCompute
Conduct large-scale data warehousing with MaxCompute
Learn More
AnalyticDB for PostgreSQL
An online MPP warehousing service based on the Greenplum Database open source program
Learn More
PolarDB for PostgreSQL
Alibaba Cloud PolarDB for PostgreSQL is an in-house relational database service 100% compatible with PostgreSQL and highly compatible with the Oracle syntax.
Learn MoreMore Posts by PM - C2C_Yuan
