A cloud-native replacement of ReplicatedMergeTree engines on Object Storage Service-SharedMergeTree table engine family - ApsaraDB for ClickHouse

SharedMergeTree is the default table engine family in ApsaraDB for ClickHouse Enterprise Edition. It is a cloud-native replacement for the ReplicatedMergeTree engines, built on top of Object Storage Service (OSS).

How it works

ReplicatedMergeTree stores data in object storage, but keeps metadata on each ClickHouse server. Every replication event requires that metadata be copied across all replicas.

SharedMergeTree eliminates replica-to-replica communication. Instead, all replicas coordinate through shared storage and ClickHouse Keeper, which stores metadata centrally.

This architecture — asynchronous leaderless replication with centralized metadata — delivers several advantages over ReplicatedMergeTree:

Higher insert throughput
Improved throughput of background merges
Improved throughput of mutations
Faster scaling operations — because metadata does not need to be redistributed as the cluster grows, each table can support hundreds of replicas and scale out without resharding. This also enables distributed query execution across more computing resources.
More lightweight strong data consistency for SELECT queries

Create a table

SharedMergeTree-based engines are the default in ApsaraDB for ClickHouse Enterprise Edition. No configuration changes are needed — create a table the same way you always have, and the engine is applied automatically.

Basic table

CREATE TABLE my_table(
  key UInt64,
  value String
)
ENGINE = MergeTree
ORDER BY key

Because default_table_engine is set to MergeTree in Enterprise Edition, the ENGINE clause is optional:

CREATE TABLE my_table(
  key UInt64,
  value String
)
ORDER BY key

Variant engines

The following engines are automatically converted to their SharedMergeTree equivalents: ReplacingMergeTree, CollapsingMergeTree, AggregatingMergeTree, SummingMergeTree, VersionedCollapsingMergeTree, and GraphiteMergeTree.

CREATE TABLE myFirstReplacingMT
(
    `key` Int64,
    `someCol` String,
    `eventTime` DateTime
)
ENGINE = ReplacingMergeTree
ORDER BY key;

Run SHOW CREATE TABLE to confirm the actual engine in use:

SHOW CREATE TABLE myFirstReplacingMT;

The output shows the SharedMergeTree equivalent:

CREATE TABLE default.myFirstReplacingMT
( `key` Int64, `someCol` String, `eventTime` DateTime )
ENGINE = SharedReplacingMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}')
ORDER BY key
SETTINGS index_granularity = 8192

Introspection

Most system tables used for ReplicatedMergeTree introspection are available for SharedMergeTree. Two tables are not — system.replication_queue and system.replicated_fetches — because data and metadata replication does not occur. SharedMergeTree provides direct replacements:

ReplicatedMergeTree table	SharedMergeTree replacement	Description
`system.replication_queue`	`system.virtual_parts`	Stores the most recent set of current data parts and future data parts in progress, including merges, modifications, and partition deletions
`system.replicated_fetches`	`system.shared_merge_tree_fetches`	Contains primary keys and checksums being loaded into memory

Engine configuration changes

SharedMergeTree changes the behavior of three settings.

insert_quorum

Not required. All inserts to SharedMergeTree are quorum inserts (data written to the shared storage). There is no need to configure this explicitly.

insert_quorum_parallel

Not required for the same reason: all inserts are already quorum inserts (data written to the shared storage).

select_sequential_consistency

Quorum inserts are not required. Set this parameter to send additional requests to ClickHouse Keeper in SELECT queries.