pg_pathman is a PolarDB for PostgreSQL extension that provides hash and range table partitioning. It handles partition creation and data migration through dedicated functions, and generates optimized execution plans for queries on partitioned tables.
To enable the partition management feature, contact us.
Feature highlights
Hash and range partitioning — partition by any supported column type, including INT, FLOAT, and DATE types, and custom domains
Automatic and manual partition management — create partitions and migrate data automatically via functions, or attach and detach existing tables manually
Optimized query planning — generates efficient execution plans for joins, subqueries, and other query patterns on partitioned tables
Runtime partition selection — uses
RuntimeAppendandRuntimeMergeAppendcustom plan nodes to select the right partitions at query timeDynamic partition filtering — the
PartitionFilternode filters partitions based on query conditionsAutomatic partition propagation — creates new partitions automatically when inserted data falls outside existing partition bounds (range partitioning only)
Direct COPY support — reads and writes partitioned tables directly using
COPY FROM/TOPartition key updates — supports updating partition keys via a trigger (avoid adding the trigger if partition key updates are not needed, as it reduces write performance)
Creation callbacks — invokes a custom callback function each time a partition is created
Non-blocking data migration — migrates data from a primary table to partitions concurrently without blocking normal operations
Foreign table support — inserts data into foreign tables managed by Foreign Data Wrappers (FDW); configure using
pg_pathman.insert_into_fdw=(disabled | postgres | any_fdw)
For the full reference, see pg_pathman on GitHub.
Prerequisites
Before you begin, ensure that you have:
A PolarDB for PostgreSQL cluster
Permission to create extensions (contact your cluster administrator if unsure)
Install the extension
CREATE EXTENSION IF NOT EXISTS pg_pathman;Verify the installed version:
SELECT extname, extversion FROM pg_extension WHERE extname = 'pg_pathman';Expected output:
extname | extversion
------------+------------
pg_pathman | 1.5
(1 row)Upgrade the extension
PolarDB for PostgreSQL upgrades pg_pathman on a regular basis. To upgrade manually, upgrade the cluster to the latest version.
Views and tables
pg_pathman creates the following configuration tables and views.
pathman_config
Stores the partition configuration for each partitioned table.
CREATE TABLE IF NOT EXISTS pathman_config (
partrel REGCLASS NOT NULL PRIMARY KEY,
attname TEXT NOT NULL,
parttype INTEGER NOT NULL,
range_interval TEXT,
CHECK (parttype IN (1, 2))
);| Column | Description |
|---|---|
partrel | OID of the primary table |
attname | Partition key column name |
parttype | Partitioning type: 1 = hash, 2 = range |
range_interval | Range covered by each partition |
pathman_config_params
Stores per-table advanced settings.
CREATE TABLE IF NOT EXISTS pathman_config_params (
partrel REGCLASS NOT NULL PRIMARY KEY,
enable_parent BOOLEAN NOT NULL DEFAULT TRUE,
auto BOOLEAN NOT NULL DEFAULT TRUE,
init_callback REGPROCEDURE NOT NULL DEFAULT 0
);| Column | Description |
|---|---|
enable_parent | Whether to include the primary table in query plans |
auto | Whether to create new partitions automatically |
init_callback | OID of the partition creation callback function |
pathman_partition_list
A view that lists all partitions and their bounds.
CREATE OR REPLACE VIEW pathman_partition_list
AS SELECT * FROM show_partition_list();
-- Columns: parent, partition, parttype, partattr, range_min, range_maxpathman_concurrent_part_tasks
A view that shows active background data migration tasks.
CREATE OR REPLACE VIEW pathman_concurrent_part_tasks
AS SELECT * FROM show_concurrent_part_tasks();
-- Columns: userid, pid, dbid, relid, processed, statusPartition management
Range partitioning
Create range partitions
Use create_range_partitions to partition a table by a numeric or date range:
create_range_partitions(
relation REGCLASS,
attribute TEXT,
start_value ANYELEMENT,
p_interval ANYELEMENT,
p_count INTEGER DEFAULT NULL,
partition_data BOOLEAN DEFAULT TRUE
)An overloaded variant accepts an INTERVAL type for date-based partitioning:
create_range_partitions(
relation REGCLASS,
attribute TEXT,
start_value ANYELEMENT,
p_interval INTERVAL,
p_count INTEGER DEFAULT NULL,
partition_data BOOLEAN DEFAULT TRUE
)| Parameter | Description |
|---|---|
relation | OID of the primary table |
attribute | Partition key column |
start_value | Lower bound of the first partition |
p_interval | Interval between partitions — use ANYELEMENT for all types, or INTERVAL for date/time keys |
p_count | Number of partitions to create |
partition_data | Whether to migrate data immediately. Set to false and use partition_table_concurrently instead |
Alternatively, use create_partitions_from_range to specify an explicit end value instead of a partition count:
create_partitions_from_range(
relation REGCLASS,
attribute TEXT,
start_value ANYELEMENT,
end_value ANYELEMENT,
p_interval ANYELEMENT,
partition_data BOOLEAN DEFAULT TRUE
)Example: partition a table by month
Create the primary table and insert test data. The partition key column must have the
NOT NULLconstraint.CREATE TABLE part_test(id int, info text, crt_time timestamp NOT NULL); INSERT INTO part_test SELECT id, md5(random()::text), clock_timestamp() + (id || ' hour')::interval FROM generate_series(1, 10000) t(id);Create 24 monthly partitions starting from October 2016. Pass
falseforpartition_datato skip immediate data migration.SELECT create_range_partitions( 'part_test'::regclass, 'crt_time', '2016-10-25 00:00:00'::timestamp, interval '1 month', 24, false );Migrate data to partitions without blocking ongoing queries:
-- Before migration, all data is still in the primary table SELECT count(*) FROM ONLY part_test; -- count: 10000 SELECT partition_table_concurrently('part_test'::regclass, 10000, 1.0); -- After migration, the primary table is empty SELECT count(*) FROM ONLY part_test; -- count: 0Disable the primary table so it is excluded from execution plans:
SELECT set_enable_parent('part_test'::regclass, false); EXPLAIN SELECT * FROM part_test WHERE crt_time = '2016-10-25 00:00:00'::timestamp;Expected output (primary table excluded):
QUERY PLAN --------------------------------------------------------------------------------- Append (cost=0.00..16.18 rows=1 width=45) -> Seq Scan on part_test_1 (cost=0.00..16.18 rows=1 width=45) Filter: (crt_time = '2016-10-25 00:00:00'::timestamp without time zone) (3 rows)
When using range partitioning:
The partition key column must have a NOT NULL constraint.Create enough partitions to cover all existing data.
Migrate data non-blocking using partition_table_concurrently.After migration, disable the primary table with set_enable_parent.Hash partitioning
Create hash partitions
create_hash_partitions(
relation REGCLASS,
attribute TEXT,
partitions_count INTEGER,
partition_data BOOLEAN DEFAULT TRUE
)| Parameter | Description |
|---|---|
relation | OID of the primary table |
attribute | Partition key column |
partitions_count | Number of partitions |
partition_data | Whether to migrate data immediately. Set to false and use partition_table_concurrently instead |
Hash partitions work with any column type — the hash function handles type conversion automatically. pg_pathman also rewrites queries transparently, so statements like SELECT * FROM part_test WHERE crt_time = '...' work correctly even with hash partitions.
Example: create 128 hash partitions
Create the primary table and insert test data:
CREATE TABLE part_test(id int, info text, crt_time timestamp NOT NULL); INSERT INTO part_test SELECT id, md5(random()::text), clock_timestamp() + (id || ' hour')::interval FROM generate_series(1, 10000) t(id);Create 128 partitions without migrating data yet:
SELECT create_hash_partitions('part_test'::regclass, 'crt_time', 128, false);Migrate data non-blocking:
SELECT partition_table_concurrently('part_test'::regclass, 10000, 1.0);Disable the primary table:
SELECT set_enable_parent('part_test'::regclass, false); EXPLAIN SELECT * FROM part_test WHERE crt_time = '2016-10-25 00:00:00'::timestamp;Expected output (only the matching partition is scanned):
QUERY PLAN --------------------------------------------------------------------------------- Append (cost=0.00..1.91 rows=1 width=45) -> Seq Scan on part_test_122 (cost=0.00..1.91 rows=1 width=45) Filter: (crt_time = '2016-10-25 00:00:00'::timestamp without time zone) (3 rows)
When using hash partitioning:
The partition key column must have a NOT NULL constraint.Migrate data non-blocking using partition_table_concurrently.After migration, disable the primary table with set_enable_parent.Migrate data to partitions
Use partition_table_concurrently to move data from a primary table to its partitions without locking.
partition_table_concurrently(
relation REGCLASS,
batch_size INTEGER DEFAULT 1000,
sleep_time FLOAT8 DEFAULT 1.0
)| Parameter | Description |
|---|---|
relation | OID of the primary table |
batch_size | Rows moved per transaction |
sleep_time | Seconds to wait before retrying a locked batch; the task ends after 60 failed attempts |
Example:
SELECT partition_table_concurrently('part_test'::regclass, 10000, 1.0);Monitor active migration tasks:
SELECT * FROM pathman_concurrent_part_tasks;Split a range partition
Split a large partition into two at a specified value:
split_range_partition(
partition REGCLASS,
split_value ANYELEMENT,
partition_name TEXT DEFAULT NULL
)| Parameter | Description |
|---|---|
partition | OID of the partition to split |
split_value | Value at which to split |
partition_name | Name for the new partition |
Example — split part_test_1 (covering 2016-10-25 to 2016-11-25) at 2016-11-10:
SELECT split_range_partition(
'part_test_1'::regclass,
'2016-11-10 00:00:00'::timestamp,
'part_test_1_2'
);After the split, data is redistributed automatically:
part_test_1: covers 2016-10-25 to 2016-11-10 (373 rows)part_test_1_2: covers 2016-11-10 to 2016-11-25 (360 rows)
Merge range partitions
Merge two adjacent range partitions into one:
merge_range_partitions(partition1 REGCLASS, partition2 REGCLASS)The partitions must be adjacent. Merging non-adjacent partitions returns an error:
ERROR: merge failed, partitions must be adjacentExample — merge part_test_1 and part_test_1_2 back into one:
SELECT merge_range_partitions('part_test_1'::regclass, 'part_test_1_2'::regclass);After the merge, part_test_1_2 is dropped and its data is merged into part_test_1 (733 rows total).
Add a range partition
Three functions are available for adding range partitions to an existing partitioned table.
Append a partition
Appends a new partition at the high end of the range, using the interval from pathman_config. Note: the tablespace parameter is accepted in the function signature but cannot be specified.
append_range_partition(
parent REGCLASS,
partition_name TEXT DEFAULT NULL,
tablespace TEXT DEFAULT NULL
)Example:
SELECT append_range_partition('part_test'::regclass);
-- Creates part_test_25 covering 2018-10-25 to 2018-11-25Prepend a partition
Prepends a new partition at the low end of the range:
prepend_range_partition(
parent REGCLASS,
partition_name TEXT DEFAULT NULL,
tablespace TEXT DEFAULT NULL
)Example:
SELECT prepend_range_partition('part_test'::regclass);
-- Creates part_test_26 covering 2016-09-25 to 2016-10-25Add a partition with an explicit range
Creates a partition for any non-overlapping range, including gaps in the existing range:
add_range_partition(
relation REGCLASS,
start_value ANYELEMENT,
end_value ANYELEMENT,
partition_name TEXT DEFAULT NULL,
tablespace TEXT DEFAULT NULL
)Example — add a partition for January 2020 without filling the gap between 2018 and 2020:
SELECT add_range_partition(
'part_test'::regclass,
'2020-01-01 00:00:00'::timestamp,
'2020-02-01 00:00:00'::timestamp
);
-- Creates part_test_27 covering 2020-01-01 to 2020-02-01Drop partitions
Drop a single partition
drop_range_partition(
partition TEXT,
delete_data BOOLEAN DEFAULT TRUE
)Set delete_data to false to move the partition's data back to the primary table before dropping. Set it to true to delete the data permanently.
Examples:
-- Move data to the primary table, then drop
SELECT drop_range_partition('part_test_1', false);
SELECT drop_range_partition('part_test_2', false);
-- Delete the partition and its data permanently
SELECT drop_range_partition('part_test_3', true);Drop all partitions
drop_partitions(
parent REGCLASS,
delete_data BOOLEAN DEFAULT FALSE
)When delete_data is false (the default), data is copied back to the primary table before dropping partitions.
Example:
SELECT drop_partitions('part_test'::regclass, false);Attach a table as a partition
Attach an existing table as a range partition of a primary table. The table must have the same schema as the primary table (same columns, same dropped-column history as tracked in pg_attribute).
attach_range_partition(
relation REGCLASS,
partition REGCLASS,
start_value ANYELEMENT,
end_value ANYELEMENT
)Example:
-- Create a standalone table with the same schema
CREATE TABLE part_test_1 (LIKE part_test INCLUDING ALL);
-- Attach it as a partition covering January 2019
SELECT attach_range_partition(
'part_test'::regclass,
'part_test_1'::regclass,
'2019-01-01 00:00:00'::timestamp,
'2019-02-01 00:00:00'::timestamp
);pg_pathman adds the inheritance relationship and check constraint automatically.
Detach a partition
Detach a partition from the primary table, converting it back to a standalone table. The data is preserved; only the inheritance relationship and constraints are removed.
detach_range_partition(partition REGCLASS)Example:
-- Before detaching: part_test has 9256 rows, part_test_2 has 733 rows
SELECT detach_range_partition('part_test_2');
-- After: part_test has 8523 rows; part_test_2 is now a standalone table with 733 rowsAdvanced partition management
Disable the primary table
After migrating all data to partitions, exclude the primary table from execution plans:
set_enable_parent(relation REGCLASS, value BOOLEAN)The default depends on the partition_data parameter used during initial partitioning:
If
partition_datawastrue, data was migrated immediately and the primary table is disabled automatically.If
partition_datawasfalse, the primary table remains enabled until you disable it explicitly.
Example:
SELECT set_enable_parent('part_test', false);Enable or disable automatic partition propagation
For range partitioning, pg_pathman can create new partitions automatically when inserted data falls outside existing partition bounds.
set_auto(relation REGCLASS, value BOOLEAN)
-- Default: true (enabled)Disable automatic partition propagation for most production workloads. If inserted data falls far outside the current range, pg_pathman must create many intermediate partitions to bridge the gap, which can take a long time. For example, inserting a row with crt_time = '2222-01-01' into a monthly-partitioned table starting in 2016 requires creating thousands of partitions.
Example:
SELECT set_auto('part_test', false);Disable pg_pathman for a table
Remove pg_pathman management from a primary table. This deletes the table's entry from pathman_config and drops any associated triggers, but leaves existing partitions and their data intact.
disable_pathman_for is irreversible. After calling this function, pg_pathman no longer manages the table's partitions, and custom scan nodes are no longer used in execution plans.
SELECT disable_pathman_for('part_test');After disabling, execution plans revert to standard PostgreSQL inheritance behavior (including the primary table in scans even if it is empty):
EXPLAIN SELECT * FROM part_test WHERE crt_time = '2017-06-25 00:00:00'::timestamp; QUERY PLAN
---------------------------------------------------------------------------------
Append (cost=0.00..16.00 rows=2 width=45)
-> Seq Scan on part_test (cost=0.00..0.00 rows=1 width=45)
Filter: (crt_time = '2017-06-25 00:00:00'::timestamp without time zone)
-> Seq Scan on part_test_10 (cost=0.00..16.00 rows=1 width=45)
Filter: (crt_time = '2017-06-25 00:00:00'::timestamp without time zone)
(5 rows)Configure a partition creation callback
Register a callback function that pg_pathman calls each time a partition is created (for both hash and range partitions):
set_init_callback(relation REGCLASS, callback REGPROC DEFAULT 0)The callback must have this signature:
part_init_callback(args JSONB) RETURNS VOIDThe args JSON object contains different fields depending on the partition type:
/* Range partition */
{
"parent": "abc",
"parttype": "2",
"partition": "abc_4",
"range_max": "401",
"range_min": "301"
}
/* Hash partition */
{
"parent": "abc",
"parttype": "1",
"partition": "abc_0"
}Example: log partition creation events
Create a callback function that records partition metadata to an audit table:
CREATE OR REPLACE FUNCTION f_callback_test(jsonb) RETURNS void AS $$ DECLARE BEGIN CREATE TABLE IF NOT EXISTS rec_part_ddl( id serial primary key, parent name, parttype int, partition name, range_max text, range_min text ); IF ($1->>'parttype')::int = 1 THEN INSERT INTO rec_part_ddl(parent, parttype, partition) VALUES (($1->>'parent')::name, ($1->>'parttype')::int, ($1->>'partition')::name); ELSIF ($1->>'parttype')::int = 2 THEN INSERT INTO rec_part_ddl(parent, parttype, partition, range_max, range_min) VALUES ( ($1->>'parent')::name, ($1->>'parttype')::int, ($1->>'partition')::name, $1->>'range_max', $1->>'range_min' ); END IF; END; $$ LANGUAGE plpgsql STRICT;Register the callback and create partitions:
CREATE TABLE tt(id int, info text, crt_time timestamp NOT NULL); SELECT set_init_callback('tt'::regclass, 'f_callback_test'::regproc); SELECT create_range_partitions( 'tt'::regclass, 'crt_time', '2016-10-25 00:00:00'::timestamp, interval '1 month', 24, false );Verify the callback was invoked for each partition:
SELECT * FROM rec_part_ddl;Expected output (24 rows, one per partition):
id | parent | parttype | partition | range_max | range_min ----+--------+----------+-----------+---------------------+--------------------- 1 | tt | 2 | tt_1 | 2016-11-25 00:00:00 | 2016-10-25 00:00:00 2 | tt | 2 | tt_2 | 2016-12-25 00:00:00 | 2016-11-25 00:00:00 ... 24 | tt | 2 | tt_24 | 2018-10-25 00:00:00 | 2018-09-25 00:00:00 (24 rows)
Usage notes
| Scenario | Guidance |
|---|---|
| Partition key column | Must have a NOT NULL constraint for both hash and range partitioning |
| Initial data migration | Pass partition_data = false when creating partitions, then call partition_table_concurrently for non-blocking migration |
| Primary table after migration | Call set_enable_parent(..., false) after all data is migrated to exclude the empty primary table from execution plans |
| Automatic propagation | Disable with set_auto(..., false) for most workloads to avoid unintended creation of large numbers of partitions |
| Partition key updates | Adding a trigger enables partition key updates but reduces write performance; only add the trigger if needed |
| Merging partitions | Only range partitions can be merged, and only if they are adjacent |
| Disabling pg_pathman | disable_pathman_for is irreversible — it removes pg_pathman management but preserves partitions and data |
What's next
pg_pathman on GitHub — full API reference and additional examples