Decoderbufs is a PostgreSQL logical decoder output extension for logical replication in the Protocol Buffers format.
Prerequisites
Decoderbufs is supported on the PolarDB for PostgreSQL (Compatible with Oracle) clusters that run the following engines:
PolarDB for PostgreSQL (Compatible with Oracle) 2.0 (revision version 2.0.14.1.0 or later)
NoteYou can execute the following statements to view the revision version of a PolarDB for PostgreSQL (Compatible with Oracle) cluster:
SHOW polar_version;Decoderbufs depends on the following libraries:
PostgreSQL 9.6 or later
Protobuf-c: 1.2 or later for data serialization
PostGIS: 2.1 or later to support PostgreSQL Geo type.
Configuration and usage
Decoderbufs is automatically loaded by default. You must modify the wal_level parameter to logical.
This parameter can be modified in the console. For more information, see Configure cluster parameters. The cluster restarts after you modify the parameter. Proceed with caution.
Examples for logical replication:
Create a logical replication slot and set the codec plug-in to decoderbufs.
SELECT * FROM pg_create_logical_replication_slot('decoderbufs_demo', 'decoderbufs');Modify the published table.
View WAL logs by using decoderbufs debug mode.
SELECT data FROM pg_logical_slot_peek_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');Obtain WAL changes by using decoderbufs to update the WAL position.
SELECT data FROM pg_logical_slot_get_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');Check the WAL position of logical replicators.
SELECT * FROM pg_replication_slots WHERE slot_type = 'logical';
Data type mappings
The following table lists how current PostgreSQL type OIDs are mapped to decoderbuf fields.
PostgreSQL type OID | Decoderbuf field |
BOOLOID | datum_boolean |
INT2OID | datum_int32 |
INT4OID | datum_int32 |
INT8OID | datum_int64 |
OIDOID | datum_int64 |
FLOAT4OID | datum_float |
FLOAT8OID | datum_double |
NUMERICOID | datum_double |
CHAROID | datum_string |
VARCHAROID | datum_string |
BPCHAROID | datum_string |
TEXTOID | datum_string |
JSONOID | datum_string |
XMLOID | datum_string |
UUIDOID | datum_string |
TIMESTAMPOID | datum_string |
TIMESTAMPTZOID | datum_string |
BYTEAOID | datum_bytes |
POINTOID | datum_point |
PostGIS geometry | datum_point |
PostGIS geography | datum_point |
Usage notes
Decoderbufs depends on the following libraries:
PostgreSQL 9.6 or later
Protobuf-c:1.2 or later for data serialization
PostGIS:2.1 or later to support PostgreSQL Geo type.
Configuration and usage
Decoderbufs is automatically loaded by default.
You can configure logical replication in the postgresql.conf file.
# MODULES
shared_preload_libraries = 'decoderbufs'
# REPLICATION
wal_level = logical # minimal, archive, hot_standby, or logical (change requires restart)
max_wal_senders = 8 # max number of walsender processes (change requires restart)
wal_keep_segments = 4 # in logfile segments, 16MB each; 0 disables
#wal_sender_timeout = 60s # in milliseconds; 0 disables
max_replication_slots = 4 # max number of replication slots (change requires restart)Examples for logical replication:
Create a logical replication slot and set the codec plug-in to decoderbufs.
SELECT * FROM pg_create_logical_replication_slot('decoderbufs_demo', 'decoderbufs');Modify the published table.
View WAL logs by using decoderbufs debug mode.
SELECT data FROM pg_logical_slot_peek_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');Obtain WAL changes by using decoderbufs to update the WAL position.
SELECT data FROM pg_logical_slot_get_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');Check the WAL position of logical replicators.
SELECT * FROM pg_replication_slots WHERE slot_type = 'logical';
Data type mappings
The following table lists how current PostgreSQL type OIDs are mapped to decoderbuf fields.
PostgreSQL type OID | Decoderbuf field |
BOOLOID | datum_boolean |
INT2OID | datum_int32 |
INT4OID | datum_int32 |
INT8OID | datum_int64 |
OIDOID | datum_int64 |
FLOAT4OID | datum_float |
FLOAT8OID | datum_double |
NUMERICOID | datum_double |
CHAROID | datum_string |
VARCHAROID | datum_string |
BPCHAROID | datum_string |
TEXTOID | datum_string |
JSONOID | datum_string |
XMLOID | datum_string |
UUIDOID | datum_string |
TIMESTAMPOID | datum_string |
TIMESTAMPTZOID | datum_string |
BYTEAOID | datum_bytes |
POINTOID | datum_point |
PostGIS geometry | datum_point |
PostGIS geography | datum_point |