All Products
Search
Document Center

PolarDB:Decoderbufs

Last Updated:Oct 28, 2024

Decoderbufs is a PostgreSQL logical decoder output extension for logical replication in the Protocol Buffers format.

Prerequisites

  • Decoderbufs is supported on the PolarDB for PostgreSQL (Compatible with Oracle) clusters that run the following engines:

    • PolarDB for PostgreSQL (Compatible with Oracle) 2.0 (revision version 2.0.14.1.0 or later)

    Note

    You can execute the following statements to view the revision version of a PolarDB for PostgreSQL (Compatible with Oracle) cluster:

    SHOW polar_version;
  • Decoderbufs depends on the following libraries:

    • PostgreSQL 9.6 or later

    • Protobuf-c: 1.2 or later for data serialization

    • PostGIS: 2.1 or later to support PostgreSQL Geo type.

Configuration and usage

Decoderbufs is automatically loaded by default. You must modify the wal_level parameter to logical.

Note

This parameter can be modified in the console. For more information, see Configure cluster parameters. The cluster restarts after you modify the parameter. Proceed with caution.

Examples for logical replication:

  1. Create a logical replication slot and set the codec plug-in to decoderbufs.

    SELECT * FROM pg_create_logical_replication_slot('decoderbufs_demo', 'decoderbufs');
  2. Modify the published table.

  3. View WAL logs by using decoderbufs debug mode.

    SELECT data FROM pg_logical_slot_peek_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');
  4. Obtain WAL changes by using decoderbufs to update the WAL position.

    SELECT data FROM pg_logical_slot_get_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');
  5. Check the WAL position of logical replicators.

    SELECT * FROM pg_replication_slots WHERE slot_type = 'logical';

Data type mappings

The following table lists how current PostgreSQL type OIDs are mapped to decoderbuf fields.

PostgreSQL type OID

Decoderbuf field

BOOLOID

datum_boolean

INT2OID

datum_int32

INT4OID

datum_int32

INT8OID

datum_int64

OIDOID

datum_int64

FLOAT4OID

datum_float

FLOAT8OID

datum_double

NUMERICOID

datum_double

CHAROID

datum_string

VARCHAROID

datum_string

BPCHAROID

datum_string

TEXTOID

datum_string

JSONOID

datum_string

XMLOID

datum_string

UUIDOID

datum_string

TIMESTAMPOID

datum_string

TIMESTAMPTZOID

datum_string

BYTEAOID

datum_bytes

POINTOID

datum_point

PostGIS geometry

datum_point

PostGIS geography

datum_point

Usage notes

Decoderbufs depends on the following libraries:

  • PostgreSQL 9.6 or later

  • Protobuf-c:1.2 or later for data serialization

  • PostGIS:2.1 or later to support PostgreSQL Geo type.

Configuration and usage

Decoderbufs is automatically loaded by default.

You can configure logical replication in the postgresql.conf file.

# MODULES
shared_preload_libraries = 'decoderbufs'

# REPLICATION
wal_level = logical             # minimal, archive, hot_standby, or logical (change requires restart)
max_wal_senders = 8             # max number of walsender processes (change requires restart)
wal_keep_segments = 4           # in logfile segments, 16MB each; 0 disables
#wal_sender_timeout = 60s       # in milliseconds; 0 disables
max_replication_slots = 4       # max number of replication slots (change requires restart)

Examples for logical replication:

  1. Create a logical replication slot and set the codec plug-in to decoderbufs.

    SELECT * FROM pg_create_logical_replication_slot('decoderbufs_demo', 'decoderbufs');
  2. Modify the published table.

  3. View WAL logs by using decoderbufs debug mode.

    SELECT data FROM pg_logical_slot_peek_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');
  4. Obtain WAL changes by using decoderbufs to update the WAL position.

    SELECT data FROM pg_logical_slot_get_changes('decoderbufs_demo', NULL, NULL, 'debug-mode', '1');
  5. Check the WAL position of logical replicators.

    SELECT * FROM pg_replication_slots WHERE slot_type = 'logical';

Data type mappings

The following table lists how current PostgreSQL type OIDs are mapped to decoderbuf fields.

PostgreSQL type OID

Decoderbuf field

BOOLOID

datum_boolean

INT2OID

datum_int32

INT4OID

datum_int32

INT8OID

datum_int64

OIDOID

datum_int64

FLOAT4OID

datum_float

FLOAT8OID

datum_double

NUMERICOID

datum_double

CHAROID

datum_string

VARCHAROID

datum_string

BPCHAROID

datum_string

TEXTOID

datum_string

JSONOID

datum_string

XMLOID

datum_string

UUIDOID

datum_string

TIMESTAMPOID

datum_string

TIMESTAMPTZOID

datum_string

BYTEAOID

datum_bytes

POINTOID

datum_point

PostGIS geometry

datum_point

PostGIS geography

datum_point