Realtime Compute for Apache Flink:Manage Hologres catalogs

Create a Hologres table

This example creates a table named holotable in the holodb database within the holocatalog catalog.

Before you create a table:

• The connector parameter is required in the WITH clause, and its value must be hologres. Other parameters such as endpoint can be omitted.

• You cannot directly add or modify WITH parameters in a Hologres table definition. Use SQL hints in INSERT statements instead.

CREATE TABLE `holocatalog`.`holodb`.`holotable` (
  id INT,
  name STRING
) WITH (
  'connector' = 'hologres',
  'table_property.orientation' = 'column',
  'table_property.distribution_key' = 'a',
  'table_property.clustering_key' = 'b:desc',
  'table_property.bitmap_columns' = 'a,b',
  'table_property.segment_key' = 'c',
  'table_property.time_to_live_in_seconds' = '86400',
  'table_property.binlog.level' = 'replica',
  'table_property.binlog.ttl' = '86400'
);

Modify a Hologres table

Read from and write to Hologres tables

Read data from Hologres

By default, Flink reads Hologres source tables in batch mode and does not consume newly written data in real time. To read in real time, use one of the following methods:

• Configure binary logging at catalog creation (recommended for persistent settings): Enable binary logging when creating the catalog using CREATE CATALOG. See the real-time consumption example above. Then query the table normally:

  Syntax	Example
  INSERT INTO ${other_sink_table} SELECT ... FROM \${catalog_name}\.\${db_name}\.\${table_name}\;	INSERT INTO sink_table SELECT id, name FROM \holocatalog\.\holodb\.\holotable\;

• Use a SQL hint (for per-query overrides): Add the /*+ OPTIONS('binlog'='true') */ hint to the query.

  INSERT INTO sinktable
  SELECT id, name
  FROM `holocatalog`.`holodb`.`holotable`
  /*+ OPTIONS ('binlog' = 'true') */;

Write data to Hologres

Use as a CTAS destination

CTAS (Create Table As Select) creates a Hologres table and populates it from a source table in one statement. Set physical table properties in the WITH clause—they are applied when the destination table is created.

CREATE TABLE IF NOT EXISTS `${catalog_name}`.`${db_name}`.`${table_name}`
WITH (
  'connector' = 'hologres'
) AS TABLE ${other_source_table};

Example:

CREATE TABLE IF NOT EXISTS `holocatalog`.`holodb`.`holotable`
WITH (
  'connector' = 'hologres'
) AS TABLE source_table;

Use as a CDAS destination

CDAS WITH parameters apply globally—you cannot set different properties for each destination table. To set per-table properties, create the destination tables manually before starting the CDAS job. See Create a Hologres table for supported physical table properties.

CDAS (Create Database As Database) synchronizes an entire source database to Hologres.

CREATE DATABASE IF NOT EXISTS `${catalog_name}`.`${db_name}`
WITH (
  'sink.parallelism' = '5' -- Set the parallelism for each sink table
) AS DATABASE ${other_source_database};

Example:

CREATE DATABASE IF NOT EXISTS `holocatalog`.`holodb`
WITH (
  'sink.parallelism' = '5'
) AS DATABASE source_database;

WITH parameters in CDAS apply to all downstream sink tables. For details on supported parameters, see Hologres sink table.

Additional parameters: