This topic describes how to create a Hologres source table. It also describes data definition language (DDL) syntax, parameters in the WITH clause, data type mapping, and sample code used when you create a Hologres source table.

Limits

This topic applies only to Blink 3.6.0 and later. If you use Blink 3.6.0 or its earlier version, we recommend that you update your Blink version to 3.7.0 or later.

Usage notes

  • You can use Hologres source tables to process streaming data and batch data.
  • Hologres source tables support projection pushdown. This allows you to read data only from the required columns in the Hologres source tables.
  • Blink jobs execute snapshot statements to read existing data from Hologres source tables at a high rate. After the read operation is complete, the jobs end. If the jobs fail to read data, they try to read the data again.
  • Parallel Blink jobs can read data from one or more Hologres shards. We recommend that the number of parallel Blink jobs be no more than the number of Hologres shards.
  • If you want to use Realtime Compute for Apache Flink to consume data of Hologres source tables in real time, you must enable the binary logging feature. For more information about how to enable the binary logging feature, see Subscribe to Hologres binlogs.

Introduction to Hologres

Hologres is compatible with the PostgreSQL protocol and closely connected to the big data ecosystem. Hologres allows you to analyze and process petabytes of data in high parallelism and low latency scenarios. Hologres provides an easy method for you to use the existing business intelligence (BI) tools to perform multidimensional analysis and explore your business.

DDL syntax

create table mysource(
  name varchar,
  age BIGINT,
  birthday BIGINT
) with (
  type='hologres',
  dbname='...',
  tablename='...',
  username='...',
  password='...',
  endpoint='...',
  field_delimiter='...' -- This parameter is optional. 
);

Parameters in the WITH clause

Parameter Description Required Remarks
type The type of the source table. Yes Set the value to hologres.
dbname The name of the database. Yes N/A.
tablename The name of the table.
Note If the public schema is not used, you must set the tableName parameter to schema.tableName.
Yes N/A.
username The username that is used to log on to the database. You must enter the AccessKey ID of your Alibaba Cloud account. Yes N/A.
password The password that is used to log on to the database. You must enter the AccessKey secret of your Alibaba Cloud account. Yes N/A.
endpoint The endpoint of Hologres. Yes For more information, see Endpoints for connecting to Hologres.
field_delimiter The delimiter used between rows when data is being exported.
Notice Delimiters cannot be inserted into the data.
Yes Default value: "\u0002".
bulkread Specifies whether to read full data from a column-oriented table. No Valid values:
  • true: Realtime Compute for Apache Flink reads full data from a column-oriented table.
  • false: Realtime Compute for Apache Flink does not read full data from a column-oriented table.

Data data mapping

Hologres BLINK
INT INT
INT[] ARRAY<INT>
BIGINT BIGINT
BIGINT[] ARRAY<BIGINT>
REAL FLOAT
REAL[] ARRAY<FLOAT>
DOUBLE PRECISION DOUBLE
DOUBLE PRECISION[] ARRAY<DOUBLE>
BOOLEAN BOOLEAN
BOOLEAN[] ARRAY<BOOLEAN>
TEXT VARCHAR
TEXT[] ARRAY<VARCHAR>
NUMERIC DECIMAL
DATE DATE
TIMESTAMP WITH TIMEZONE TIMESTAMP

Sample code

The following sample code shows how to create a Hologres source table in a Realtime Compute for Apache Flink job.
create table mysource(
  name varchar,
  age BIGINT,
  birthday BIGINT
) with (
  type='hologres',
  dbname='...',
  tablename='...',
  username='...',
  password='...',
  endpoint='...',
  field_delimiter='...' -- This parameter is optional. 
);


create table print_output(
 a varchar,
 b BIGINT,
 c BIGINT 
) with (
  type='print'
);


INSERT INTO print_output
SELECT 
  a, b, c
from mysource;