All Products
Search
Document Center

Hologres:Hologres source table

Last Updated:Jul 24, 2023

This topic describes how to create a Hologres source table by using Blink in exclusive mode.

Usage notes

By default, a Hologres source table reads data from a Hologres table in batch mode. In this case, all the data in the Hologres table is scanned only once. The Hologres source table does not read the data that is imported to the Hologres table after the scan. If you want to consume data in Hologres in real time, you need to subscribe to Hologres binary logs. For more information, see Subscribe to Hologres binary logs.

Limits

In Hologres, you can use Holo-blink connectors to read data from Hologres source tables and join dimension tables. Holo-blink connectors have the following limits:

  • By default, you can use Holo-blink connectors to read data only from row-oriented tables. If you want to read data from column-oriented tables, you must add bulkread='true' to the code.

  • When you create a row-oriented table that has a primary key, you must configure the primary key as the clustering key of the table. For example, you can execute the following statements to create a Hologres source table:

    begin;
    create table test(a int primary key, b text, c text[], d float8, e int8);
    call set_table_property('test', 'orientation', 'row');
    call set_table_property('test', 'clustering_key', 'a');
    commit;
  • Hologres connections are not supported by Blink in exclusive mode of versions earlier than V3.6. You must use a Holo-blink connector and reference relevant JAR files to create a Hologres source table. You can contact technical support in the DingTalk group to obtain the JAR files and execute a data definition language (DDL) statement to create a Hologres source table. For more information about how to obtain online technical support, see Obtain online support for Hologres.

DDL syntax

  • Hologres connections are supported by Blink in exclusive mode of V3.6 and later. For example, you can execute the following statements to create a Hologres source table:

    CREATE TABLE holo_dim_table 
    (
        pk VARCHAR
        ,seller_id VARCHAR
        ,seller_bc_type VARCHAR
        ,seller_tag  VARCHAR
        ,PRIMARY KEY (pk)
    ) with (
        type = 'hologres',
        `endpoint` = '<yourEndpoint>', -- The virtual private cloud (VPC) endpoint of your Hologres instance. 
        `username` = '<yourUsername>', -- The AccessKey ID of your Alibaba Cloud account. 
        `password` = '<yourPassword>', -- The AccessKey secret of your Alibaba Cloud account. 
        `dbname` = '<yourDbname>', -- The name of the Hologres database to which you want to connect. 
        `tablename` = '<yourTablename>', -- The name of the Hologres table from which you want to read data. 
        `bulkread`='true'
    );

The following table describes the parameters in the with object.

Parameter

Description

Required

type

The type of the data source.

Default value: hologres.

Yes

dbname

The name of the Hologres database.

Yes

tablename

The name of the Hologres table from which you want to read data.

Yes

username

The AccessKey ID of your Alibaba Cloud account.

You can obtain the AccessKey ID from the AccessKey Pair page.

Yes

password

The AccessKey secret of your Alibaba Cloud account.

You can obtain the AccessKey secret from the AccessKey Pair page.

Yes

endpoint

The VPC endpoint of your Hologres instance.

You can view the VPC endpoint of the Hologres instance in the Network Information section of the instance details page in the Hologres console. The VPC endpoint must contain a port number and follow the format of IP address:Port number.

Yes

bulkread

Valid values:

  • true: allows you to use column-oriented tables as Hologres source tables.

  • false: does not allow you to use column-oriented tables as Hologres source tables. Hologres source tables must use the row-oriented storage mode.

Note

If you need to use column-oriented tables as Hologres source tables, set this parameter to true.

Yes

Data type mappings

For information about the mappings between the data types supported by Blink in exclusive mode and those supported by Hologres, see Data types.

Use Blink in exclusive mode to consume Hologres binary logs in real time

In Blink-based Realtime Compute V3.7 and later, you can use Holo-blink connectors to consume binary logs in real time. For more information, see Use Realtime Compute for Apache Flink or Blink to consume Hologres binary logs in real time.