Lindorm:Use Flink to access LindormTable

Open source HBase connector

For information about how to use the open source HBase connector to create a dimension table and a result table, see HBase connector documentation.

Open source JDBC connector

When you use the open source JDBC connector to access a LindormTable, currently only using the LindormTable as a result table is supported. For overall usage, refer to the JDBC connector official documentation. However, there are some points that require special attention, as follows:

• Dependency requirements
  Compared with the relatively broad range of dependency package versions in the official documentation, the dependency package versions used by the JDBC connector to access LindormTables are currently restricted to the following list:
  

  • flink-connector-jdbc-core-4.0.0-2.0.jar

  • flink-connector-jdbc-mysql-4.0.0-2.0.jar

  • mysql-connector-j-8.3.0.jar

  The dependency package of the MySQL JDBC driver can be downloaded from the community.

• JDBC connector parameters
  Because using as a source table and dimension table is not supported, parameters related to source table and dimension table functionality are not supported (such as parameters prefixed with scan like scan.fetch-size, and parameters prefixed with lookup like lookup.cache).
  Recommendations for some JDBC connector parameters:
  
  

  • url: We recommend that you follow Use Java JDBC APIs to develop applications for configuration.

  • username: Use the username created in the Lindorm instance.

  • password: The password of the username.

  • connector, table-name: Follow the JDBC connector community recommendations.

  • sink parameters: Fine-tune based on the actual situation of the job.

• Data type mapping
  The mapping between Flink data types and Lindorm data types can generally follow that of MySQL data types (refer to the Data type mapping section of the JDBC connector). However, some data types in Lindorm do not align with MySQL. For example, the following MySQL types claimed to be supported in the JDBC connector are not supported by Lindorm:
  

  • MEDIUMINT type

  • DATETIME type

  • UNSIGNED types other than BIGINT UNSIGNED

The following example uses Flink SQL to define a job that accesses a LindormTable via the JDBC connector. In this example, assume that a table named testflink has been defined in the Lindorm wide table engine.

# Create Flink table and start the job
CREATE TABLE source_table(
    c1 INT,
    c2 STRING
) WITH (
  'connector' = 'datagen',
  'rows-per-second' = '2',
  'fields.c2.length' = '5',
  'fields.c1.min' = '1',
  'fields.c1.max' = '100'
);


CREATE TABLE sink_table(
    c1 INT,
    c2 STRING
) WITH (
    'connector' = 'jdbc',
    'url' = 'jdbc:mysql://ld-xxxxx-proxy-lindorm.lindorm.rds.aliyuncs.com:33060/default?sslMode=disabled&allowPublicKeyRetrieval=true&useServerPrepStmts=true&useLocalSessionState=true&rewriteBatchedStatements=true&cachePrepStmts=true&prepStmtCacheSize=300&prepStmtCacheSqlLimit=50000000',
    'username' = 'root',
    'password' = 'root',
    'table-name' = 'testflink'
);


INSERT INTO sink_table SELECT * FROM source_table;

Cloud-native multi-model database Lindorm connector

• In Realtime Compute for Apache Flink job development, you can develop jobs that access the Lindorm wide table engine by using Flink SQL. For details about Realtime Compute for Apache Flink job development, see Job development overview.

• For information about how to use the connector to create a dimension table and a result table, see Cloud-native multi-model database Lindorm connector documentation.