Realtime Compute for Apache Flink:Manage StarRocks catalogs

UI

1. Go to the Catalogs page.

   1. Log on to the Realtime Compute for Apache Flink console. In the workspace list, find your target workspace and click Console in the Actions column.

   2. In the navigation pane on the left, click Catalogs.

2. Click Create Catalog, select StarRocks, and then click Next.

3. Configure the parameters.

   Important

   Once created, a catalog's parameters cannot be modified. To change the configuration, you must delete and recreate the catalog.

   Parameter	Description	Required	Notes
   catalog name	The name of the StarRocks catalog.	Yes	The name can contain only lowercase letters (a-z) and digits (0-9). Uppercase letters, hyphens (-), underscores (_), and other special characters are not allowed.
   endpoint	The endpoint of the StarRocks frontend (FE).	Yes	Specify the IP address and JDBC port of the FE in the format jdbc:mysql://ip:port. The port is generally 9030.
   dbname	The default database to access in StarRocks.	Yes	N/A
   username	The username for accessing StarRocks.	Yes	N/A
   password	The password for accessing StarRocks.	Yes	To prevent credential exposure, we recommend using a variable.

4. Click Confirm.

5. In the Catalogs pane on the left, view the catalog that you created.

SQL

1. In the text editor on the Scripts page, enter the command to create a StarRocks catalog.

   CREATE CATALOG <catalogname> WITH (
     'type' = 'starrocks',
     'endpoint' = '<ip>:<port>', 
     'username' = '<userName>',
     'password' = '<password>',
     'dbname' = '<dbname>'
   );

   VVR 11 or later

   Parameter	Description	Required	Notes
   catalogname	The name of the StarRocks catalog.	Yes	The name can contain only lowercase letters (a-z) and digits (0-9). Uppercase letters, hyphens (-), underscores (_), and other special characters are not allowed.
   type	The type of the catalog.	Yes	The value is fixed to starrocks.
   endpoint	The endpoint of the StarRocks FE.	No	Specify the IP address and JDBC port of the FE in the format jdbc:mysql://ip:port. The port is usually 9030.
   jdbc-url	The URL used to access the MySQL server on the FE nodes.	No	Separate multiple addresses with a comma (,). Format: jdbc:mysql://<fe_host1>:<fe_query_port1>,<fe_host2>:<fe_query_port2>. Important You can configure either the endpoint parameter or both the jdbc-url and http-url parameters. The two methods achieve the same result.
   http-url	The URL used to access the HTTP server on the FE nodes.	No	Use semicolons (;) to separate multiple addresses. Format: <fe_host1>:<fe_http_port1>;<fe_host2>:<fe_http_port2>. Important You can configure either the endpoint parameter or both the jdbc-url and http-url parameters. The two methods achieve the same result.
   username	The username for accessing StarRocks.	Yes	N/A
   password	The password for accessing StarRocks.	Yes	To prevent credential exposure, we recommend using a variable.
   dbname	The default database to access in StarRocks.	No	N/A
   default-database	The default database to access in StarRocks.	No	This parameter is compatible with community versions and has the same effect as dbname. You can use either parameter.
   table.num-buckets	The default number of buckets for each partition in tables created using the catalog.	No	No default value.
   table.char-expand-multiple	The multiplier for the length of varchar and char columns in tables created using the catalog, relative to the source table.	No	The default multiplier is 4.
   table.binary-padding	The number of characters to add to the length of varbinary and binary columns in tables created using the catalog, relative to the source table.	No	The default value is 2.
   table.key-type	The type of table to be created using the catalog.	No	Valid values: PRI: primary key table UNI: unique key table AGG: aggregate table DUP: duplicate key table Default value: PRI.
   table.keys	The key columns of tables created using the catalog.	No	No default value.
   table.distribution-keys	The distribution columns (bucket keys) of tables created using the catalog.	No	If this parameter is not specified, primary key tables use the primary key as the bucket key by default. Important Do not set this parameter to an empty string (''). Otherwise, a primary key table may fail with the error Distribution column() doesn't exist because the bucket key is empty. To use the primary key as the bucket key, leave this parameter unset.
   table.enable-null-primary-key	Specifies whether to allow nullable columns from the source to be mapped as primary key columns in StarRocks.	No	Valid values: false (default) true Important Setting this parameter to true forces Flink to treat a nullable primary key column in StarRocks as non-nullable. Ensure that the upstream input data does not contain NULL values to prevent data loss or errors.

   VVR 8 or earlier

   Parameter	Description	Required	Notes
   catalogname	The name of the StarRocks catalog.	Yes	The name can contain only lowercase letters (a-z) and digits (0-9). Uppercase letters, hyphens (-), underscores (_), and other special characters are not allowed.
   type	The type of the catalog.	Yes	The value is fixed to starrocks.
   endpoint	The endpoint of the StarRocks FE.	Yes	Specify the FE IP address and JDBC port in the format jdbc:mysql://ip:port. The port is usually 9030.
   username	The username for accessing StarRocks.	Yes	N/A
   password	The password for accessing StarRocks.	Yes	To prevent credential exposure, we recommend using a variable.
   dbname	The default database to access in StarRocks.	Yes	N/A

2. Select the CREATE CATALOG statement and click Run next to the line numbers on the left.

   CREATE CATALOG starrocks WITH (
       'type' = 'starrocks',
       'endpoint' = '<ip>:<port>',
       'username' = '<userName>',
       'password' = '<password>',
       'dbname' = '<dbname>'
   );

Read data from a StarRocks table

INSERT INTO ${other_sink_table}
SELECT ...
FROM `${catalog_name}`.`${db_name}`.`${table_name}`

INSERT INTO ${other_sink_table}
SELECT ...
FROM `${catalog_name}`.`${db_name}`.`${table_name}`
    /*+
        OPTIONS('connector' = 'starrocks', 'jdbc-url' = 'jdbc:mysql://ip:port', 'scan-url' = 'ip:port')
    */

Write data to a StarRocks table

INSERT INTO `${catalog_name}`.`${db_name}`.`${table_name}`
SELECT ... 
FROM ${other_source_table}

INSERT INTO `${catalog_name}`.`${db_name}`.`${table_name}`
 /*+
        OPTIONS('connector' = 'starrocks', 'jdbc-url' = 'jdbc:mysql://ip:port', 'load-url' = 'ip:port', 'sink.buffer-flush.interval-ms' = '5000','sink.properties.row_delimiter' = '\x02',
'sink.properties.column_separator' = '\x01')
    */
SELECT ... 
FROM ${other_source_table}

CTAS and CDAS target

use catalog sr_catalog;
CREATE TABLE IF NOT EXISTS `${table_name}`
WITH (
  ...
) AS TABLE ${other_source_table};
CREATE DATABASE IF NOT EXISTS `${db_name}`
WITH (
 ...
) AS DATABASE ${other_source_database};

For more information, see Use CTAS to synchronize data from MySQL to StarRocks by using Realtime Compute for Apache Flink.

UI

1. Go to the Catalogs page.

   1. Log on to the Realtime Compute for Apache Flink console.

   2. In the list of workspaces, find the target workspace and click Console in the Actions column.

   3. In the navigation pane on the left, click Catalogs.

2. On the Catalog List page, find the target catalog and click Delete in the Actions column.

3. In the confirmation dialog box that appears, click Delete.

4. In the Catalogs pane on the left, confirm that the target catalog is deleted.

SQL

1. In the text editor on the Scripts page, enter the following command.

   DROP CATALOG `<catalog name>`;

   Where, <catalog name> is the name of your StarRocks catalog.

   Important

   While deleting a catalog does not affect running deployments, it can disrupt unpublished deployments or those requiring suspension and resumption. Proceed with caution.

2. Select the DROP CATALOG command and click Run.

3. In the Catalogs pane on the left, confirm that the target catalog is deleted.