Realtime Compute for Apache Flink:Getting started with a Flink SQL deployment

Prerequisites

• The RAM user or RAM role that you use to access the console of fully managed Flink has the required permissions. For more information, see Permission management.

• A workspace is created. For more information, see Activate Realtime Compute for Apache Flink.

Step 1: Create an SQL draft

1. Go to the New Draft dialog box.

   1. Log on to the Realtime Compute for Apache Flink console.

   2. On the Fully Managed Flink tab, find the workspace that you want to manage and click Console in the Actions column.

   3. In the left-side navigation pane, click SQL Editor.

2. In the upper-left corner of the SQL Editor page, click New.

3. On the SQL Scripts tab of the New Draft dialog box, click Blank Stream Draft.

   Fully managed Flink provides various code templates and supports data synchronization. Each code template is applicable to specific scenarios and provides code samples and instructions for you. You can click a template to learn about the features and related syntax of Flink and implement your business logic. For more information, see Code templates and Data synchronization templates.

4. Click Next.

5. In the New Draft dialog box, configure the parameters of the draft. The following table describes the parameters.

   Parameter	Description	Example
   Name	The name of the draft that you want to create. Note The draft name must be unique in the current project.	flink-test
   Location	The folder in which the code file of the draft is stored. You can click the icon to the right of a folder to create a subfolder.	Development
   Engine Version	The engine version of Flink that is used by the draft. For more information about engine versions, version mappings, and important time points in the lifecycle of each version, see Engine version.	vvr-6.0.7-flink-1.15

6. Click Create.

Step 2: Write code for the draft

Copy the following code to the SQL editor.

-- Create a temporary table named datagen_source. 
CREATE TEMPORARY TABLE datagen_source(
  randstr VARCHAR
) WITH (
  'connector' = 'datagen'
);

-- Create a temporary table named print_table. 
CREATE TEMPORARY TABLE print_table(
  randstr  VARCHAR
) WITH (
  'connector' = 'print',
  'logger' = 'true'
);

-- Display the data of the randstr field in the datagen_source table. 
INSERT INTO print_table
SELECT SUBSTRING(randstr,0,8) from datagen_source;

Step 3: View the configuration information

Step 4: Perform a syntax check

In the upper-right corner of the SQL Editor page, click Validate to perform a syntax check.

Check the SQL semantics of the draft, network connectivity, and the metadata information of the tables that are used by the draft. You can also click SQL Advice in the calculated results to view information about SQL risks and related optimization suggestions.

(Optional) Step 5: Debug the draft

In the upper-right corner of the SQL Editor page, click Debug.

You can enable the debugging feature to simulate deployment running, check outputs, and verify the business logic of SELECT and INSERT statements. This feature improves development efficiency and reduces the risks of poor data quality. To debug a deployment, you must create a session cluster. For more information, see Debug a deployment.

Step 6: Deploy the draft

Step 7: Start the deployment for the draft and view the startup result

1. In the left-side navigation pane, click Deployments.

2. Find the desired deployment and click Start in the Actions column.

   In the Start Job dialog box, select Initial Mode and click Start. When the deployment status changes to RUNNING, the deployment is running as expected. For more information about the parameters that you must configure when you start a deployment, see Start a deployment.

3. On the Deployments page, view the computing result.

   1. On the Deployments page, click the name of the desired deployment.

   2. On the Logs tab, click the Running Logs tab. On the Running Logs tab, click Running Task Managers and click the value in the Path, ID column.

   3. Click Logs. On the Logs tab, search for logs related to PrintSinkOutputWriter.

      

(Optional) Step 8: Cancel the deployment

If you modify the SQL code for a deployment, add or delete parameters to or from the WITH clause, or change the version of a deployment, you must publish the deployment again, cancel the deployment, and then restart the deployment to make the changes take effect. If the deployment fails and cannot reuse the state data to recover, you must cancel and then restart the deployment. For more information about how to cancel a deployment, see Cancel a deployment.

References

• For more information about how to create a JAR deployment, see Getting started with a Flink JAR deployment.

• For more information about how to create a Python deployment, see Getting started with a Flink Python deployment.

• For more information about how to ingest data into data warehouses in real time, see Ingest data into data warehouses in real time.

• After you publish an SQL draft as a deployment, you can configure automatic tuning for the deployment to improve resource utilization. For more information about automatic tuning, see Configure automatic tuning.

• For more information about how to build a real-time data warehouse by using Realtime Compute for Apache Flink and Hologres, see Build a real-time data warehouse by using Realtime Compute for Apache Flink and Hologres.

• For more information about how to build a streaming data lakehouse by using Realtime Compute for Apache Flink and Apache Paimon, see Build a streaming data lakehouse by using Realtime Compute for Apache Flink and Apache Paimon.