Quick start of a Flink SQL job

Prerequisites

• An Alibaba Cloud account is created and has a sufficient balance.
  • For more information about how to create an Alibaba Cloud account, go to the Sign up to Alibaba Cloud page page.
  • The Alibaba Cloud account has a balance of at least USD 100 or a voucher or coupon of the equivalent value.
• A role is assigned to the Alibaba Cloud account. For more information, see Assign a role to an Alibaba Cloud account.

Procedure

1. Step 1: Create a fully managed Flink instance
   Create a fully managed Flink instance that uses the pay-as-you-go billing method in the China (Beijing) region.
2. Step 2: Create an SQL job and write business code
   On the Draft Editor page, create an SQL streaming job and write DDL and DML code of the job.
3. Step 3: Start the job and view the computing results
   On the Developments page, start the job and view the job status.

Step 1: Create a fully managed Flink instance

1. Log on to the Realtime Compute for Apache Flink console.
2. In the Fully Managed Flink section, click Purchase.
3. On the buy page, enter the configuration information.
   

   Section	Parameter	Example value	Description
   Basic Configurations	Billing Method	Pay-as-you-go	The Subscription and Pay-as-you-go billing methods are supported.
   	The region where fully managed Flink is supported.	Beijing	The Singapore (Singapore) region is supported. Note We recommend that you select the same region as the upstream and downstream storage.
   	Zone	Zone F	We recommend that you select the same zone as the upstream and downstream storage.
   Network	SLB Service	-	The SLB service is selected and activated by default.
   	VPC	flink-test-vpc	We recommend that you select the same VPC as the upstream and downstream storage.
   	vSwitch	flinktest-vsw-2ze4fyq366itq6xqp****	An IP address is assigned to each TaskManager instance and each JobManager instance of a Flink job. You can select one to five vSwitches to properly plan the Classless Inter-Domain Routing (CIDR) blocks based on the scale of Flink jobs.
   Workspace	Workspace Name	flink-test	The name must be 1 to 60 characters in length and can contain letters, digits, and hyphens (-). It must start with a letter.
   Storage	OSS Bucket	flink-test-oss	The OSS bucket is used to store job information, such as checkpoints, logs, and JAR packages. The fully managed Flink service creates the following directories in the bucket that you select to store different types of data: artifacts: The uploaded JAR files are stored in this directory. flink-jobs: The high availability (HA) information and checkpoints of Flink jobs are stored in this directory. flink-savepoints: If you click Savepoint in the console of fully managed Flink, the savepoint operation is triggered and the final savepoint file is stored in this directory. logs: If you set Log Template to OSS for your job, the logs of your job are stored in this directory. sql-artifacts: Files on which user-defined functions (UDFs) and connectors depend are stored in this directory. plan: In Expert mode, the configured resource information is stored in this directory. flink-sessionclusters: The HA information and checkpoints of session clusters are stored in this directory. Note After the fully managed Flink service is activated, OSS Bucket cannot be changed. The OSS bucket must be in the same region as the fully managed Flink service. For more information about how to select an OSS bucket, see Usage notes.
   Monitoring	Monitoring Service	-	Prometheus Service is selected and activated by default.

4. Click Confirm Order and complete the payment to activate the fully managed Flink service.
   Note After you complete the payment, click Console. On the Fully Managed Flink tab, you can view the workspace that is being created. In most cases, the workspace can be created in 5 to 10 minutes after you complete the payment.

Step 2: Create an SQL job and write business code

1. Create an SQL job.
   1. Log on to the Realtime Compute for Apache Flink console.
   2. On the Fully Managed Flink tab, find the workspace that you want to manage and click Console in the Actions column.
   3. In the left-side navigation pane, click Draft Editor.
   4. Click New.
   5. In the New Draft dialog box, configure the parameters of the job. The following table describes the parameters.

      Parameter	Example value	Description
      Name	flink-test	The name of the job. Note The job name must be unique in the current project.
      Type	STREAM / SQL	Streaming jobs and batch jobs support the following file types: SQL JAR PYTHON Note Ververica Platform (VVP) 2.4.1 and its later versions and Ververica Runtime (VVR) 3.0.1 and its later versions support batch jobs.
      Deployment Target	vvp-workload	The name of the Flink cluster in which you want to deploy the job. Fully managed Flink supports two cluster types: per-job cluster and session cluster. For more information about the differences between the two types of clusters, see Configure a development and test environment (session cluster).
      Locate	Development	The folder in which the code file of the job is saved. By default, the code file of the job is stored in the Development folder. You can also click the icon next to an existing folder to create a subfolder.

   6. Click OK.
2. Copy the following job code to the code editor.
   Create the datagen source table that generates random data, and insert the data in the name field in the datagen source table into a blackhole result table. The following example shows the job code.

   CREATE TEMPORARY TABLE datagen_source(
     randstr VARCHAR
   ) WITH (
     'connector' = 'datagen'
   );
   
   CREATE TEMPORARY TABLE print_table(
     randstr  VARCHAR
   ) WITH (
     'connector' = 'print'',
     'logger' = 'true'
   );
   
   INSERT INTO print_table
   SELECT SUBSTRING(randstr,0,8) from datagen_source;

3. Click Validate to check the syntax.
4. After the verification succeeds, click Publish.

Step 3: Start the job and view the computing results

1. In the left-side navigation pane, click Deployments.
2. Find the job that you want to start and click Start in the Actions column.
   
3. In the Deployment Starting Configuration dialog box, click Confirm Running.
   After you click Confirm Running, you can view the transition process from a current state to a desired state and the final result. When the state changes to RUNNING, the job is running properly.
4. On the Deployments page, click the name of the job that is started. On the Task Manager tab, view the computing results in logs.
   

References

• For more information about how to create a project or modify the configurations of project resources, see Manage projects.
• For more information about how to debug a job to improve the resource utilization of JobManager, see Configure a development and test environment (session cluster).
• For more information about how to enable the Autopilot feature, see Configure Autopilot.
• For more information about how to configure the monitoring and alerting feature, see Configure alert rules in the console of fully managed Flink.