Quick start of a Flink SQL job

Prerequisites

• An Alibaba Cloud account is created and the account balance is sufficient.
  • For more information about how to create an Alibaba Cloud account, go to the Sign up to Alibaba Cloud page page.
  • The Alibaba Cloud account has a balance of at least USD 100 or a voucher or coupon of the equivalent value.
• A role is assigned to the Alibaba Cloud account. For more information, see Assign a role to an Alibaba Cloud account.

Procedure

1. Step 1: Create a fully managed Flink instance
   Create a fully managed Flink instance that uses the pay-as-you-go billing method in the China (Beijing) region.
2. Step 2: Create an SQL job and write business code
   On the Draft Editor page, create an SQL streaming job and write DDL and DML code of the job.
3. Step 3: Start the job and view the computing results
   On the Developments page, start the job and view the job status.

Step 1: Create a fully managed Flink instance

1. Log on to the Realtime Compute for Apache Flink console.
2. In the Fully Managed Flink section, click Purchase.
3. On the buy page, configure the parameters.
   

   Section	Parameter	Example value	Description
   Basic configurations	Billing Method	Pay-as-you-go	The Subscription and Pay-as-you-go billing methods are supported.
   	Region	China (Beijing)	The Singapore (Singapore) region is supported. Note We recommend that you select the same region as the region in which upstream and downstream storage reside.
   	Zone	Zone F	We recommend that you select the same zone as the zone in which upstream and downstream storage reside.
   Network	SLB Service	-	The SLB service is selected and activated by default.
   	VPC	flink-test-vpc	We recommend that you select the same virtual private cloud (VPC) as the upstream and downstream storage.
   	vSwitch	flinktest-vsw-2ze4fyq366itq6xqp****	An IP address is assigned to each TaskManager instance and each JobManager instance of a fully managed Flink job. You can select one to five vSwitches to properly plan the CIDR blocks based on the scale of fully managed Flink jobs.
   Workspace settings	Workspace Name	flink-test	The name must be 1 to 60 characters in length and can contain letters, digits, and hyphens (-). The name must start with a letter.
   Storage	OSS Bucket	flink-test-oss	The OSS bucket is used to store job information, such as checkpoints, logs, and JAR packages. The fully managed Flink service creates the following directories in the bucket that you select to store different types of data: artifacts: The uploaded JAR files are stored in this directory. flink-jobs: The high availability (HA) information and checkpoints of fully managed Flink jobs are stored in this directory. flink-savepoints: If you click Savepoint in the console of fully managed Flink, the savepoint operation is triggered and the final savepoint file is stored in this directory. logs: If you set Log Template to OSS for your job, the logs of your job are stored in this directory. sql-artifacts: Files on which user-defined functions (UDFs) and connectors depend are stored in this directory. plan: In Expert mode, the configured resource information is stored in this directory. flink-sessionclusters: The HA information and checkpoints of session clusters are stored in this directory. Note After the fully managed Flink service is activated, OSS Bucket cannot be changed. The OSS bucket must be in the same region as the fully managed Flink service. For more information about how to select a bucket, see Usage notes.
   Monitoring	Monitoring Service	-	Prometheus Service is selected and activated by default.

4. Click Confirm Order and complete the payment to activate the fully managed Flink service.
   Note After you complete the payment, click Console. On the Fully Managed Flink tab, you can view the workspace that is being created. In most cases, the workspace can be created in 5 to 10 minutes after you complete the payment.

Step 2: Create an SQL job and write business code

1. Create an SQL job.
   1. Log on to the Realtime Compute for Apache Flink console.
   2. On the Fully Managed Flink tab, find the workspace that you want to manage and click Console in the Actions column.
   3. In the left-side navigation pane, click Draft Editor.
   4. Click New.
   5. In the New Draft dialog box, configure the parameters of the job. The following table describes the parameters.

      Parameter	Example value	Description
      Name	flink-test	The name of the job. Note The job name must be unique in the current project.
      Type	STREAM / SQL	Streaming jobs and batch jobs support the following file types: SQL JAR PYTHON Note Batch jobs are supported in Ververica Platform (VVP) 2.4.1 and later minor versions and Ververica Runtime (VVR) 3.0.1 and later minor versions.
      Deployment Target	vvp-workload	The name of the Flink cluster in which you want to deploy the job. Fully managed Flink supports two types of clusters: per-job cluster and session cluster. For more information about the differences between the two types of clusters, see Configure a development and test environment (session cluster).
      Location	Development	The folder in which the code file of the job is saved. By default, the code file of the job is stored in the Development folder. You can click the icon to the right of an existing folder to create a subfolder.

   6. Click OK.
2. Copy the following job code to the code editor.
   Create the datagen source table that generates random data, and insert the data in the name field in the datagen source table into a blackhole result table. The following example shows the job code.

   CREATE TEMPORARY TABLE datagen_source(
     randstr VARCHAR
   ) WITH (
     'connector' = 'datagen'
   );
   
   CREATE TEMPORARY TABLE print_table(
     randstr  VARCHAR
   ) WITH (
     'connector' = 'print',
     'logger' = 'true'
   );
   
   INSERT INTO print_table
   SELECT SUBSTRING(randstr,0,8) from datagen_source;

3. Click Validate to check the syntax.
4. After the verification succeeds, click Publish.

Step 3: Start the job and view the computing results

1. In the left-side navigation pane, click Deployments.
2. Find the job that you want to start and click Start in the Actions column.
   
3. In the Deployment Starting Configuration dialog box, click Confirm Running.
   After you click Confirm Running, you can view the transition process from a current state to a desired state and the final result. When the state changes to RUNNING, the job is running as expected.
4. On the Deployments page, click the name of the job that is started. On the Task Manager tab, view the computing results in logs.
   

References

• For more information about how to create a project or modify the configurations of project resources, see Create and manage a project.
• For more information about how to debug a job to improve the resource utilization of JobManager, see Configure a development and test environment (session cluster).
• For more information about how to enable the Autopilot feature, see Configure Autopilot.
• For more information about how to configure monitoring and alerts, see Configure alert rules in the console of fully managed Flink.