To develop and manage Flink tasks in DataWorks, you must first associate your Realtime Compute for Apache Flink cluster as a fully managed Flink computing resource. After the cluster is associated, you can use it for data development in your DataWorks workspace.
Prerequisites
You have activated a Realtime Compute for Apache Flink cluster.
NoteWe recommend that you purchase a Flink cluster in the same Region as the DataWorks workspace where you want to create the fully managed Flink computing resource. If the regions do not match, you cannot associate the cluster to the workspace as a computing resource.
You have created a DataWorks workspace. The Resource Access Management (RAM) user who performs this operation has been added to the workspace and assigned the Workspace Administrator role.
ImportantThis feature is supported only for workspaces that are set to Use Data Studio (New Version).
You have created a Serverless resource group and associated it to the destination workspace.
Limits
Region availability: This feature is available only in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Germany (Frankfurt), UK (London), US (Silicon Valley), and US (Virginia).
Permissions:
Operator
Required permissions
Alibaba Cloud account
No extra permissions are required.
Resource Access Management (RAM) user/RAM role
DataWorks management permissions: Only workspace members with the O&M or Workspace Administrator role, or members with the
AliyunDataWorksFullAccesspermission, can create computing resources. For more information about authorization, see Grant a user the Workspace Administrator permissions.Flink service permissions:
The
AliyunStreamFullAccessaccess policy. Otherwise, you cannot configure the fully managed Flink computing resource.Management permissions on the project in the Realtime Compute for Apache Flink cluster. For more information, see Project authorization.
Go to the computing resource list page
Log on to the DataWorks console. Switch to the destination region. In the navigation pane on the left, choose . From the drop-down list, select your workspace and click Go To Management Center.
In the navigation pane on the left, click Computing Resources.
Associate a Fully Managed Flink computing resource
On the computing resources page, you can configure and associate a fully managed Flink computing resource.
Select the type of computing resource to associate.
Click Associate Computing Resource to navigate to the Associate Computing Resource page.
On the Associate Computing Resource page, set the computing resource type to Fully Managed Flink, which opens the Associate Fully Managed Flink Computing Resource configuration page.
Configure the fully managed Flink computing resource.
On the Associate Fully Managed Flink Computing Resource page, configure the parameters as described in the following table.
Parameter
Description
Flink Workspace
Select your activated Realtime Compute for Apache Flink cluster. You can also click Create in the drop-down menu to create a new cluster. For more information, see Activate a Realtime Compute for Apache Flink cluster.
NoteIf you chose to isolate the production and development environments when you created the workspace, you must select a Flink workspace for both the production and development environments.
Flink NameSpace
Select the default Flink project that was generated when you activated the Realtime Compute for Apache Flink cluster. You can also click Create in the drop-down menu to create a Flink project. For more information, see Manage projects.
Default Resource Queue in Which Namespace is Deployed
Select the default queue that was generated when you activated the Realtime Compute for Apache Flink cluster. You can also click Create in the drop-down menu to create a queue. For more information, see Manage resource queues.
Default Access Identity
The default value is Executor.
Computing Resource Instance Name
Enter a custom name for the computing resource. At runtime, you can select the computing resource for a task based on this name.
Click OK to complete the configuration.
What to do next
After you associate the fully managed Flink computing resource, you can use it to develop node tasks in Data Studio For more information, see Flink SQL Streaming nodes and Flink SQL Batch nodes.