To develop and manage Flink tasks in DataWorks, associate a Realtime Compute for Apache Flink cluster as a computing resource in your workspace. Once associated, the cluster's Flink projects and resource queues become available in Data Studio for developing and running Flink SQL Streaming and Flink SQL Batch node tasks.
Prerequisites
Before you begin, ensure that you have:
An activated Realtime Compute for Apache Flink cluster in the same region as your DataWorks workspace. Clusters in a different region cannot be associated.
A DataWorks workspace set to Use Data Studio (New Version). The Resource Access Management (RAM) user performing this operation must be a workspace member with the Workspace Administrator role.
A Serverless resource group created and associated with the workspace.
Limitations
Region availability: This feature is available in the following regions only: China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Germany (Frankfurt), UK (London), US (Silicon Valley), and US (Virginia).
Permissions:
| Operator | Required permissions |
|---|---|
| Alibaba Cloud account | No extra permissions required |
| RAM user or RAM role | DataWorks: O&M or Workspace Administrator role, or the AliyunDataWorksFullAccess permission. See Grant a user the Workspace Administrator permissions. |
| RAM user or RAM role | Flink: The AliyunStreamFullAccess access policy, and management permissions on the Flink project. See Project authorization. |
Associate the computing resource
Log on to the DataWorks console and switch to the target region. In the left navigation pane, choose More > Management Center, select your workspace from the drop-down list, and click Go To Management Center.
In the left navigation pane, click Computing Resource.
Click Associate Computing Resource. On the Associate Computing Resource page, set the computing resource type to Fully Managed Flink to open the configuration page.
On the Associate Fully Managed Flink Computing Resource page, configure the following parameters.
NoteIf you chose to isolate the production and development environments when you created the workspace, you must select a Flink Workspace for both the production environment and the development environment.
Parameter Description Suggestion Flink Workspace The Realtime Compute for Apache Flink cluster to associate. Click Create in the drop-down menu to create a new cluster. Select the cluster you activated. Flink NameSpace The Flink project within the cluster. Click Create in the drop-down menu to create a new project. See Manage projects. Select the default project generated when you activated the cluster. Default resource queue in which namespace is deployed The resource queue used by the namespace. Click Create in the drop-down menu to create a new queue. See Manage resource queues. Select the default queue generated when you activated the cluster. Default access identity The identity used to run tasks. The default value is Executor. Keep the default unless you have a specific access control requirement. Computing Resource Instance Name A custom name for this computing resource. At runtime, select the computing resource for a task by this name. Enter a name that reflects the cluster's purpose or environment, such as flink-prod.Click OK. After association, DataWorks registers the Flink workspace and namespace as a named computing resource. When creating Flink SQL tasks in Data Studio, select this resource by its instance name to route task execution to the associated cluster.
What's next
Use this computing resource when creating Flink SQL node tasks in Data Studio: