DataWorks:Add data source via RAM role

Prerequisites

Before you begin, make sure that you have:

• An Alibaba Cloud account, or a RAM user with both the AliyunDataWorksFullAccess and AliyunRAMFullAccess policies attached

For RAM user permission setup, see Manage RAM user permissions.

If you are using an Alibaba Cloud account, skip this prerequisite.

The following figure shows how to add permissions.

Access modes

DataWorks supports two modes for data source access:

RAM role-based authorization works with OSS, AnalyticDB for MySQL, LogHub, Tablestore, and Hologres data sources.

How AccessKey mode works

To configure an OSS data source in AccessKey mode, enter the AccessKey of an account that has permission to access a specific OSS bucket on the Configure Data Source page. Each time a sync task runs, it uses that AccessKey to read or write data.

If an AccessKey leaks, all OSS data accessible by that key is exposed.

How RAM role-based authorization works

Grant a RAM role with OSS access permissions to the DataWorks service account. When a sync task runs, the service account assumes that role through STS to access the data source. No static credentials are stored in DataWorks.

This mode also supports fine-grained permission management: assign roles with different permission scopes to different data sources, and control which users can invoke each role.

Process overview

The full workflow involves five steps and three roles:

Anyone who creates a data source (Step 3) or runs a sync task (Step 5) must first be granted the PassRole permission in Step 2. Without this permission, all sync tasks configured with that data source will fail.

Step 1: Create the role to be assumed

Only an Alibaba Cloud account or a RAM user with AliyunRAMFullAccess can perform this step.

Create a custom RAM role for the DataWorks service account to assume. By using a custom role instead of the preset AliyunDataWorksAccessingOSSRole role, you can restrict access to a specific bucket rather than all buckets in the account.

Example scenario: A company has 100 OSS buckets. The big data team needs access to only one of them. The account administrator creates a custom role named BigDataOssRole so the team can access only the required bucket, reducing the risk of unauthorized access to other buckets.

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "oss:GetObject",
                "oss:ListObjects",
                "oss:GetObjectMetadata",
                "oss:GetObjectMeta",
                "oss:GetBucketAcl",
                "oss:GetBucketInfo",
                "oss:PutObject",
                "oss:DeleteObject",
                "oss:PutBucket"
            ],
            "Resource": [
                "acs:oss:*:*:bucket_name_1",
                "acs:oss:*:*:bucket_name_1/*"
            ]
        }
    ]
}

{
    "Statement": [
        {
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "di.dataworks.aliyuncs.com"
                ]
            }
        }
    ],
    "Version": "1"
}

Step 2: Grant users permission to use the role

After defining the role, grant a policy that includes the PassRole permission to the users who will create data sources or run sync tasks. Two policy templates are available:

Template 1 — broad access (use with caution)

Allows the grantee to use all roles associated with DataWorks Data Integration:

{
    "Version": "1",
    "Statement": [
        {
            "Action": "ram:PassRole",
            "Resource": "*",
            "Effect": "Allow",
            "Condition": {
                "StringEquals": {
                    "acs:Service": "di.dataworks.aliyuncs.com"
                }
            }
        }
    ]
}

Template 2 — role-specific access (recommended)

Maps specific users to a specific role. This template restricts the grantee to using only the BigDataOssRole role.

Only an Alibaba Cloud account or a RAM user with AliyunRAMFullAccess can perform this step.

Create a policy named BigDataOssRoleAllowUse using the following content. Replace 19122324**** with the UID of your Alibaba Cloud account.

{
    "Version": "1",
    "Statement": [
        {
            "Action": "ram:PassRole",
            "Resource": "acs:ram::19122324****:role/BigDataOssRole",
            "Effect": "Allow",
            "Condition": {
                "StringEquals": {
                    "acs:Service": [
                        "oss.aliyuncs.com",
                        "di.dataworks.aliyuncs.com"
                    ]
                }
            }
        }
    ]
}

For steps to create the policy, see Create a custom policy.

After creating the policy, attach BigDataOssRoleAllowUse to the RAM users who need to create data sources or run sync tasks using the BigDataOssRole role. Those users can then select BigDataOssRole as the access identity when configuring a data source.

Assign roles with the narrowest permission scope needed. For each team or data source, create a separate role-specific PassRole policy rather than using Template 1. Review and audit role assignments regularly.

Step 3: Create the data source

After the administrator grants the PassRole permission to the data source creator, the creator can set up the data source.

1. Log on to DataWorks and go to Data Integration.

2. Create and configure an OSS data source. Set Access Mode to RAM role-based authorization. For the full configuration steps, see Configure an OSS data source.

   This topic uses an OSS data source as an example. The actual interface may vary depending on the data source type. If you are using a standard mode workspace, configure the Development or Production environment as needed.

   

   Parameter	Description
   Data source name	Letters, digits, and underscores (_) only. Cannot start with a digit or an underscore.
   Data source description	A brief description. Cannot exceed 80 characters.
   Endpoint	The OSS endpoint in the format http://oss.aliyuncs.com. The endpoint varies by region. Do not add the bucket name before oss.aliyuncs.com with a period — for example, http://xxx.oss.aliyuncs.com may pass the connectivity test but will cause synchronization failures.
   Bucket	The OSS bucket that contains the data to synchronize. Only objects in this bucket are accessible from sync tasks.
   Access Mode	Select RAM role-based authorization. DataWorks uses STS to let the service account assume the role you specify.
   Select role	Select a RAM role from the drop-down list.
   Region	Select the region from the drop-down list.

3. In the Connection configuration section, click Test connectivity for each resource group used by your sync tasks. A sync task can use only one resource group type. Test connectivity for every resource group to confirm that the Data Integration resource group can reach the data source. To test multiple resource groups at once, select them and click Batch test connectivity. For troubleshooting network connectivity issues, see Network connectivity solutions.

4. After the connectivity test passes, click Complete.

Step 4: Create a sync task

After the data source is ready, a developer can create a sync task in Data Studio based on that data source. For detailed steps, see Configure a sync task.

Step 5: Run the sync task

The task executor runs the sync task on the Data Studio or Operation Center page.

The task executor must have been granted the PassRole permission in Step 2. Without this permission, the task will fail.

What's next

• To learn more about data source configuration for other data source types, see the relevant data source configuration topics in DataWorks Data Integration.

• To manage permissions and roles across teams, use Template 2 from Step 2 to create role-specific PassRole policies for each team.