This topic uses two basic services as examples to describe how to create and manage a project. Before you create and manage a project, we recommend that you read Security management and Target users to learn about the security models of MaxCompute and DataWorks.
Create an ETL project
In this scenario, multiple users work together as members in an extract, transform, and load (ETL) project. This project involves development, debug, and publish procedures.
- DataWorks enables multiple users to work together in one project.
- DataWorks provides basic roles such as Project Manager, Development, O&M, Deployment, and Visitor, which can be assigned to members to help divide responsibilities.
- DataWorks enables you to create and distinguish between development and production projects. This helps to manage the permissions to view production data and ensures that each project goes through development, debug, and publish procedures.
- Create a project.
For details about how to create the project, see Create a workspace. The following figure shows the parameter settings for the project.
- If you set Mode to Development and Production Environments, one DataWorks workspace is bound to two MaxCompute projects: one development project and one production project.
- The Identity to Access MaxCompute for the development project is Private Account. The project members use their private accounts to compile and debug code.
- The Identity to Access MaxCompute for the production project is Workspace Owner. This is to ensure that the production project runs smoothly and securely and to limit the permissions of the project members to submit jobs, delete production tables, and modify project data.
- Add members to the development project.
Add members to the development project and assign roles to the members in DataWorks. The system automatically assigns roles to RAM users in the development project. The following are the roles available:
- Project Manager
A user with the Project Manager role in DataWorks has all the permissions of the Development and O&M roles and can operate the project such as adding members, deleting members, and assign custom resource groups to roles. This user is also assigned the role_project_admin role in MaxCompute.
A user with the Development role in DataWorks can design UIs for compiling code and maintain workflows in Data Analytics. This user is also assigned the role_project_dev role in MaxCompute.
A user with the O&M role in DataWorks can manage all tasks in Maintenance Center. In MaxCompute, this user is also assigned the role_project_pe role.
A user with the Deployment role in DataWorks can review code and decide whether to submit the code to users with the O&M role. This user is also assigned the role_project_deploy role in MaxCompute.
A user with the Visitor role in DataWorks can only view workflows and code in Data Analytics. In MaxCompute, this user is also assigned the role_project_guest role.
- Safety Manager
A user with the Safety Manager role in DataWorks has only the Data Security Guard permission. In MaxCompute, this user is also assigned the role_project_security role.
- Project Manager
- Run a task for debugging code.
Log on to the DataWorks console as a member with the Development role. Then navigate to Data Analytics and debug your code. If required, you can apply for the permissions for production tables in Data Analytics.
- Publish the task to the production project.
Package the task, and ask a user with the O&M role to review your code. You need to personally notify this user of the code review request. After reviewing your code, this user packages the task and publishes it to the production project only upon approval. For more information, see Publish a task.
- Test the production task.
After your task is published to the production project, navigate to Maintenance Center and test your task as a member with the Development role. If the task is executed, view logs to check whether the task execution is successful. Furthermore, you can view the result tables in Data Analytics to check whether output data is properly generated. By default, private accounts do not have the permissions for the tables that are generated in the production project. If your private account requires the permissions, you can navigate to Data Management to apply for them.
- DataWorks enables multiple users to compile code in Data Analytics. All the members in the development project can view the code. Some members can even edit the code after they obtain the edit permission. As a result, some crucial, security-sensitive code has the potential risk of being leaked. We recommend that you group confidential tasks and data into a separate project, on which only the specified users can operate.
- In the production project, only the project owner account has the permissions to create tables, functions, and resources in MaxCompute. As a result, you may find that you create a table but the table owner is not your private account, or that you do not have the permissions to view the tables that you create.
- The development and production projects share one project owner account. Do not publish a task to the production project, read and write the production tables into the development project, and then obtain production data from the development project.
Create a project in Single Environment mode
This project provides a limited number of services, for which the same roles are used. No new services will be added to the project in the future. For example, a carrier only wants to obtain data for analysis and does not need to compile code. In this example, the carrier requires only the query and download services for obtaining data from other projects.
- The owner of this project is the same as the owner of the development or production project from which data is to be obtained.
- The Identity to Access MaxCompute for this project is set to Private Account, so that each member can use their private accounts to query and download data.
- Permissions are properly defined for the default role that is assigned to each member of this project in DataWorks after the Identity to Access MaxCompute is set to Private Account. This is to enable each member to have only the permissions to operate their own tables.
- Create a project.
For details about how to create the project, see Create a workspace. The following figure shows the parameter settings for this project.
- Create MaxCompute custom roles and grant permissions to them by using the project owner account.
For more information, see Client.
create role custom_dev;--Create a custom role. grant List, CreateInstance,CreateTable,CreateFunction,CreateResource on project prj_name to role custom_dev;--Grant permissions to the custom role.
- Enable Allow object creators to access objects for the project in MaxCompute by using the project owner account.
Alternatively, navigate to MaxCompute Management, and enable Allow object creators to access objects in Basic Settings.
set ObjectCreatorHasAccessPermission=true; --This parameter is set to true by default. To view the parameter setting, run the following command: show SecurityConfiguration;
- Add members to the project.
Add RAM users as members in DataWorks. For example, after you add a member with the Development role in DataWorks, this member is assigned the role_project_dev role in MaxCompute. To view the members in the project, run the show grants for ram$Alibaba Cloud Account:RAM User; command by using the project owner account.
- Modify the permissions of new members in MaxCompute by using the project owner account.
revoke role_project_dev from ram$Alibaba Cloud Account:RAM User; --Remove a new member from its default role. grant custom_dev to ram$Alibaba Cloud Account:RAM User; --Assign a custom role to a new member.
- If you assign a member with its default role in DataWorks again after you remove this member from its default role, the role_project_dev role in MaxCompute is also assigned to this member.
- Each member can view only their own tables (objects). However, each member can view their own tasks in addition to the tasks that are created by other members.
- The members in this project can query the tables from other projects only after they apply for the permissions in Data Management in DataWorks. Alternatively, you can add these tables to a package, install the package in this project, and then grant the package to the members. For more information, see Manage users, roles, and permissions.