DataWorks:Overview

Permission description

• The Open Platform administrator and tenant administrator have read and write permissions on the developer backend. Other tenant-level roles and workspace-level roles have read-only permissions on the developer backend. For more information, see Manage permissions on global-level services.

• Only Alibaba Cloud accounts or RAM users to which the AliyunDataWorksFullAccess policy is attached can be used to configure settings on pages in the DataWorks console. For more information, see Manage permissions on the DataWorks services and the entities in the DataWorks console by using RAM policies.

Limits

The Extensions module is in public preview for users of DataWorks Enterprise Edition and is available in the following regions: China (Beijing), China (Hangzhou), China (Shanghai), China (Zhangjiakou), China (Shenzhen), China (Chengdu), US (Silicon Valley), US (Virginia), Germany (Frankfurt), Japan (Tokyo), China (Hong Kong), and Singapore.

Precautions

• If your DataWorks Enterprise Edition expires or extensions are unpublished or disabled, the extensions become invalid, and the checks and blocks performed by the extensions on related processes do not take effect and the related processes directly pass the checks.

• If an extension is in the Testing or Published state, the extension cannot automatically take effect. To make the extension take effect, the workspace administrator must go to the Extension page in SettingCenter to enable the extension.

  • If an extension is in the Testing state, the extension can be enabled only in test workspaces.

  • If an extension is in the Published state, the extension can be enabled in all workspaces in the current region.

• Before you publish an extension, you must submit it to DataWorks for review. After the extension passes the review, the extension is published.

  Note

  In most cases, an extension review can be complete within T+3 business days after you submit the extension. T indicates the point in time when you submit the extension for review.

• Extension point events are classified into tenant-level events and workspace-level events. You can select one of the preceding types of events when you register an extension. For information about the types of extension point events that are supported by DataWorks, see Supported extension point events.

• Extensions that are deployed based on Function Compute can process only a pre-event for data download.

Features

DataWorks provides multiple built-in extension points to support end-to-end big data development and governance. For example, in a workspace in standard mode, after you create a node and develop the code for the node, you must commit the node to the development environment for verification before it can be deployed to the production environment for scheduling based on scheduling configurations. If you no longer need the node, you can delete the node.

This process consists of several built-in extension points. You can add custom extension points based on your business requirements. Examples:

• Add an extension point for code review:

  After you click the Submit icon to commit a node to the development environment, the node code is reviewed based on the extension point for code review. The node is committed to the development environment only after the node code passes the review.

• Add an extension point for node review:

  After you click the Deploy icon to deploy a node to the production environment, the node is reviewed based on the extension point for node review. The node is deployed to the production environment only after the node passes the review.

The preceding examples show that you can use file committing and file deployment, which are built-in extension points, and custom extension points in the extension that you registered in DataWorks to perform the process check.

In DataWorks, you can register and deploy your application as an extension by using one of the methods described in the following table to implement custom process control.

Procedure

You can perform the following steps to implement custom process control:

1. Develop and deploy an extension.

   • For information about how to develop and deploy an extension based on a self-managed service, see Develop and deploy an extension based on a self-managed service.

   • For information about how to develop and deploy an extension based on Function Compute, see Develop and deploy an extension based on Function Compute.

   You can configure parameters and define options for an extension. This allows you to use the extension in a flexible and efficient manner. For more information, see Advanced feature: Configure extension parameters and Advanced feature: Define options for an extension.

2. Test and enable the extension.

   After the extension is developed and deployed, test whether the extension performs as expected in a test workspace. If the extension passes the test, publish the extension. Then, workspace administrators can enable the extension for workspaces as needed. For more information, see Test and enable an extension.

Supported extension point events

The following table describes different types of extension point events that can be processed by DataWorks extensions.

The following figure shows the process of committing a node in DataStudio. For more information about the check processes for various types of extension point events, see Trigger event checking during data development and Trigger event checking in Operation Center.