Document Center
DataWorks
Update Records
Release notes of key features
Version history
DataWorks V2.0
DataWorks V3.0
Change history
Product Introduction
What is DataWorks?
Basic concepts
Scenarios
Data development process
Basic mode and standard mode
Pricing
Purchase guide
Billing
View spending details
Pay-as-you-go
Subscription (pre-payment)
DataWorks exclusive resources
DataWorks resource plans
DataWorks advanced editions
Appendix
DataWorks resource groups
Feature comparison among DataWorks editions
Feature downgrades of DataWorks advanced editions upon expiration
Performance metrics and billing standards of exclusive resource groups for Data Integration
Performance metrics and billing standards of exclusive resource groups for scheduling
DataWorks (pay-as-you-go) billing details
Preparation
Administrator Operations
Prepare an Alibaba Cloud account
Prepare a RAM user
Create a workspace
Add workspace members
Work as a workspace administrator
Use a RAM user
Quick Start
Overview
Create tables and import data
Create a workflow
Create a batch synchronization node
Configure recurrence and dependencies for a node
Run a node and troubleshoot errors
Optional: Use an ad hoc query node to run SQL statements
Tutorials
Workshop
Overview
Prepare the environment
Collect data
Process data
Monitor data quality
Visualize data on a dashboard
Use Function Studio to develop a UDF
DataWorks for EMR Workshop
Prepare the environment
Collect data
Process data
Collect and view metadata
Monitor data quality
Build and optimize a data warehouse
Overall process
Business survey
Determine requirements
Analyze business processes
Divide data domains
Define dimensions and build a bus matrix
Specify statistical metrics
Architecture and model design
Select a technical architecture
Divide a data warehouse into layers
Design a data model
Specify the principles for using data at different layers
Project allocation and security
Performance benchmark establishment
Data warehouse performance optimization
Result verification
Build an online operation analysis platform
Business scenario and development process
Prepare the environment
Prepare data
Build data models and analyze data
Create tables
Design workflows
Configure nodes
Commit and test nodes
Display data on dashboards
Guarantee data quality
Overview
Data quality management process
Classify data
Verify data during data generation
Use Data Quality to monitor data accuracy, consistency, and integrity
Use Monitor to monitor data timeliness
Automatically identify users who steal electricity
Overview
Prepare the environment
Prepare data
Process data
Build a data model
Workbench
DataWorks console overview
Workspace list
Resource group list
Add and use exclusive resource groups for scheduling
View resource groups
View the usage of exclusive resource groups
O&M Assistant
Add a route
Create custom resource groups for scheduling
Exclusive resource mode
Compute engine list
MaxCompute Management
Workspace management
Configure a workspace
Manage workspace members
Permission list
Configure MaxCompute
Manage connections
Upgrade the workspace mode
Data Integration
Overview
Homepage
Supported data stores and plug-ins
Common configurations
Test data store connectivity
Configure whitelists
Configure a security group
Batch data synchronization
Connections
Supported data stores and plug-ins
Connection isolation
Manage connection permissions
Troubleshooting for connections
Node configuration
Create a sync node by using the codeless UI
Create a sync node by using the code editor
Synchronize incremental data
Synchronize data from tables in multiple databases to the same destination table
Migrate databases to MaxCompute
Rules and restrictions
Migrate a MySQL database
Migrate an Oracle database
Migrate multiple tables to the cloud at a time
Migrate multiple tables to the cloud at a time
Add multiple connections at a time
Add a prefix to the names of the destination tables
Resources for batch sync nodes
Real-time synchronization
Create, commit, and manage real-time sync nodes
Supported data stores
Fields used for real-time synchronization
Reader
MySQL binlog reader
Datahub reader
LogHub reader
Kafka reader
ApsaraDB for PolarDB reader
Writer
MaxCompute writer
Hologres writer
Datahub writer
Kafka writer
Transformation
Data filtering
String replacement
Sync solutions
Node optimization
Optimize synchronization performance
Optimize a sync node
Resource groups
Overview
Use the default resource group
Use exclusive resource groups for data integration
Add a custom resource group
RAM authorization mode
Service linked role of DataWorks Data Integration
Use the RAM authorization mode to configure connections to data stores
Appendixes
Connection configuration
Reader configuration
Writer configuration
Data Development
Create a solution
SQL coding guidelines and specifications
GUI elements
Overview
Workflow Parameters
Lineage
Versions
Code Structure
Workflow
Overview
Node group
Node types
Data Integration
Batch Sync node
MaxCompute
ODPS SQL node
SQL script template
ODPS Spark node
PyODPS node
Create a PyODPS 3 node
ODPS Script node
ODPS MR node
Table
Resource
Function
AnalyticDB for PostgreSQL
AnalyticDB for PostgreSQL node
Table
E-MapReduce
EMR Presto node
EMR HIVE node
EMR MR node
EMR SPARK SQL node
EMR SPARK node
Hologres
Create a Hologres SQL node
Algorithm
Machine Learning experiment node
General
do-while node
Assignment node
for-each node
Zero-load node
Shell node
Branch node
OSS Object Inspection node
MERGE node
Cross-tenant collaboration node
Custom nodes
Create a Data Lake Analytics node
Create a Hologres development node
Create an AnalyticDB for MySQL node
MaxCompute
MaxCompute modules
MaxCompute functions
MaxCompute resources
Schedule
Basic properties
Scheduling parameters
Time properties
Dependencies
Configure the resource group
Immediate instance generation
Cross-cycle dependencies
Context-based parameters
Logic of scheduling dependencies
Script template management
Create a script template
Use a script template
Create an ad hoc query
Workspace tables
Runtime logs
Manage manually triggered workflows
Manage tables
Functions
Recycle bin
Setup
Setup
Configuration center
Workspace settings
Template management
Folder management
Level management
Workspace backup and restoration
Editor shortcuts
External table
Node configuration
Overview
Create a data quality wrapper
Create a custom node type
Create a wrapper
Data Application Development
Overview
Version history
Get started
Features
Navigation page
Project management
Version control
Code editing
Debugging
Collaborative coding
App deployment
Access third-party services
WYSIWYG designer
DataService Studio
Overview
Features
Create an API
Manage API tags
Use workflows
Use filters
Use prefilters
Use post filters
View API versions
Use business processes
Manage business processes
Manage APIs
Manage functions
Manage workflows
Generate API
Configure a connection
Create an API in the codeless UI
Create an API in the code editor
Register an API
View API statistics
Test an API
Publish an API
Authorize access to an API
Call an API
DataAnalysis
Overview
Workbook
Create and manage a workbook
Import data to a workbook
Edit a workbook and analyze data
Export, share, and download a workbook
Manage a template
MaxCompute table
Create and manage a MaxCompute table
Edit a MaxCompute table
Import data to a MaxCompute table
Share a MaxCompute table
Report
Create and manage a report
Edit a report
Save a report as a template
Share a report
Charts
Area charts
Pie charts
Horizontal bar charts
Scatter charts
Column charts
Stock charts
Line charts
Task Operation
Operation Center
Overview
View the dashboard
Auto triggered node O&M
Auto triggered nodes
Retroactive instances
Auto triggered node instances
Instance diagnosis
Test instances
Manually triggered node O&M
Manually triggered nodes
Manually triggered node instances
Monitor
Overview
Features
Instructions
Publish nodes
Deploy a node
Delete a node
Cross-workspace cloning
Overview
Clone nodes across workspaces
Data governance
Data Guard
Data Guard overview
Quick start
My Permissions
Authorizations
Approval Center
Data quality
Overview
Go to the Overview page
View my subscriptions
Configure monitoring rules
View monitoring results
Manage templates
Manage report templates
Manage rule templates
User manual
Configure monitoring rules for Datahub
Configure monitoring rules for MaxCompute
Built-in rule templates for offline data
Data security guard
Access Data Security Guard
Data distribution
Access analysis
Data risks
Audit
Rule setting
Customize desensitization rules
Classification management
Manual ajust
Manage risk rules
Data Map
Overview
View the overall information
Manage data
Manage configurations
Table details
View table details
Apply for table permissions
Add a table to favorites
Go to DataService Studio to create API operations
Data discovery
Collect metadata from an E-MapReduce data store
Collect metadata from a MySQL data store
Collect metadata from an SQL Server data store
Collect metadata from a PostgreSQL data store
Collect metadata from an Oracle data store
Collect metadata from an AnalyticDB for PostgreSQL data store
Collect metadata from an AnalyticDB for MySQL 2.0 data store
Collect metadata from an AnalyticDB for MySQL 3.0 data store
Collect metadata from an OSS data store
Requirements Management
Overview
Create a requirement
Search for a requirement
Manage a requirement
Best practices
Best Practices
Data migration
Migrate data across DataWorks workspaces
Best practices of migrating data from Hadoop to MaxCompute
Migrate data from a Kafka cluster to MaxCompute
Migrate JSON data from OSS to MaxCompute
Migrate JSON data from MongoDB to MaxCompute
Migrate data from an Elasticsearch cluster to MaxCompute
Connect an exclusive resource group for Data Integration to a data store that is deployed in a VPC
Add a relational database driver for RDBMS Reader
Use Data Integration to import data to Elasticsearch
Migrate data from a user-created MySQL database on an ECS instance to MaxCompute
Use Data Integration to import data to DataHub
Use Data Integration to ship data collected by LogHub to destinations
Migrate data from Oracle to MaxCompute
Automatically synchronize IoT data to the cloud
Data development
Best practices for setting scheduling dependencies
Use MaxCompute to analyze IP sources
Reference a third-party package in a PyODPS node
Performing a task at a specific time with branch node
Connect DataV to DataWorks DataService Studio
Use a PyODPS node to segment Chinese text based on Jieba
Build a data warehouse for an enterprise based on AnalyticDB for MySQL
Use a PyODPS node to send emails
Data security
Grant access to a specific UDF to a specified user
Log on to DataWorks through a specific IP address with a RAM user
Data analysis
Use MaxCompute to analyze data and Quick BI to present the analysis results
Analyze offline data in the Internet and e-commerce industries
Intelligently recommend items on e-commerce websites
FAQ
Data Integration
Batch synchronization
Management of permissions on data sources
Custom resource groups
Exclusive resource groups
Scheduling parameters
Scheduling properties
Dependencies
Data Analytics
Monitor
DataService Studio
Security Center
App Studio
Stream Studio
Videos
DataWorks for MaxCompute Workshop
API
Tenant
ListCalcEngines
CreateConnection
DeleteConnection
ListConnections
UpdateConnection
ListResourceGroups
GetProjectDetail
AddProjectMemberToRole
CreateProjectMember
DeleteProjectMember
ListProjectRoles
RemoveProjectMemberFromRole
ListProjectMembers
ListProjects
Metadata
GetMetaDBInfo
GetMetaDBTableList
CheckMetaTable
CheckMetaPartition
SearchMetaTables
GetMetaTableBasicInfo
GetMetaTableColumn
GetMetaTableFullInfo
GetMetaTablePartition
GetMetaTableOutput
GetMetaTableChangeLog
GetMetaTableIntroWiki
GetMetaTableLineage
GetMetaColumnLineage
CreateMetaCategory
DeleteMetaCategory
UpdateMetaCategory
GetMetaCategory
GetMetaTableListByCategory
AddToMetaCategory
DeleteFromMetaCategory
CreateTable
UpdateTable
DeleteTable
GetDDLJobStatus
UpdateMetaTable
UpdateMetaTableIntroWiki
ListMetaDB
UpdateTableModelInfo
CreateTableTheme
DeleteTableTheme
UpdateTableTheme
ListTableTheme
CreateTableLevel
DeleteTableLevel
UpdateTableLevel
ListTableLevel
Data Development
CreateFile
CreateUdfFile
DeleteBusiness
CreateFolder
UpdateBusiness
ListFiles
EstablishRelationTableToBusiness
SubmitFile
ListBusiness
GetFileVersion
DeleteFolder
DeleteFile
GetDeployment
UpdateFile
GetFolder
ListFileVersions
GetFile
ListFolders
DeployFile
UpdateUdfFile
GetBusiness
UpdateFolder
Operation Center
ListAlertMessages
GetBaselineConfig
GetNodeOnBaseline
ListBaselineConfigs
GetBaselineKeyPath
GetBaselineStatus
ListBaselineStatuses
DeleteRemind
UpdateRemind
CreateRemind
GetRemind
ListReminds
ListTopics
GetTopic
GetTopicInfluence
GetNode
ListNodeIO
ListNodes
GetNodeCode
GetInstance
ListInstances
GetInstanceLog
StopInstance
RestartInstance
SetSuccessInstance
SuspendInstance
ResumeInstance
CreateDagComplement
CreateDagTest
GetSuccessInstanceTrend
GetInstanceCountTrend
GetInstanceConsumeTimeRank
GetInstanceErrorRank
ListProgramTypeCount
GetInstanceStatusCount
GetNodeTypeListInfo
Data Quality
UpdateQualityFollower
UpdateQualityRule
GetQualityRule
ListQualityRules
CreateQualityRule
DeleteQualityFollower
DeleteQualityRule
GetQualityFollower
CreateQualityFollower
DeleteQualityEntity
GetQualityEntity
CreateQualityEntity
CreateQualityRelativeNode
DeleteQualityRelativeNode
ListQualityResultsByEntity
ListQualityResultsByRule
DataService Studio
CreateDataServiceApi
PublishDataServiceApi
DeleteDataServiceApi
ListDataServiceApis
GetDataServiceApi
UpdateDataServiceApi
GetDataServicePublishedApi
ListDataServicePublishedApis
ListDataServiceApiAuthorities
ListDataServiceAuthorizedApis
AbolishDataServiceApi
GetDataServiceApplication
ListDataServiceApplications
CreateDataServiceFolder
GetDataServiceFolder
ListDataServiceFolders
CreateDataServiceGroup
GetDataServiceGroup
ListDataServiceGroups
CreateDataServiceApiAuthority
DeleteDataServiceApiAuthority
SDK
Install the Alibaba Cloud SDK for Java
All Products
Search
Document Center
DataWorks
Data Integration
Appendixes
Connection configuration
This Product
This Product
All Products
DataWorks
Configure an AnalyticDB for MySQL 2.0 connection
Configure an SQL Server connection
Configure a MongoDB connection
Configure a Datahub connection
Configure a DM connection
Configure a DRDS connection
Configure an FTP connection
Configure an HDFS connection
Configure a LogHub connection
Configure a Memcache connection
Configure a MySQL connection
Configure an Oracle connection
Configure an OSS connection
Configure a Table Store connection
Configure a PostgreSQL connection
Configure a Redis connection
Configure a HybridDB for MySQL connection
Configure an AnalyticDB for PostgreSQL Connection
Configure a PolarDB connection
Configure an AnalyticDB for MySQL 3.0 connection
Configure a DLA connection
Configure a MaxCompute connection
Configure a GDB connection
Configure a GBase 8a connection
Configure a Hologres connection
Configure an HBase connection
Configure an Elasticsearch connection
Configure an ApsaraDB for OceanBase connection
Configure a Vertica connection
Configure a Hive connection
Free Trial
Free Trial