Add and manage data sources - DataWorks - Alibaba Cloud Documentation Center

DataWorks allows you to add and manage data sources on the Data Source page of the DataWorks console. After you add a data source to DataWorks, you can use the data source in subsequent data development operations. For example, when you configure a data synchronization node, you can select a data source that is added to DataWorks to read data from or write data to the data source. This topic describes how to add and manage data sources.

Background information

Workspaces in standard mode allow you to isolate data sources. You can separately add data sources for the development environment and production environment to isolate data sources. This helps keep your data secure. For more information about data source isolation, see Isolate a data source in the development and production environments.
Data sources are classified into data sources that are automatically generated when you associate compute engines with a workspace and the external data sources that are added to DataWorks on the Data Source page. For example, a default MaxCompute data source named odps_first is automatically generated on the Data Source page when you associate a MaxCompute compute engine with a workspace.

Add data sources

Log on to the DataWorks console and go to the Workspace Management page.
In the left-side navigation pane, click Data Source.
Add data sources on the Data Source page.
In the upper-right corner of the Data Source page, click Add data source or Batch add data source to add data sources.
- Batch add data source: You can add multiple MySQL, PolarDB, SQL Server, OSS, or Oracle data sources at a time.
  DataWorks provides templates that you can use to add multiple data sources at a time. You can download a template, configure the fields in the template, and then upload the template. The progress and results are displayed in the Start new field of the Batch add data source dialog box.
  Fields in the template: DataSourceType, DataSourceName, description, Environment classification (0dev, 1prod), JDBC URL, username, and password.
- Add data source: In the dialog box that appears, you can select a data source type and add a data source of the selected type. For information about the data source types that are supported by DataWorks and the references for adding the data sources, see the Supported data source types section in this topic.
  - Data sources in the development environment: You can select such a data source when you configure a data synchronization node. Then, you can run the node in the development environment. You cannot commit the node to the production environment for running.
  - Data sources in the production environment: You can use such a data source only in the production environment. You cannot select such a data source when you configure a data synchronization node.
  Note The same data source must have the same name in the development environment and production environment.

Manage data sources

On the Data Source page, you can search for the desired data source by specifying filter conditions Data Source Type and Data Source Name. In addition, you can modify, delete, clone, and share the desired data source.

Modify a data source: You can click Edit in the Operation column of a data source and modify the configurations of the data source in the dialog box that appears.
Delete a data source:
- Before you delete a data source from the development environment and production environment, check whether the data source is used by a data synchronization node in the production environment. The delete operation cannot be rolled back. After the data source is deleted, you cannot select it when you configure a data synchronization node in the development environment.
  If a data synchronization node in the production environment uses the data source, the data synchronization node cannot be run after the data source is deleted. Before you delete the data source, delete the data synchronization node that uses the data source.
- Before you delete a data source from the development environment, check whether the data source is used by a data synchronization node in the production environment. The delete operation cannot be rolled back. After the data source is deleted, you cannot select it when you configure a data synchronization node in the development environment.
  If a data synchronization node in the production environment uses the data source, after the data source is deleted, you cannot obtain metadata when you edit the data synchronization node. However, the data synchronization node can be run in the production environment.
- Before you delete a data source from the production environment, check whether the data source is used by a data synchronization node in the production environment. If you select the data source when you configure a data synchronization node in the development environment, you cannot commit the data synchronization node to the production environment after the data source is removed.
  If a data synchronization node in the production environment uses the data source, the data synchronization node cannot be run after the data source is deleted.
Clone a data source: You can click Cloning in the Operation column of a data source. In the Clone Data Source dialog box, enter a name in the New Data Source Name field and click Cloning. Then, the system automatically generates a data source whose type and connection information are the same as the original data source.
Share a data source: DataWorks allows you to share data sources in a workspace with another workspace or a specific member in another workspace by managing permissions on the data sources. After data sources are shared, you can view the shared data sources in the destination workspace. For more information, see Manage permissions on data sources.

Supported data source types

The following table lists the data source types that are supported by DataWorks and the references for adding these types of data sources.


Data source type	References
Amazon S3	Add an Amazon S3 data source
AnalyticDB for MySQL 2.0	Add an AnalyticDB for MySQL 2.0 data source
AnalyticDB for MySQL 3.0	Add an AnalyticDB for MySQL 3.0 data source
AnalyticDB for PostgreSQL	Add an AnalyticDB for PostgreSQL data source
ApsaraDB for OceanBase	Add an ApsaraDB for OceanBase data source
ClickHouse	Add a ClickHouse data source
DataHub	Add a DataHub data source
Data Lake Analytics (DLA)	Add a DLA data source
DB2	Add a Db2 data source
Dameng (DM)	Add a DM data source
PolarDB-X 1.0	Add a PolarDB-X 1.0 data source
Elasticsearch	Add an Elasticsearch data source
FTP	Add an FTP data source
GBase8a	Add a GBase8a data source
HBase	Add an HBase data source
HDFS	Add an HDFS data source
Hive	Add a Hive data source
Hologres	Add a Hologres data source
HybridDB for MySQL	Add a HybridDB for MySQL data source
Kafka	Add a Kafka data source
KingbaseES	Add a KingbaseES data source
LogHub (SLS)	Add a LogHub (SLS) data source
MaxCompute	Add a MaxCompute data source
Memcache	Add a Memcache data source
MongoDB	Add a MongoDB data source
MySQL	Add a MySQL data source
Oracle	Add an Oracle data source
OSS	Add an OSS data source
PolarDB	Add a PolarDB data source
PostgreSQL	Add a PostgreSQL data source
Redis	Add a Redis data source
RestAPI (HTTP RESTful API)	Add a RestAPI data source
SAP HANA	Add a SAP HANA data source
StarRocks	Add a StarRocks data source
SQL Server	Add an SQL Server data source
Tablestore	Add a Tablestore data source
Vertica	Add a Vertica data source

What to do next: Perform data development based on a data source

After a data source is added to DataWorks, you can perform data development based on the data source in different modules in DataWorks. For example, you can synchronize data from the data source in Data Integration and create an API based on the data source in DataService Studio. For more information, see Configure a batch synchronization node by using the codeless UI and Create an API by using the codeless UI.