Data Integration is a stable, efficient, and scalable data synchronization service. It provides transmission channels for offline data that is stored in Alibaba Cloud services such as MaxCompute, AnalyticDB for PostgreSQL, and Hologres.

Important
  • If the network connectivity test is successful, an exclusive resource group for Data Integration can be used to synchronize data across accounts or regions, or synchronize data from or to a data center. For example, data in a MySQL database that belongs to Account A can be synchronized to a MongoDB database that belongs to Account B.
  • You can synchronize data from or to data sources only after the data sources pass the network connectivity test. For more information, see Establish a network connection between a resource group and a data source.
  • Data Integration allows you to configure a batch synchronization node by using the codeless user interface (UI) or code editor. In the following scenarios, you can configure a batch synchronization node only by using the code editor:
    • The data source that you want to use cannot be added to DataWorks.
    • The data source that you want to use does not support the codeless UI.
    • The parameters of the related Reader or Writer plug-in can be configured only by using the code editor.

Data source types that support batch synchronization of data from a single table

DataWorks allows you to create a batch synchronization node to perform single-table batch synchronization on the DataStudio page.

Data source typeReader plug-inWriter plug-in
Amazon S3Amazon S3 ReaderNot supported
AnalyticDB for MySQL 2.0AnalyticDB for MySQL 2.0 ReaderAnalyticDB for MySQL 2.0 Writer
AnalyticDB for MySQL 3.0AnalyticDB for MySQL 3.0 ReaderAnalyticDB for MySQL 3.0 Writer
AnalyticDB for PostgreSQLAnalyticDB for PostgreSQL ReaderAnalyticDB for PostgreSQL Writer

ApsaraDB For Oceanbase

Note This type of data source supports only exclusive resource groups for Data Integration.
ApsaraDB For Oceanbase ReaderApsaraDB For Oceanbase Writer
ClickHouse
Note This type of data source supports only exclusive resource groups for Data Integration.
ClickHouse ReaderClickHouse Writer
DataHubDataHub ReaderDataHub Writer
DB2
Note This type of data source supports only exclusive resource groups for Data Integration.
DB2 ReaderDB2 Writer
Dameng (DM)
Note This type of data source supports only exclusive resource groups for Data Integration.
DM ReaderDM Writer
DRDSDRDS ReaderDRDS Writer

Elasticsearch

Note This type of data source of V6.X or later supports only exclusive resource groups for Data Integration.
Elasticsearch ReaderElasticsearch Writer
FTPFTP ReaderFTP Writer

GBase8a

Note This type of data source supports only exclusive resource groups for Data Integration.

Supported

Supported

HBase

Note This type of data source supports only exclusive resource groups for Data Integration.
HDFS
Note This type of data source supports only exclusive resource groups for Data Integration.
HDFS ReaderHDFS Writer

Hive

Note This type of data source supports only exclusive resource groups for Data Integration.
Hive ReaderHive Writer

Hologres

Note This type of data source supports only exclusive resource groups for Data Integration.

Supported

Supported

HybridDB for MySQLHybridDB for MySQL ReaderHybridDB for MySQL Writer
Kafka
Note This type of data source supports only exclusive resource groups for Data Integration.
Kafka ReaderKafka Writer
KingbaseES
Note This type of data source supports only exclusive resource groups for Data Integration.
KingbaseES ReaderKingbaseES Writer
Lindorm
Note This type of data source supports only exclusive resource groups for Data Integration.
Lindorm ReaderLindorm Writer
LogHub (SLS)LogHub (SLS) ReaderLogHub (SLS) Writer
MaxComputeMaxCompute ReaderMaxCompute Writer
MaxGraphNot supportedMaxgraph Writer
MemcacheNot supportedMemcache Writer
MetaQ
Note This type of data source supports only exclusive resource groups for Data Integration.
MetaQ ReaderNot supported
MongoDBMongoDB ReaderMongoDB Writer
MySQLMySQL ReaderMySQL Writer
OpenSearch
Note This type of data source supports only exclusive resource groups for Data Integration.
Not supportedOpenSearch Writer
OracleOracle ReaderOracle Writer
OSSOSS ReaderOSS Writer
OTSStreamOTSStream ReaderNot supported
PolarDBPolarDB ReaderPolarDB Writer
PostgreSQLPostgreSQL ReaderPostgreSQL Writer
RedisNot supportedRedis Writer
HTTP RESTful API
Note This type of data source supports only exclusive resource groups for Data Integration.
RestAPI ReaderRestAPI Writer
SAP HANA
Note This type of data source supports only exclusive resource groups for Data Integration.
SAP HANA ReaderSAP HANA Writer
StarRocks
Note This type of data source supports only exclusive resource groups for Data Integration.
StarRocks ReaderStarRocks Writer
StreamStream ReaderStream Writer
SQL ServerSQL Server ReaderSQL Server Writer
TablestoreTablestore ReaderTablestore Writer
TSDB
Note This type of data source supports only exclusive resource groups for Data Integration.
Not supportedTSDB Writer

Vertica

Note This type of data source supports only exclusive resource groups for Data Integration.
Vertica ReaderVertica Writer

Data source types that support batch synchronization of all data from a database

DataWorks allows you to create a data synchronization solution to perform batch synchronization of all data from a database in Data Integration. You can use the solution to quickly synchronize data from all tables in a source database to a destination. The solution frees you from creating multiple data synchronization nodes to synchronize data and helps you save time.

DataWorks allows you to use a data synchronization solution to synchronize all data from a database to a MaxCompute, Object Storage Service (OSS), or Elasticsearch data source. The following table lists the data source types that support batch synchronization of all data from a database and the related references.

DestinationSourceReferences
MaxComputeCreate a batch synchronization solution to synchronize all data in a database to MaxCompute
OSSCreate a batch synchronization solution to synchronize all data in a database to OSS
ElasticsearchCreate a batch synchronization solution to synchronize all data in a database to Elasticsearch