Data Integration is a streamlined and efficient data synchronization platform on Dataphin, offering robust data preprocessing and high-speed, stable synchronization across various data sources.
Background information
In response to big data applications across industries, Data Integration addresses needs such as simple and efficient configuration of numerous data table synchronization tasks, integration of multiple data sources, light preprocessing of data, and optimization of synchronization tasks for enhanced fault tolerance, throttling, and concurrency.
Feature overview
If you purchased Dataphin after April 2020, the data synchronization capability has been upgraded to Data Integration.
Dataphin has enhanced the Data Integration capabilities to provide a simple, efficient, secure, and reliable platform for your data synchronization needs:
Data Integration efficiency is enhanced with features like whole database migration for quickly generating batch sync tasks, and one-click target table creation, eliminating the need for manual table setup in MaxCompute. For more information, see configure integration tasks through whole database migration.
Supports Flow and Transform components, enabling data preprocessing for data sources, including traffic scrubbing, transformation, field desensitization, calculation, merging, distribution, filtering, and more. For additional details, see create integration tasks through a single pipeline.
Supports Dev-Prod and Basic development patterns, allowing flexible selection based on your business scenarios.
Facilitates quick synchronization of logical tables created in Dataphin to the destination database.
Allows for user-defined components to meet diverse data synchronization requirements. RDBMS database components connect via JDBC, while non-RDBMS database components require JAR package uploads.
Data Integration supports a variety of components, enabling the creation of offline single pipelines through intuitive drag-and-drop, configuration, and assembly. It facilitates quick batch sync task generation, with whole database migration sources including MySQL, SQL Server, Oracle, and MaxCompute as the target. Additionally, Data Integration accommodates user-defined component types to fulfill unique data synchronization needs.
Data integration entry
Quick entry (recommended)
On the Dataphin home page, click Data Import in the Dataphin product usage path for rapid access to Data Integration.
Regular entry
On the Dataphin home page, in the top menu bar, select Development > Data Integration to navigate to the Data Integration page.
