Step-by-step tutorials for specific data source and target combinations in DataWorks Data Integration, covering batch sync, real-time CDC, JSON migration, and IoT data ingestion.
Each tutorial covers a specific source-to-destination combination and walks you through configuration, execution, and verification. For higher-level planning and architecture guidance, see [Common business scenarios]().
Choose a sync method
Before picking a tutorial, identify your sync requirements:
|
Requirement |
Recommended method |
Typical use case |
|
One-time or scheduled full copy |
Batch sync node |
Historical data migration, periodic snapshots |
|
Low-latency change tracking from relational databases |
Real-time CDC sync |
MySQL or Oracle to MaxCompute with near-real-time updates |
|
Serverless, no resource group to manage |
Serverless sync task |
Ad hoc or lightweight pipelines without dedicated resources |
|
Semi-structured source (JSON files in OSS) |
Batch sync with JSON reader |
Log files, event data stored as JSON in OSS |
|
Device telemetry at scale |
IoT data ingestion pipeline |
Time-series sensor data from IoT Platform |
If you are unsure which method fits your workload, consider these factors:
Latency: CDC sync delivers changes within seconds. Batch sync runs on a schedule (minutes to hours).
Source type: CDC requires a relational database with binary logging or change tracking enabled. Batch sync supports a broader range of sources.
Infrastructure: Batch sync and real-time sync require a dedicated resource group. Serverless sync tasks do not.
Available tutorials
Sync from relational databases
MySQL to MaxCompute (real-time CDC) — Sync incremental changes from MySQL using change data capture. Covers enabling binlog, configuring the CDC reader, and verifying row-level changes in MaxCompute.
MySQL to MaxCompute (batch) — Schedule a full or incremental batch sync from MySQL. Covers watermark-based incremental extraction and partition writing in MaxCompute.
Oracle to MaxCompute — Migrate Oracle tables to MaxCompute with type mapping and partition configuration.
Sync from document and object stores
OSS JSON files to MaxCompute — Parse and load semi-structured JSON files from OSS into MaxCompute tables. Covers schema mapping, nested field flattening, and handling malformed records.
MongoDB to MaxCompute (batch read) — Extract documents from MongoDB collections and write to MaxCompute. Covers collection filtering, field projection, and write mode configuration.
IoT and streaming ingestion
IoT Platform to MaxCompute — Ingest time-series device telemetry from Alibaba Cloud IoT Platform into MaxCompute. Covers topic subscription, timestamp parsing, and partition-by-time configuration.
Related topics
[Common business scenarios]() — Architecture patterns and scenario overviews for planning your integration strategy.
[Data Integration overview]() — Supported connectors, resource group requirements, and sync node types.
[Configure a batch sync node]() — Reference for all reader and writer plugin parameters.
[Configure a real-time sync node]() — Reference for CDC source configuration and exactly-once delivery settings.