AnalyticDB for MySQL can ingest data from databases, object storage, big data platforms, message queues, and local files into either a data warehouse or a data lake. The right import method depends on your data source and performance requirements.
How it works
AnalyticDB for MySQL supports two ingestion paths:
Data warehouse ingestion: Data is pre-processed before loading into AnalyticDB for MySQL's proprietary Xuanwu analytic storage engine. This path delivers high-throughput real-time writes and high-performance real-time queries, making it the right choice when query performance is your top priority.
Data lakehouse ingestion: Raw data is stored in open-source table formats (Iceberg and Paimon), either in ADB's built-in lake storage or in your own Object Storage Service (OSS) bucket. Because the data stays in open formats, it can be queried by both the Spark and XIHE engines of AnalyticDB for MySQL and by external engines such as MaxCompute. Use this path when your architecture requires open-source compatibility or multi-engine access. For higher read performance, enable LakeCache to reduce latency compared to reading directly from OSS.
Data lakehouse ingestion is available only for Enterprise Edition, Basic Edition, or Data Lakehouse Edition clusters.
Data warehouse ingestion
| Category | Data source | Import method | Edition | Documentation |
|---|---|---|---|---|
| Database | RDS MySQL | External table | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from RDS MySQL using an external table |
| Database | RDS MySQL | DTS | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DTS |
| Database | RDS MySQL | DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks |
| Database | RDS MySQL | Zero-ETL | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Synchronize data using zero-ETL |
| Database | RDS SQL Server | DTS | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DTS |
| Database | RDS SQL Server | DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks |
| Database | PolarDB Distributed Edition (formerly DRDS) | DTS | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DTS |
| Database | PolarDB Distributed Edition (formerly DRDS) | DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks |
| Database | PolarDB Distributed Edition (formerly DRDS) | One-stop synchronization | Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Automatically synchronize PolarDB-X metadata |
| Database | PolarDB for MySQL | Federated analytics | Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Synchronize data using the federated analytics feature |
| Database | PolarDB for MySQL | DTS | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DTS |
| Database | PolarDB for MySQL | Zero-ETL | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Synchronize data using zero-ETL |
| Database | MongoDB | External table | Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from MongoDB using an external table |
| Database | MongoDB | Zero-ETL | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Synchronize data using zero-ETL |
| Database | Lindorm | Zero-ETL | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from Lindorm |
| Database | Oracle | DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from Oracle |
| Database | Self-managed MySQL | External table | Data Warehouse Edition | Import data from a self-managed MySQL database |
| Database | Self-managed HBase | DTS | Data Warehouse Edition | Import data from a self-managed HBase cluster |
| Storage | OSS | External table | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from OSS using an external table |
| Storage | OSS | DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks |
| Storage | Tablestore | External table | Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Query and import data from Tablestore |
| Storage | HDFS | External table | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from HDFS using an external table |
| Storage | HDFS | DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks |
| Big data | MaxCompute | External table | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data from MaxCompute using an external table |
| Big data | MaxCompute | DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks |
| Big data | Flink | Flink | Data Warehouse Edition | Import data from Flink |
| Message queue | Kafka | DataWorks | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import data using DataWorks |
| Message queue | Kafka | Logstash plugin | Data Warehouse Edition | Import data using Logstash |
| Log data | Log data | Data synchronization | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Synchronize log data using data synchronization |
| Log data | Log data | Logstash plugin | Data Warehouse Edition | Import data using Logstash |
| Local data | Local files | SQLAlchemy | Data Warehouse Edition, Enterprise Edition, Basic Edition, or Data Lakehouse Edition | Import DataFrame data using SQLAlchemy |
| Local data | Local files | LOAD DATA | Data Warehouse Edition | Import data using LOAD DATA |
| Local data | Local files | Import tool | Data Warehouse Edition | Import data using the import tool |
| Local data | Local files | Kettle | Data Warehouse Edition | Import data using Kettle |
Data lakehouse ingestion
Available only for Enterprise Edition, Basic Edition, or Data Lakehouse Edition clusters.
| Category | Data source | Import method | Documentation |
|---|---|---|---|
| Message queue | Kafka | Data synchronization (Recommended) | Synchronize Kafka data using data synchronization |
| Log data | Simple Log Service (SLS) | Data synchronization (Recommended) | Synchronize SLS data using data synchronization |
| Big data | Hive | Data migration | Import data from Hive |
| Storage | OSS | Metadata discovery | Import data using metadata discovery |
References
AnalyticDB for MySQL also supports asynchronous submission of import tasks. For more information, see Submit an asynchronous import task.