Enterprises develop their businesses based on real-time log data. The data includes that for Elastic Compute Service (ECS) instances, containers, mobile terminals, open-source software, website services, and JavaScript. This topic describes how to migrate log data to MaxCompute by using Tunnel, DataHub, Log Service, and DataWorks Data Integration.
Migration method | Description | Scenario |
---|---|---|
Tunnel | Use Tunnel in MaxCompute to upload log data to MaxCompute.
For more information, see Use Tunnel to upload log data to MaxCompute. |
This method is used to upload large volumes of offline data to MaxCompute tables. Tunnel is suitable for offline computing. |
DataHub | Use DataHub to migrate data to MaxCompute. DataHub DataConnector synchronizes streaming
data from DataHub to MaxCompute. You only need to write data to DataHub and configure
the data synchronization feature in DataHub. Then, you can use the data in MaxCompute.
For more information, see Use DataHub to migrate log data to MaxCompute. |
This method is mainly used for public preview and development. DataHub is used to
upload data in real time. It is suitable for stream processing.
After data is uploaded to DataHub, the data is stored in a table for real-time processing. DataHub executes scheduled tasks to synchronize the data to a MaxCompute table within a few minutes for offline computing. |
DataWorks Data Integration | Configure batch synchronization nodes and synchronization tasks in DataWorks Data
Integration to synchronize data to MaxCompute.
For more information, see Use DataWorks Data Integration to migrate log data to MaxCompute. |
This method is used after you configure batch synchronization tasks in DataWorks Data Integration. The tasks are executed on a regular basis. |