After you connect a log source to Log Service, Log Service starts to collect logs in real time. You can consume and ship logs by using the Log Service console, SDKs, or API operations. You can use the LogShipper feature to create log shipping tasks in Log Service. Then, you can ship logs from the LogHub module of Log Service to Alibaba Cloud storage services in real time, such as Object Storage Service (OSS), Tablestore, and MaxCompute. The LogShipper feature allows you to call multiple API operations to check the status of log shipping tasks. If a task fails, you can retry the task.

Scenarios

The LogShipper feature applies to connections with data warehouses.

Log source

The LogShipper feature allows you to ship logs that are collected by the LogHub module of Log Service. After these logs are generated, Log Service collects and sends the logs to LogHub in real time and ships them to other cloud services for storage and analysis.

Log shipping destinations

  • OSS (object storage service)
    • Ship logs to OSS.
    • We recommend that you use E-MapReduce to convert data formats in OSS.
    • After you ship logs to OSS, you can use Data Lake Analytics to analyze the logs.
  • Tablestore (NoSQL data storage service):
  • MaxCompute (big data computing service)

    Ship logs to MaxCompute by using the Data Integration service of DataWorks. For more information, see Ship data to MaxCompute through DataWorks.

Billing

  • On August 25, 2020, Alibaba Cloud will start billing the data that is shipped to Object Storage Service (OSS),MaxCompute ,AnalyticDB for MySQL, and Time Series Database (TSDB) for storage or analysis.
  • You are billed based on the uncompressed size of the data that is shipped from Logstores. Each gigabyte of uncompressed data costs USD 0.05.