All Products
Search
Document Center

Simple Log Service:Download logs

Last Updated:Jul 10, 2024

Simple Log Service allows you to download logs or query and analysis results to your on-premises machine. This topic describes how to download logs or query and analysis results.

Download methods

You can download logs or query and analysis results by using the Simple Log Service console, Cloud Shell, Simple Log Service CLI, or Simple Log Service SDK.

Important
  • If you download data by using Simple Log Service CLI or Simple Log Service SDK, the amount of data that you can download is unlimited. However, interruptions may occur due to network errors.

  • If you want to download a compressed package of logs, you can use only the Simple Log Service console.

  • If your Logstore uses the pay-by-ingested-data billing mode, you are not charged when you download the query and analysis results of a query statement. For more information, see Pay-by-ingested-data.

  • If your Logstore uses the pay-by-feature billing mode, you are charged when you download the query and analysis results of a query statement because the Dedicated SQL feature is used during the download. You are charged for the Dedicated SQL feature. For more information, see Billable items of pay-by-feature.

Item

Download data by using the Simple Log Service console (recommended)

Download data by using Simple Log Service CLI

Download data by using Cloud Shell

Download data by using Simple Log Service SDK

Limit on data amounts

  • Query only: You can download up to 20 million lines of log data from a Logstore that resides in the China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), and Singapore regions. You can download up to 1 million lines of log data from a Logstore that resides in other regions. The total amount of the log data cannot exceed 20 GB.

  • SQL analysis: You can download up to 100,000 lines of log data. The total amount of the log data cannot exceed 2 GB.

  • Query only: unlimited.

  • SQL analysis: You can download up to 1 million lines of log data. The total amount of the log data cannot exceed 2

  • GB.

  • Query only: unlimited.

  • SQL analysis: You can download up to 1 million lines of log data. The total amount of the log data cannot exceed 2 GB.

  • Query only: unlimited.

  • SQL analysis: You can download up to 1 million lines of log data. The total amount of the log data cannot exceed 2 GB.

Required deployment

None.

Simple Log Service CLI is manually installed.

Cloud Shell is automatically deployed. The first time you run Cloud Shell, you need to wait until Cloud Shell is initialized.

Simple Log Service SDK is manually installed, and you need to write code based on your business requirements.

Authorization

The permissions to query data in the Simple Log Service console are used. No additional permissions are required.

Manual configuration.

Automatic configuration.

Manual configuration.

Dedicated SQL

When you download the query and analysis results of a query statement, the Dedicated SQL feature is used.

Not used.

Not used.

You can configure Dedicated SQL-related parameters based on your business requirements.

Read traffic over the Internet

None.

If Simple Log Service CLI is deployed on an Elastic Compute Service (ECS) instance that resides in the region of the required Simple Log Service project and an internal Simple Log Service endpoint is used, no fees are generated for read traffic over the Internet.

If your Simple Log Service project resides in the China (Shanghai) region, no fees are generated for read traffic over the Internet.

If Simple Log Service SDK is deployed on an ECS instance that resides in the region of the required Simple Log Service project and an internal Simple Log Service endpoint is used, no fees are generated for read traffic over the Internet.

Integration with Apsara File Storage NAS

None.

Manual configuration is required if necessary.

Automatic configuration.

Manual configuration is required if necessary.

You can ship logs to Object Storage Service (OSS) and download the logs in OSS. For more information, see Create an OSS data shipping job (new version).

Procedure

Download data by using the Simple Log Service console

You can download logs or query and analysis results to your on-premises machine by using the Simple Log Service console. The operations that are used to download logs are similar to the operations that are used to download query and analysis results. In this topic, logs are used to show the download procedure. If you want to download query and analysis results, you can click Download Log on the Graph tab after you execute a query statement.

Important
  • The number of log lines and the amount of log data that you can download at a time cannot exceed the upper limits. If the number of log lines and the amount of log data that you want to download exceed the upper limits, the excess logs cannot be downloaded. To download the full logs in this scenario, you can shorten your query time range and repeat the download operations until the full logs are downloaded.

    You can view the download tasks in the Log Export History dialog box.

  • Simple Log Service allows for up to three concurrent download operations within each Alibaba Cloud account. Simple Log Service does not limit the total number of download operations. If more than three download operations are concurrently performed or multiple Resource Access Management (RAM) users are used to download logs at the same time, an error may be reported. In this case, you must wait until other download operations are complete and try again.

  • Simple Log Service stores only the export records that are generated within the previous 24 hours and deletes the export records 24 hours after the records are generated.

  • If a download task fails because network errors occur or query results are inaccurate, Simple Log Service automatically retries the task. If the download task fails after three retries, the download task enters the Failed state.

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

  3. On the Log Storage > Logstores tab, click the Logstore from which you want to download logs.

  4. Enter a query statement and specify a query time range.

    For more information, see Query and analyze logs.

  5. On the Raw Logs tab, choose 下载日志 > Download Log.

    Note

    If no logs are available, Download Log is unavailable.

  6. In the Log Download dialog box, configure the following parameters and click OK.

    Parameter

    Description

    Task Name

    The name of the download task.

    Log Quantity

    The number of logs that you want to download.

    Data Format

    The data format of the file to which you want to download logs. The CSV and JSON formats are supported.

    • If you select CSV, the column names in the file are generated based on the fields of the first 100 logs. If the subsequent logs contain new fields, the new fields are stored in the last column of the file in the JSON format. The name of the last column is left empty.

    • If you select JSON, each log is converted to the JSON format and written to the file as a single line.

    Quote

    The quote that is used to enclose special characters in logs. The enclosed special characters are not escaped.

    Download Inaccurate Results

    Specifies whether to download inaccurate results. If you select No, the download task fails when the query results are inaccurate.

    Compression Method

    The method that is used to compress logs. Valid values: gzip, lz4, zstd, and Uncompressed.

    If you want to download a large number of logs, we recommend that you select a compression method. This way, the log amount can be significantly reduced, and the logs can be quickly downloaded.

    Sorting Rule

    The rule that is used to sort logs.

  7. Wait until the status of the download task in the Log Export History dialog box changes to Successful. Then, click Download in the Actions column to download the logs to your on-premises machine.

    You can also choose 下载日志 > Log Export History on the Raw Logs tab to open the Log Export History dialog box and view the download task.

Download data by using Cloud Shell

The Cloud Shell server resides in the China (Shanghai) region. If you download logs from a Logstore that does not reside in the China (Shanghai) region, you are charged for read traffic over the Internet when you download logs. For more information, see Simple Log Service Pricing and Use Cloud Shell to download logs from Simple Log Service.

Download data by using Simple Log Service CLI

If you want to download a large number of logs, you can use Simple Log Service CLI.

  1. Install Simple Log Service CLI. For more information, see Install Simple Log Service CLI.

  2. Obtain the AccessKey pair of the current account. For more information, see AccessKey pair.

  3. Obtain the command that is used to download logs.

    1. Log on to the Simple Log Service console.

    2. In the Projects section, click the project that you want to manage.

    3. On the Log Storage > Logstores tab, click the Logstore from which you want to download logs.

    4. Enter a query statement and specify a query time range.

      For more information, see Query and analyze logs.

    5. On the Raw Logs tab, choose 下载日志 > Download Log.

      Note

      If no logs are available, Download Log is unavailable.

    6. In the Download Log dialog box, click Download with CLI and click OK.

    7. Click Copy Command.

      Simple Log Service generates a command that is used to download logs based on your asset information.

      Important

      If the Simple Log Service CLI server resides in the same region as the current Simple Log Service project, we recommend that you click Switch to Internal Endpoint. This way, logs can be downloaded faster, and you are not charged for Internet traffic because data is not transferred over the Internet.

      通过命令行方式下载

  4. Replace the value of access-id and access-key in the download command with the AccessKey pair that you obtained.

  5. Run the download command in the CLI.

For more information, see Overview of Simple Log Service CLI.

Download data by using Simple Log Service SDK

If you want to download a large number of logs, you can use Simple Log Service SDK.

Note

You can call the same API operation that is encapsulated in the SDK to query and download logs.

The following sample code provides an example on how to download logs by using Simple Log Service SDK for Python:

import os
import time

from aliyun.log import LogClient
from aliyun.log import GetLogsRequest

# The Simple Log Service endpoint. 
endpoint = 'cn-qingdao.log.aliyuncs.com'
# In this example, the AccessKey ID and AccessKey secret are obtained from environment variables. 
accessKeyId = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_ID', '')
accessKey = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_SECRET', '')
# The name of the project. 
project = 'Project name'
# The name of the Logstore. 
logstore = 'Logstore name'
client = LogClient(endpoint, accessKeyId, accessKey)
request = GetLogsRequest("project1", "logstore1", fromTime=int(time()-3600), toTime=int(time()), topic='', query="*", line=100, offset=0, reverse=False)
# or
# request = GetLogsRequest("project1", "logstore1", fromTime="2018-1-1 10:10:10", toTime="2018-1-1 10:20:10", topic='', query="*", line=100, offset=0, reverse=False)


res = client.get_logs(request)
res.log_print()

For more information, see Overview of Simple Log Service SDK.