Download SLS logs via console, OpenAPI, CLI, Cloud Shell, or SDK - Simple Log Service

Simple Log Service (SLS) supports downloading logs or query and analysis results to your local machine. This topic describes the download methods and the procedure.

Download methods

Simple Log Service lets you download logs or query and analysis results through the console, Cloud Shell, the Simple Log Service command-line interface (CLI), or an SDK.

Important

The CLI and SDK methods have no data volume limit for query-based downloads. However, downloads may be interrupted by network instability.
Compressed downloads are supported only through the console.
The maximum length of an SQL query statement is 32 KB. If the limit is exceeded, split the query into multiple batches.
If the LogStore uses the pay-by-ingested-data billing method, downloading query and analysis results (SQL results) is free. For more information, see pay-by-ingested-data.
If the LogStore uses the pay-by-feature billing method, downloading query and analysis results (SQL results) requires Dedicated SQL, which incurs fees. For more information about the fees, see Billable items for the pay-by-feature billing method.

Comparison item	Console (recommended)	OpenAPI	CLI	Cloud Shell	SDK
Data volume limit	Query only: Up to 20 million rows in the China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), and Singapore regions. Up to 1 million rows in other regions. Maximum data volume: 20 GB. SQL analysis: Maximum data volume: 2 GB.	Query only: Up to 20 million rows in the China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), and Singapore regions. Up to 1 million rows in other regions. Maximum data volume: 20 GB. SQL analysis: Maximum data volume: 2 GB.	Query only: Unlimited SQL analysis: Maximum data volume: 2 GB.	Query only: Unlimited SQL analysis: Maximum data volume: 2 GB.	Query only: Unlimited SQL analysis: Maximum data volume: 2 GB.
Deployment	None	None	Manually install the Simple Log Service CLI.	Automatic deployment. Wait for initialization on the first run.	Manually install a Simple Log Service SDK and write custom code.
Authorization	Download permission configuration	Download permission configuration	Manual configuration	Automatic configuration	Manual configuration
Dedicated SQL	Required for downloading SQL analysis results.	Required for downloading SQL analysis results.	Not used	Not used	Configure parameter settings as needed.
Outbound internet traffic	None	None	Fees are waived if the tool is deployed on an ECS instance in the same region as the project and uses a Simple Log Service private endpoint.	Fees are waived if the project is in the China (Shanghai) region.	Fees are waived if the tool is deployed on an ECS instance in the same region as the project and uses a Simple Log Service private endpoint.
Network Attached Storage (NAS) integration	None	None	Manual configuration, if required.	Automatic configuration	Manual configuration, if required.

Alternatively, use LogShipper to ship logs to Object Storage Service (OSS) and download them from OSS. For more information about log shipping, see Create an OSS shipping task (New).

Download permission configuration (click to view)

{
  "Version": "1",
  "Statement": [
    {
      "Action": [
        "log:ListDownloadJobs",
        "log:CreateDownloadJob",
        "log:GetDownloadJob",
        "log:DeleteDownloadJob"
      ],
      "Resource": [
        "acs:log:*:*:project/your-project-name/downloadjob/*"
      ],
      "Effect": "Allow"
    }
  ]
}

Procedure

Console

The Simple Log Service console supports downloading both logs and query and analysis results. The following steps use log download as an example. To download query and analysis results, run a query statement, and then on the Graph tab, click Download Log.

Important

If logs exceed the single-task limit, only the maximum allowed number is downloaded. To get the full logs, narrow the time range and download in batches.
Each Alibaba Cloud account supports up to three concurrent download tasks, with no limit on the total number of downloads. If this limit is exceeded or multiple Resource Access Management (RAM) users initiate downloads simultaneously, wait for running tasks to finish before retrying.
Download records are automatically deleted after one day.
The system retries a download task automatically if a network error occurs or the query is inaccurate. After three retries, the task status changes to Failed.
Difference between online download (Save As) and offline download (Task Management):
- Save As: Downloads the results currently displayed on the page. Suitable for small datasets. Exporting large volumes, such as hundreds of thousands of rows, may cause the browser to become unresponsive or crash. In such cases, use the offline download feature in Task Management, or narrow the query time range to reduce the volume.
- Task Management (offline download): Designed for large datasets. Tasks run asynchronously in the background. You can close the page after creating the task and download the file once it completes.
- The actual number of downloaded rows from an offline task may differ slightly from the count displayed in the frontend query results. This is expected behavior.
- If a page refresh causes copy operations to fail or online download to fail, use the log download feature instead of manual copy-paste.
An offline download job has no fixed duration. The time that a job takes depends on network conditions, the compression ratio of your data, the current system load, and whether the job must read data from Infrequent Access storage or Archive storage. Simple Log Service provides no duration SLA and no formula for estimating the duration, so a slow job is not necessarily a failed job. If a download is slow, or a job makes no progress for a long time, narrow the query time range to one day or one week, create the download jobs in batches, and then merge the downloaded files to obtain the full logs. Downloading in batches improves both the speed and the stability of the download. If the query range covers Infrequent Access storage or Archive storage data in tiered storage, the read time increases significantly. Narrow the time range further, or make the query statement more selective, to reduce the volume of data that is read from those storage tiers.
Troubleshooting when online download fails but offline download succeeds:
- Check whether the network connection is stable and whether the query is too complex.
- Try offline download (Task Management) or CLI/SDK, which are more stable and better suited for large data volumes.
- If online download continues to fail, narrow the time range and download in smaller batches.
If the Download Log button is grayed out and unavailable, check the following:
- Confirm that you are on the Raw Logs tab. Other views such as statistical charts do not support downloading.
- Confirm that the query returns results. If the query result is empty, the download button is grayed out. Adjust your query statement or time range and run the query again.
- Confirm that the RAM user has been granted download-related permissions, including log:CreateDownloadJob and log:ListDownloadJobs. For more information, see the Download permission configuration section in this document.
- Confirm that the number of concurrent download tasks has not reached the limit. Each Alibaba Cloud account supports up to three concurrent download tasks. If the limit is reached, wait for existing tasks to complete before retrying.
- If the query returns very few results, such as only 20 rows, the filter conditions may be too restrictive. Try widening the time range or adjusting the query statement.

Log on to the Simple Log Service console.
In the Projects section, click the target project.
In the left-side navigation pane, choose Log Storage > Logstores. On the LogStores page, click the target LogStore.
Enter a query statement and select a time range.
On the Raw Logs tab, choose > Download Log.

In the Download Log dialog box, configure the following parameters and click OK.

Parameter	Description
Task Name	Download task name.
Log Quantity	Number of logs to download.
Data Format	CSV or JSON. CSV: Column names come from the first 100 log fields. New fields in subsequent logs are stored in JSON format in an unnamed final column. JSON: Each log entry is a single JSON line.
Quote	Encloses fields that contain special characters to prevent unintended escaping.
Download Inaccurate Results	If you select No, the download fails if the query results are inaccurate.
Compression Method	gzip, lz4, zstd, or none. Compression significantly reduces file size and download time for large datasets.

In the Download Tasks dialog box, wait for the Status of the task to change to Successful, and then click Download to save the logs to your local machine.

Alternatively, on the Raw Logs tab, you can choose > Download Tasks to open the Download Tasks dialog box and view download records.

OpenAPI

Call the CreateDownloadJob operation to create a log download task.

Important

If logs exceed the single-task limit, only the maximum allowed number is downloaded. To get the full logs, narrow the time range and download in multiple batches.
Each Alibaba Cloud account supports up to three concurrent download tasks, with no limit on the total number of downloads. If this limit is exceeded or multiple RAM users initiate downloads simultaneously, wait for running tasks to complete before retrying.
Download records are automatically deleted after one day.
The system retries a download task automatically if a network error occurs or the query is inaccurate. After three retries, the task status changes to Failed.

Note

Notes for OpenAPI batch download:

Index configuration requirement: When calling the GetLogs API or running SQL queries, ensure that the queried fields are configured in the LogStore index. If a queried field is not indexed, a ParameterInvalid error is returned. For more information, see CreateIndex.
Download specific fields: To download specific fields such as message, use a SELECT statement to filter the results, for example, * | SELECT message, before executing the download. This reduces unnecessary data transfer.

Cloud Shell

Cloud Shell is deployed in China (Shanghai). Downloading from LogStores in other regions incurs outbound internet traffic fees. Pricing.

Large-scale log export: For large volumes of logs, such as hundreds of millions of entries, use Cloud Shell or CLI/SDK instead of downloading in batches through the console. Cloud Shell requires no manual dependency installation and suits large-scale export tasks.

For detailed steps, see Use Cloud Shell to download log data.

CLI

For large log volumes, use the command line interface.

Install the command line interface. For more information, see Install the CLI.
Obtain the AccessKey pair of your account. For more information, see AccessKey pair.

Obtain the command that downloads logs. For the detailed steps, see get_log_all.

Example: The following command downloads logs and saves them to downloaded_data.txt in the working directory.

aliyunlog log get_log_all --project="aliyun-test-project" --Logstore="aliyun-test-Logstore" --from_time="2024-07-01 15:33:00+8:00" --to_time="2024-07-09 15:23:00+8:00"  --query="status:200|select request_method as method,COUNT(*) as pv group by method order by pv" --region-endpoint="cn-hangzhou.log.aliyuncs.com" --format-output=json --access-id="LT***CyGg" --access-key="8P***zi" >> ./downloaded_data.txt

The log download command of the Simple Log Service command-line interface (CLI) does not currently support --compress and --query in the same command. If you need to filter logs by condition and compress the download of a large data volume at the same time, call the CreateDownloadJob operation to create a download job instead.

For more information, see Use the SLS CLI.

SDK

For large log volumes, use a Simple Log Service software development kit (SDK).

The SDK uses the same API for both downloading and querying logs.

The following code is a Python SDK example:

import os
import time

from aliyun.log import LogClient
from aliyun.log import GetLogsRequest

# The endpoint of Simple Log Service.
endpoint = 'cn-qingdao.log.aliyuncs.com'
# This example obtains the AccessKey ID and AccessKey secret from environment variables.
accessKeyId = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_ID', '')
accessKey = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_SECRET', '')
# The name of the project.
project = 'your-project-name'
# The name of the Logstore.
Logstore = 'your-Logstore-name'
client = LogClient(endpoint, accessKeyId, accessKey)
request = GetLogsRequest(project, Logstore, fromTime=int(time()-3600), toTime=int(time()), topic='', query="*", line=100, offset=0, reverse=False)
# Or
# request = GetLogsRequest(project, Logstore, fromTime="2018-1-1 10:10:10", toTime="2018-1-1 10:20:10", topic='', query="*", line=100, offset=0, reverse=False)

res = client.get_logs(request)
res.log_print()

For more information, see SDK Reference Overview.

FAQ

How do I download only specific fields through the console?

To download only specific fields through the console, first make sure that each field that you want, such as pod_name, time, or content, has a field index created in the LogStore index and has statistics enabled. A field index applies only to the logs that are written after the index is created or updated, so historical logs must be reindexed before you can query and aggregate them by field.

On the query and analysis page, run an SQL statement that keeps only the fields that you want to download. The statement takes the form query condition | SELECT "field1", field2. For example:

__tag__:__container_name__: tms-prod-bp-server | SELECT "__tag__:__pod_name__", __time__, content

After the query runs, go to the Graph tab and click Download Log to download the query and analysis results. To export results that contain a specific keyword and to raise the limit on returned rows, add limit to the statement. For example:

keyword | select content limit 999999

You can also download specific fields through OpenAPI. The OpenAPI section of this topic describes the index configuration requirement and the use of the SELECT statement. Downloading SQL analysis results uses Dedicated SQL. For a LogStore whose billing method is pay-by-ingested-data, downloading query and analysis results is free. For the applicable billing rules, see the Download methods section of this topic.

What do I do when the console log list does not open, the page times out, or an online download fails?

If the console log list does not open, the page times out, or an online download fails, first clear the browser cache, open the console again in incognito mode, or switch to Chrome or Edge. Also confirm that your network connection is stable, and access the console from a network environment that is close to the region of the LogStore.

If the problem persists, check whether the number of concurrent download jobs of your Alibaba Cloud account has reached the limit. The limit is also reached when several Resource Access Management (RAM) users start downloads at the same time. The Important note block in the Console section of this topic describes the concurrency limit and the checks to run when an online download fails but an offline download succeeds.

If the console front end keeps failing, run a large log export through the command line interface (CLI), Cloud Shell, or a Simple Log Service software development kit (SDK) instead. These methods do not depend on the browser or on the console page.

How long are console log export records kept, and how do I review the download history?

The Download Tasks list in the console keeps export records for the last 1 day only. Export records that are older than 1 day are deleted automatically, so download the file before the record expires.

To review an earlier download history, such as the API calls that created the download jobs, use operation audit (ActionTrail) to query the management API operation records of your account.

How do I calculate the average size of a single log entry?

Use the offline download feature to calculate the average size of a single log entry. Select a period with a traffic spike and a period with normal traffic, create a download job for each period in the console, and download both files to your local machine. Then divide the total number of bytes of a file by the number of log entries in it. For example, run wc -c on Linux to get the byte count of the file, or use a Python script to count the bytes and the entries in one pass.

For a large volume of data, download the logs in several time ranges and calculate each range separately. A single download is bounded by the limits in the Download methods comparison table of this topic: for a query without analysis, up to 20 million rows in the China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), and Singapore regions and up to 1 million rows in other regions, with a data volume of no more than 20 GB; for SQL analysis, a data volume of no more than 2 GB. If a single export still does not meet your needs, use LogShipper to ship the logs to Object Storage Service (OSS) and download them from OSS. For more information, see Create an OSS shipping task (New).

Why are Task Name and Display Name required in the download dialog box?

Task Name and Display Name are both required parameters of the CreateDownloadJob operation that creates a log download job. Task Name uniquely identifies the download job within the same project, so that the console and the API address exactly one job when you query its status or delete it. Display Name is the identifier that is shown to you in the console list, which makes a download job easier to manage and find later.