All Products
Search

This Product

    :Create an OSS shipping task (legacy)

    Document Center

    :Create an OSS shipping task (legacy)

    Last Updated:Oct 29, 2026

    Ship collected log data from Simple Log Service to Object Storage Service (OSS) for long-term storage and analysis.

    Important

    The old version of shipping logs to OSS is discontinued. Refer to the new version.

    Prerequisites

    • A project and a LogStore are created.

    • Log data is collected.

    • An OSS bucket is created in the region where the SLS project resides. For more information, see Create buckets.

    • You must complete the Cloud Resource Access Authorization.

      To ship logs across Alibaba Cloud accounts or use a RAM user, complete RAM authorization for shipping logs to OSS first.

      On the Cloud Resource Access Authorization page, select the AliyunLogDefaultRole role, which includes OSS write permissions, and click Agree to Authorization.

    Overview

    Simple Log Service automatically archives Logstore data to OSS for long-term storage and analysis.

    • OSS lifecycle rules support long-term data retention.

    • Process OSS data with E-MapReduce, Data Lake Analytics (DLA), or custom programs.

    Shipping data

    Important

    • After enabling shipping, Simple Log Service runs shipping instances concurrently.

    • After a shipping instance is created, verify the task status and check the data shipped to OSS.

    1. Log on to the Simple Log Service console.

    2. In the Projects section, click the one you want.

      image

    3. On the Log Storage > Logstores tab, click > to the left of the target LogStore, and then choose Data Processing > Export > Object Storage Service.

    4. Move the pointer over Object Storage Service and click +.

    5. In the Data Shipping to OSS panel, configure the following parameters and click OK.

      Set Shipping Version to Old Version and configure the following parameters.

      Parameter

      Description

      OSS shipping name

      The name of the shipping task.

      OSS bucket

      The name of the OSS bucket.

      Important

      The OSS bucket must already exist, must have Write Once Read Many (WORM) disabled, and must reside in the same region as the SLS project.

      You can ship data to buckets of the Standard, Infrequent Access, Archive, Cold Archive, and Deep Cold Archive storage classes. By default, shipped objects use the same storage class as the bucket.

      Non-Standard storage classes have minimum storage duration and minimum billable size requirements. Choose the destination bucket storage class based on your needs.

      OSS prefix

      The directory prefix in the OSS bucket. The value cannot start with a forward slash (/) or a backslash (\).

      Logstore data is shipped to this directory in the target OSS bucket.

      Partition format

      A dynamically generated directory path based on the task creation time. The value cannot start with a forward slash (/). Default: %Y/%m/%d/%H/%M. For examples, see Partition format. For parameter reference, see strptime API.

      RAM role for writing to OSS

      The RAM role that grants write access to the OSS bucket for the shipping task.

      • Grants the OSS data shipping job the permissions to write data to the OSS bucket. If the LogStore and the OSS bucket belong to the same Alibaba Cloud account, select Default Role. If they belong to different accounts, select Custom Role. - Default Role: Authorizes the OSS data shipping job to write data to the OSS bucket as the AliyunLogDefaultRole system role of Alibaba Cloud. Enter the Alibaba Cloud Resource Name (ARN) of AliyunLogDefaultRole. For information about how to obtain the ARN, see Access data using a default role. - Custom Role: Authorizes the OSS data shipping job to write data to the OSS bucket as a custom role. Grant the custom role the permissions to write data to the OSS bucket, and then enter the ARN of the custom role in OSS Write RAM Role. Obtain the ARN based on the following scenarios: - If the LogStore and the OSS bucket belong to the same Alibaba Cloud account, see Step 2: Grant write permissions for an OSS bucket. - If the LogStore and the OSS bucket belong to different Alibaba Cloud accounts, see Grant write permissions for an OSS bucket across accounts.

      Shipping size

      Maximum uncompressed data size per shard. Valid values: 5 to 256 MB.

      A new shipping instance is created when data reaches this size.

      Storage format

      The file format for data stored in OSS. Supported formats: JSON format, CSV format, and Parquet format.

      Compression

      The compression method for data shipped to OSS.

      • None: Data is stored without compression.

      • snappy: Compresses data by using the snappy algorithm to reduce OSS storage usage.

      Shipping interval

      The shipping interval per shard. Valid values: 300 to 900 seconds. Default: 300 seconds.

      A new shipping instance is created when this interval elapses.

    View data in OSS

    After data is shipped to OSS, you can access the data in the OSS console or by using the OSS API, an SDK, or other methods. For more information, see Object management overview.

    Each shipping operation maps to an OSS object URL in the following format. The following table describes the partition formats, based on a shipping job created at 19:50:43 on January 20, 2022.

    oss://OSS-BUCKET/OSS-PREFIX/PARTITION-FORMAT_RANDOM-ID

    In this format, OSS-BUCKET is the bucket name, OSS-PREFIX is the directory prefix, PARTITION-FORMAT is the partition path generated from the shipping task creation time by using the strptime API, and RANDOM-ID is the unique ID of the shipping task.

    Note

    The OSS directory is based on the shipping task creation time, not the log data timestamp. For example, if a task created at 2016-06-23 00:00:00 ships data written after 2016-06-22 23:55 every 5 minutes, analyzing all data from 2016-06-22 requires checking both the 2016/06/22 directory and the first ten minutes in the 2016/06/23/00/ directory.

    Partition format

    Each shipping task maps to an OSS directory in the format oss://OSS-BUCKET/OSS-PREFIX/PARTITION-FORMAT_RANDOM-ID. PARTITION-FORMAT is derived from the task creation time. The following table provides examples based on a task created at 2017/01/20 19:50:43.

    OSS bucket

    OSS prefix

    Partition format

    OSS object path

    test-bucket

    test-table

    %Y/%m/%d/%H/%M

    oss://test-bucket/test-table/2017/01/20/19/50_1484913043351525351_2850008

    test-bucket

    log_ship_oss_example

    year=%Y/mon=%m/day=%d/log_%H%M%S

    oss://test-bucket/log_ship_oss_example/year=2017/mon=01/day=20/log_195043_1484913043351525351_2850008.parquet

    test-bucket

    log_ship_oss_example

    ds=%Y%m%d/%H

    oss://test-bucket/log_ship_oss_example/ds=20170120/19_1484913043351525351_2850008.snappy

    test-bucket

    log_ship_oss_example

    %Y%m%d/

    oss://test-bucket/log_ship_oss_example/20170120/_1484913043351525351_2850008

    Note

    This format may prevent platforms such as Hive from parsing the OSS content. We recommend that you avoid this format.

    test-bucket

    log_ship_oss_example

    %Y%m%d%H

    oss://test-bucket/log_ship_oss_example/2017012019_1484913043351525351_2850008

    When you use a big data platform such as Hive or MaxCompute, or Alibaba Cloud Data Lake Analytics (DLA), to analyze OSS data, you can set PARTITION-FORMAT in the object name to the key=value format if you want to use partition information. Example: oss://test-bucket/log_ship_oss_example/year=2022/mon=01/day=20/log_195043_1484913043351525351_2850008.parquet. In this example, three partition columns are configured: year, mon, and day.

    Manage shipping tasks

    On the OSS Shipper page, you can modify, disable, or retry shipping tasks, and view task statuses and error messages.

    • Modify a shipping task

      Click Shipping Configuration to modify the shipping task. Parameter descriptions: Ship data.

    • Disable a shipping task

      Click Disable to stop the shipping task.

    • View task status and error messages

      View shipping tasks and their statuses from the last two days.

      • Task status

        Status

        Description

        Succeeded

        The shipping task is running as expected.

        Running

        The shipping task is in progress. Check back later.

        Failed

        The task failed due to an external error and cannot be retried automatically. Troubleshoot based on the error message and retry.

      • Error messages

        If a shipping task fails, the console displays an error message.

        Error message

        Cause

        Solution

        UnAuthorized

        You do not have the required permissions.

        Check the following:

        • The owner of the OSS bucket has created the AliyunLogDefaultRole role.

        • The Alibaba Cloud account ID in the role's trust policy is correct.

        • The AliyunLogDefaultRole role has write permissions on the OSS bucket.

        • The ARN of the RAM role is correct.

        ConfigNotExist

        The shipping configuration does not exist.

        This error typically occurs because the shipping task was disabled. Re-enable the task and retry.

        InvalidOssBucket

        The OSS bucket does not exist.

        Check the following:

        • The OSS bucket is in the same region as the Simple Log Service project.

        • The bucket name is correct.

        InternalServerError

        An internal error occurred in Simple Log Service.

        Retry the task.

      • Retry tasks

        Simple Log Service automatically retries failed tasks from the last two days at exponential backoff intervals: 15 minutes, 30 minutes, 60 minutes, and so on.

        To immediately retry failed tasks, click Retry All Failed Tasks, click Retry next to the target task, or use an API or SDK to retry a specific task.