All Products
Search

This Product

    Simple Log Service:Create an OSS shipping task (legacy)

    Document Center

    Simple Log Service:Create an OSS shipping task (legacy)

    Last Updated:Jun 03, 2026

    After Simple Log Service collects data, you can ship the data to Object Storage Service (OSS) for long-term storage and analysis.

    Important

    The old version of shipping logs to OSS is discontinued. Refer to the new version.

    Prerequisites

    Overview

    Simple Log Service can automatically archive Logstore data to OSS for long-term storage and analysis.

    • OSS lifecycle rules enable long-term data retention.

    • Process OSS data with E-MapReduce, Data Lake Analytics (DLA), or custom programs.

    Shipping data

    Important

    • After enabling shipping, Simple Log Service runs shipping instances concurrently.

    • After a shipping instance is created, verify the task status and check the data shipped to OSS.

    1. Log on to the Simple Log Service console.

    2. In the Projects section, click the one you want.

      image

    3. On the Log Storage > Logstores tab, click the > icon to the left of the target logstore and choose Data Processing > Export > Object Storage Service.

    4. Hover over Object Storage Service and click the + icon.

    5. In the Data Shipping to OSS panel, configure the following parameters and click OK.

      Set Shipping Version to Old Version and configure the following parameters.

      Parameter

      Description

      OSS shipping name

      The name of the shipping task.

      OSS bucket

      The name of the OSS bucket.

      Important

      • You can ship data to a bucket with the Standard, Infrequent Access (IA), Archive, Cold Archive, or Deep Cold Archive storage class. The storage class of the generated OSS objects defaults to that of the bucket. For more information, see Storage classes.

      • Storage classes other than Standard have minimum storage durations and billable sizes. Choose a storage class for the destination bucket that meets your requirements. For more information, see Storage class comparison.

      OSS prefix

      The directory prefix in the OSS bucket. Cannot start with a forward slash (/) or a backslash (\).

      Logstore data is shipped to this directory in the target OSS bucket.

      Partition format

      Dynamically generates a directory path based on the task creation time. Cannot start with a forward slash (/). Default: %Y/%m/%d/%H/%M. Examples: Partition format. Parameter reference: strptime API.

      RAM role for writing to OSS

      The RAM role that grants the shipping task write access to the OSS bucket.

      Shipping size

      Maximum uncompressed data size per shard. Valid values: 5 to 256 MB.

      A new shipping instance is created when data reaches this size.

      Storage format

      The file format for data stored in OSS. Supported formats: JSON format, CSV format, and Parquet format.

      Compression

      The compression method for OSS data.

      • None: Data is not compressed.

      • snappy: Compresses data using the snappy algorithm to reduce OSS storage usage.

      Shipping interval

      The shipping interval per shard. Valid values: 300 to 900 seconds. Default: 300 seconds.

      A new shipping instance is created when this interval elapses.

    View data in OSS

    After data is successfully shipped to OSS, you can access it using the OSS console, an API, an SDK, or other tools. For more information, see File management.

    The OSS object path is in the following format:

    oss://OSS-BUCKET/OSS-PREFIX/PARTITION-FORMAT_RANDOM-ID

    In this format, OSS-BUCKET is the bucket name, OSS-PREFIX is the directory prefix, PARTITION-FORMAT is the partition path generated from the shipping task's creation time by using the strptime API, and RANDOM-ID is the unique ID of the shipping task.

    Note

    The OSS directory is based on the shipping task creation time, not the data timestamp. For example, if a task created at 2016-06-23 00:00:00 ships data written after 2016-06-22 23:55 every 5 minutes, analyzing all data from 2016-06-22 requires checking both the 2016/06/22 directory and the first ten minutes in the 2016/06/23/00/ directory.

    Partition format

    Each shipping task maps to an OSS directory in the format oss://OSS-BUCKET/OSS-PREFIX/PARTITION-FORMAT_RANDOM-ID. PARTITION-FORMAT is derived from the task creation time. The following table shows examples based on a task created at 2017/01/20 19:50:43.

    OSS bucket

    OSS prefix

    Partition format

    OSS object path

    test-bucket

    test-table

    %Y/%m/%d/%H/%M

    oss://test-bucket/test-table/2017/01/20/19/50_1484913043351525351_2850008

    test-bucket

    log_ship_oss_example

    year=%Y/mon=%m/day=%d/log_%H%M%S

    oss://test-bucket/log_ship_oss_example/year=2017/mon=01/day=20/log_195043_1484913043351525351_2850008.parquet

    test-bucket

    log_ship_oss_example

    ds=%Y%m%d/%H

    oss://test-bucket/log_ship_oss_example/ds=20170120/19_1484913043351525351_2850008.snappy

    test-bucket

    log_ship_oss_example

    %Y%m%d/

    oss://test-bucket/log_ship_oss_example/20170120/_1484913043351525351_2850008

    Note

    This format may prevent platforms such as Hive from parsing OSS content. Avoid this format.

    test-bucket

    log_ship_oss_example

    %Y%m%d%H

    oss://test-bucket/log_ship_oss_example/2017012019_1484913043351525351_2850008

    When you analyze OSS data with big data platforms like Hive, MaxCompute, or Alibaba Cloud Data Lake Analytics (DLA), set the partition format to a key=value format to use partition information. For example, in the path oss://test-bucket/log_ship_oss_example/year=2022/mon=01/day=20/log_195043_1484913043351525351_2850008.parquet, three partition columns are defined: year, mon, and day.

    Manage shipping tasks

    On the OSS Shipper page, you can modify, disable, or retry shipping tasks, and view task status and error messages.

    • Modify a shipping task

      Click Shipping Configuration to modify the shipping task. Parameter descriptions: Ship data.

    • Disable a shipping task

      Click Disable to stop the shipping task.

    • View task status and error messages

      View shipping tasks and their statuses from the last two days.

      • Task status

        Status

        Description

        Succeeded

        The shipping task is running as expected.

        Running

        The shipping task is in progress. Check back later.

        Failed

        The task failed due to an external error and cannot be automatically retried. Troubleshoot based on the error message and retry.

      • Error messages

        If a shipping task fails, an error message appears in the console.

        Error message

        Cause

        Solution

        UnAuthorized

        You do not have the required permissions.

        Check the following:

        • The owner of the OSS bucket has created the AliyunLogDefaultRole role.

        • The Alibaba Cloud account ID in the role's trust policy is correct.

        • The AliyunLogDefaultRole role has write permissions on the OSS bucket.

        • The ARN of the RAM role is correct.

        ConfigNotExist

        The configuration does not exist.

        This error usually occurs because the shipping task was disabled. Re-enable the task and retry.

        InvalidOssBucket

        The OSS bucket does not exist.

        Check the following:

        • The OSS bucket is in the same region as the Simple Log Service project.

        • The bucket name is correct.

        InternalServerError

        An internal error occurred in Simple Log Service.

        Retry the task.

      • Retry tasks

        Simple Log Service automatically retries failed tasks from the last two days with exponential backoff: 15 minutes, 30 minutes, 60 minutes, and so on.

        To immediately retry failed tasks, click Retry All Failed Tasks, click Retry next to the target task, or use an API or SDK to retry a specific task.