All Products
Search

This Product

    Simple Log Service:Stability and limits

    Document Center

    Simple Log Service:Stability and limits

    Last Updated:Jun 16, 2026

    The new version of data shipping from Simple Log Service (SLS) to MaxCompute provides high availability with automatic retries. This topic covers its stability guarantees, dirty data handling, and usage limits.

    Stability

    Reading from SLS

    Item

    Description

    Availability

    High availability.

    If SLS encounters an error and data cannot be read, the MaxCompute data shipping task retries at least 10 times internally. If the task still fails, an error is reported and the task restarts.

    Writing to MaxCompute

    Item

    Description

    Concurrency

    Data is partitioned by SLS shards, and shipping instances are created accordingly for rapid scale-out.

    If a source Logstore in SLS performs a shard split, the shipping instances scale out within seconds to accelerate data exporting.

    No data loss

    MaxCompute data shipping tasks use consumer groups to ensure consistency. The offset is committed only after data is delivered, which prevents data loss.

    Schema changes

    If you add a column to a MaxCompute table during data shipping, the new column is written only to new partitions, not to existing or current partitions.

    Note

    Due to MaxCompute limits, you cannot insert, update, or delete columns, or change the column order during data shipping. These operations cause the data shipping task to become abnormal and unrecoverable. For more information, see MaxCompute limits.

    Handling dirty data

    Error type

    Counted as a failed record

    Description

    Partition error

    Yes

    Common cases include invalid partitions or non-existent partition key columns. The data record is not written to MaxCompute.

    Invalid data column

    No

    Common cases include data type mismatch or type conversion failure. Only the affected column is skipped; data in other columns is written to MaxCompute as normal.

    Data column too long

    No

    A common case is that the data exceeds the length limit of the string or varchar type. The data in this column is truncated before being written to MaxCompute. Data in other columns is written as normal.

    Monitoring and alerts

    Stability

    Description

    Monitoring and alerts

    Data shipping provides comprehensive monitoring to track metrics such as latency and traffic in real time. You can configure custom alerts based on your business needs to detect shipping issues promptly, such as insufficient export instances or network quota limits. For more information, see Set an alert for a MaxCompute data shipping task (new version).

    Restarting a task

    Item

    Description

    Too many partitions

    When a task restarts, if there are too many partitions and the write operation is not complete within 5 minutes, data duplication may occur.

    Data write failure

    When a task restarts and fails to write data to MaxCompute due to authorization or network errors, partial data duplication may occur.

    Limits

    Network

    Item

    Description

    Network for intra-region shipping

    Data is transmitted over the Alibaba Cloud internal network when shipped within the same region, which ensures better network stability and speed.

    Read traffic

    Item

    Description

    Read traffic

    A maximum traffic limit applies to each project and each shard. For more information, see Data reads and writes.

    If the limit is exceeded, split the shard or request an increase to the read traffic limit for the project. Exceeding the limit causes the MaxCompute data shipping task to fail when reading data. The task retries at least 10 times internally. If it still fails, an error is reported and the task restarts.

    Writing to MaxCompute

    Item

    Description

    Concurrent instances

    The maximum number of concurrent export instances is 64.

    If the number of shards exceeds 64, multiple shards are merged into one instance for exporting. The system tries to keep the number of shards in each instance the same.

    Write threshold

    • The maximum write speed for a single MaxCompute partition is 10 MB/s.

    • The maximum write speed for a single MaxCompute project varies by region. For example, the limit for China (Hangzhou) is 1 MB/s × 300 = 300 MB/s. The formula is described as follows:

      • 1 MB/s: The speed at which a single slot writes data to MaxCompute. For more information, see Limits of Lindorm Tunnel Service.

      • 300: The maximum number of available slots for a single project in the China (Hangzhou) region. For more information, see Shared resources.

    Important

    If you exceed the MaxCompute write limit, writes become unstable and throttling is triggered on the MaxCompute side. This can cause FlowExceeded or SlotExceed errors. Contact MaxCompute on-duty engineers to resolve the issue.

    Table schema modification prohibited during shipping

    MaxCompute data shipping (new version) uses MaxCompute stream writing. During stream writing, the MaxCompute Tunnel Service prohibits schema modifications such as inserting, updating, or deleting columns, or changing the column order in the target table. For more information, see Overview of Lindorm Tunnel Service.

    Because of this restriction, you cannot use MaxCompute data shipping (new version) and MaxCompute data shipping (legacy version) to write to the same MaxCompute table at the same time.

    Unsupported special tables

    You cannot write data to MaxCompute external tables, transactional tables, or clustered tables.

    Table schema changes

    If your MaxCompute table schema changes, you must pause the MaxCompute data shipping task for 20 minutes and then restart it for the schema change to take effect.

    Start time

    • Start at a specific time: The data shipping time is calculated based on __receive_time__. The start time cannot be more than 24 hours earlier than the current time.

    • Specific time range: The data shipping time is calculated based on __receive_time__. The start time cannot be more than 24 hours earlier than the current time and cannot be later than the end time.

    Note

    Due to the slot and queries per second (QPS) limits of MaxCompute, shipping historical data can easily exceed the MaxCompute write threshold. This feature is therefore no longer supported.

    Permission management

    Item

    Description

    Write authorization

    MaxCompute write authorization supports both Resource Access Management (RAM) users and RAM roles. You must grant permissions separately in MaxCompute.

    Data types

    • Regular columns

      Type

      Example

      Description

      string

      "hello"

      Maximum length: 8 MB.

      datetime

      "2021-12-22 05:00:00"

      The data in SLS must conform to the MaxCompute data format requirements.

      date

      "2021-12-22"

      The data in SLS must conform to the MaxCompute data format requirements.

      timestamp

      1648544867

      Millisecond or second precision.

      decimal

      1.2

      The data in SLS must conform to the MaxCompute data format requirements.

      char

      "hello"

      Maximum length: 255 bytes.

      varchar

      "hello"

      Maximum length: 65,535 bytes.

      binary

      "hello"

      Maximum length: 8 MB.

      bigint

      123

      Supports up to int64.

      boolean

      1

      • 1, t, T, true, TRUE, and True are parsed as True.

      • 0, f, F, false, FALSE, and False are parsed as False.

      double

      1.2

      Supports up to 64-bit floating-point numbers.

      float

      1.2

      Supports up to 32-bit floating-point numbers.

      integer

      123

      Supports up to int32.

      smallint

      12

      Supports up to int16.

      tinyint

      12

      Supports up to int8.

    • Partition key columns

      Item

      Description

      Partition key column

      Treated as a string. Must meet the format requirements for MaxCompute partition key columns.

      Configuring a log field other than __partition_time__ or __receive_time__

      If you configure a log field other than __partition_time__ or __receive_time__ for a partition key column, data shipping performance may be affected.

    Manage shipping

    Item

    Description

    Pausing a data shipping task

    A data shipping task records the log cursor of the last delivery. When the task resumes, it continues from the recorded cursor. The following rules apply when you pause a data shipping task.

    • If you pause a task for a period that does not exceed the data retention period, the system resumes shipping from where it left off when you restart the task. No data is lost.

    • If you pause a task for a period that exceeds the data retention period, the system starts shipping from the nearest data record to the position where it was paused when you restart the task.

    MaxCompute IP whitelist

    Item

    Description

    Enabling a whitelist in MaxCompute project management, such as a classic network IP whitelist, may cause data shipping to fail

    Run the following commands in MaxCompute to resolve the data shipping failure caused by the whitelist.

    1. Run setproject; select 1; to query the current whitelist configuration.

    2. Add AliyunLogSLRService,AliyunLogDefaultService to the existing service whitelist. The complete command is as follows:

      setproject odps.security.ip.whitelist.services=AliyunLogSLRService,AliyunLogDefaultService;

    3. After the configuration is complete, run setproject; select 1; to confirm that the whitelist configuration has taken effect.

    For more information, see Job failure after enabling an IP whitelist