Function Compute:Simple log service triggers

Use cases

• Data cleansing and processing — collect, filter, and transform logs in real time before forwarding them downstream.

  

• Data shipping — build data pipelines that deliver log data to other cloud-based big data products or third-party services.

  

How it works

Each SLS trigger maps one-to-one to an ETL job in SLS. When you create the trigger, SLS starts a timer that periodically polls for shard information in the configured Logstore at the specified interval.

When new data arrives on a shard, SLS generates a trigger event as a triplet <shard_id, begin_cursor, end_cursor> and invokes your function with that event. The cursor range covers all data written during the trigger interval — for example, a 60-second interval produces a range of [now - 60s, now). Your function reads log data from the shard within that range.

If no new data is written to a shard, the function is not triggered for that shard.

Trigger frequency and catch-up behavior

Each shard is triggered independently, so a Logstore with 10 shards produces 10 function invocations per interval during normal operation.

If the current processing position on a shard falls more than 10 seconds behind the latest written data, the trigger accelerates to approximately one invocation every 2 seconds per shard while continuing to process data in windows equal to the original trigger interval. This continues until the shard catches up.

Processing functions

Two function types are supported:

• Template functions — pre-built functions for common ETL tasks. See aliyun-log-fc-functions.

• Custom functions — functions you write. The configuration format depends on your implementation. See ETL function development guide.

Limitations

The maximum number of SLS triggers you can associate with a single Project is five times the number of Logstores in that Project. Configure no more than five SLS triggers per Logstore to avoid degraded data shipping performance.

Prerequisites

Before you begin, ensure that you have:

• An event-triggered function in Function Compute. See Create an event-triggered function

• An SLS Project and two Logstores: one source Logstore for the logs to consume, and one Logstore to record trigger execution logs. See Create a Project and a Logstore

• Continuous log collection into the source Logstore — the trigger only fires when new data is written

Event reference

{
    "parameter": {},
    "source": {
        "endpoint": "http://cn-hangzhou-intranet.log.aliyuncs.com",
        "projectName": "fc-test-project",
        "logstoreName": "fc-test-logstore",
        "shardId": 0,
        "beginCursor": "MTUyOTQ4MDIwOTY1NTk3ODQ2Mw==",
        "endCursor": "MTUyOTQ4MDIwOTY1NTk3ODQ2NA=="
    },
    "jobName": "1f7043ced683de1a4e3d8d70b5a412843d81****",
    "taskId": "c2691505-38da-4d1b-998a-f1d4bb8c****",
    "cursorTime": 1529486425
}

Step 1: Create an SLS trigger

1. Log on to the Function Compute console. In the left-side navigation pane, click Functions.

2. In the top navigation bar, select a region. On the Functions page, click the function you want to manage.

3. On the function details page, click the Trigger tab, then click Create Trigger. In the Create Trigger panel, set Trigger Type to Log Service, configure the following parameters, and click OK.

After the trigger is created, it appears on the Triggers tab. To modify or delete a trigger, see Trigger Management.

Step 2: Configure permissions

1. On the function details page, select the Configuration tab. In the Advanced Settings section, click Modify.

2. In the Advanced Settings panel, select a Function Role:

   • Default role — AliyunFCServerlessDevsRole grants read-only access to SLS and covers most use cases.

   • Custom RAM role — if your function writes to SLS or accesses other services, create a RAM role with:

     1. Trusted entity: Cloud Service → Function Compute. See Create a RAM role for a trusted Alibaba Cloud service.

     2. Permissions: the SLS permissions your function needs. See Examples of custom RAM policies.

3. Click Deploy.

Step 3: Deploy and view logs

1. On the Code tab, paste your function code into the editor and click Deploy. The following Python example reads log data from the source Logstore using the cursor range from the trigger event. It uses temporary credentials from context.credentials to initialize the SLS client — no hardcoded keys required.

   """
   Reads log data from an SLS Logstore within the cursor range
   provided by the trigger event.
   """
   import logging
   import json
   from aliyun.log import LogClient
   
   logger = logging.getLogger()
   
   
   def handler(event, context):
       # Get temporary credentials from the function's execution role.
       creds = context.credentials
       access_key_id = creds.access_key_id
       access_key_secret = creds.access_key_secret
       security_token = creds.security_token
   
       # Parse the trigger event.
       event_obj = json.loads(event.decode())
       source = event_obj['source']
       endpoint = source['endpoint']
       log_project = source['projectName']
       log_store = source['logstoreName']
       shard_id = source['shardId']
       begin_cursor = source['beginCursor']
       end_cursor = source['endCursor']
   
       # Initialize the SLS client with temporary credentials.
       client = LogClient(
           endpoint=endpoint,
           accessKeyId=access_key_id,
           accessKey=access_key_secret,
           securityToken=security_token
       )
   
       # Pull all log groups in the range [begin_cursor, end_cursor).
       while True:
           response = client.pull_logs(
               project_name=log_project,
               logstore_name=log_store,
               shard_id=shard_id,
               cursor=begin_cursor,
               count=100,
               end_cursor=end_cursor,
               compress=False
           )
           log_group_cnt = response.get_loggroup_count()
           if log_group_cnt == 0:
               break
           logger.info("Got %d log group(s) from %s", log_group_cnt, log_store)
           logger.info(response.get_loggroup_list())
           begin_cursor = response.get_next_cursor()
   
       return 'success'

2. On the function details page, go to Logs > Function Logs to verify that the function is receiving and processing data. If the message "The logging feature is not enabled for the current function." appears, click Enable.

(Optional) Step 4: Test with a simulated event

1. On the Code tab, click the icon next to Test Function and select Configure Test Parameters.

2. In the Configure Test Parameters panel, select Create New Test Event or Modify Existing Test Event. Select the Log Service template, enter an event name, and click OK.

3. Click Test Function. After execution completes, view the result above the Code tab.

FAQ

SLS trigger doesn't invoke the function

Check that new data is being written to the configured Logstore — the trigger only fires when shard data changes. If data is flowing, check the trigger logs and function run logs for error messages.

Why is the invocation count high?

Each shard triggers independently. A Logstore with 10 shards produces 10 invocations per interval during normal operation. If the trigger is catching up after a delay (when processing falls more than 10 seconds behind), it accelerates to roughly one invocation every 2 seconds per shard until it catches up.

Error: "denied by sts or ram"

The function role is missing required SLS permissions. Review and update the role as described in Step 2: Configure permissions.