You can create a Log Service trigger to connect Log Service with Function Compute to trigger functions when logs are generated. This topic describes how to configure a Log Service trigger for a function in the Function Compute console, including how to create a trigger, configure input parameters, write function code, and test the function. In this example, Log Service calls Function Compute, and Function Compute obtains and prints logs.

Scenario

You can configure a Log Service trigger to periodically obtain updated data and trigger functions. Triggers like this are ideal for scenarios where you want to incrementally consume data from a Logstore. You can trigger functions to perform post-processing tasks, such as a data cleansing task and a data processing task, and deliver data to a third-party service. This example only shows how to obtain and print log data.

Note You can use a preset template or a custom template to perform post-processing functions.

Prerequisites

Step 1: Create a Log Service trigger

  1. Log on to the Function Compute console.
  2. In the left-side navigation pane, click Services and Functions.
  3. In the top navigation bar, select the region where the service resides.
  4. On the Services page, find the desired service and click Functions in the Actions column.
  5. On the Functions page, click the function that you want to manage.
  6. On the function details page, click the Triggers tab, select a version or alias from the Version or Alias drop-down list, and then click Create Trigger.
  7. In the Create Trigger panel, specify related parameters. After you specify the parameters, click OK.
    Field Description Example
    Trigger Type Select Log Service. Log Service
    Name Enter a custom trigger name. log_trigger
    Version or Alias The default value is LATEST. If you want to create a trigger for another version or alias, select a version or alias in the upper-right corner of the function details page. For more information about versions and aliases of a service, see Manage versions and Manage aliases. LATEST
    Log Service Project Select a Log Service project. aliyun-fc-cn-chengdu-2238f0df-a742-524f-9f90-976ba457****
    Logstore Select a Logstore. The trigger created in this example subscribes to the data in the Logstore at a scheduled time and sends the data to Function Compute for custom processing. function-log
    Trigger Interval Enter the time interval at which Log Service triggers the function.

    Valid values: [3,600]. Unit: seconds. Default value: 60.

    60
    Retries Enter the maximum number of retries that are allowed for a single trigger operation.

    Valid values: [0,100]. Default value: 3

    3
    Trigger Log Select a Logstore to which you want to store the logs that are generated when Log Service triggers a function. function-log2
    Invocation Parameters You can configure a custom parameter in this field. This parameter is passed to the function as the parameter field of the event parameter. The value of this parameter must be a string in JSON format.

    By default, this parameter is empty.

    None
    Role Name Select AliyunLogETLRole.
    Note After you configure the preceding parameters, click OK. If this is the first time that you create a trigger of this type, click Authorize Now in the message that appears.
    AliyunLogETLRole

    After the trigger is created, it is displayed on the Triggers tab. To modify or delete an existing trigger, see Manage triggers.

Step 2: Configure the input parameter of the function

  1. On the function details page, click the Code tab and click the xialatubiao icon. From the drop-down list that appears, select Configure Test Parameters.
  2. In the Configure Test Parameters panel, click the Create New Test Event or Modify Existing Test Event tab, and specify Event Name and the event content. After you specify the parameters, click OK.
    The event parameter is the input parameter of Function Compute. The following code shows the format of the event parameter:
    {
        "parameter": {},
        "source": {
            "endpoint": "http://cn-shanghai-intranet.log.aliyuncs.com",
            "projectName": "log-com",
            "logstoreName": "log-en",
            "shardId": 0,
            "beginCursor": "MTUyOTQ4MDIwOTY1NTk3ODQ2Mw==",
            "endCursor": "MTUyOTQ4MDIwOTY1NTk3ODQ2NA=="
        },
        "jobName": "1f7043ced683de1a4e3d8d70b5a412843d81****",
        "taskId": "c2691505-38da-4d1b-998a-f1d4bb8c****",
        "cursorTime": 1529486425
    }                       
    Field Description Example
    parameter The value of Invocation Parameters. None
    source The log block information that you want the function to read from Log Service.
    • endpoint: the endpoint of the Alibaba Cloud region in which the Log Service project is located.
    • projectName: the name of the Log Service project.
    • logstoreName: the name of the Logstore.
    • shardId: the ID of a specified shard in the Logstore.
    • beginCursor: the offset where data consumption starts.
    • endCursor: the offset where data consumption ends.
    {
        "endpoint":"http://cn-shanghai-intranet.log.aliyuncs.com",
        "projectName":"log-com",
        "logstoreName":"log-en",
        "shardId":0,
        "beginCursor":"MTUyOTQ4MDIwOTY1NTk3ODQ2Mw==",
        "endCursor":"MTUyOTQ4MDIwOTY1NTk3ODQ2NA=="
    }
    jobName The name of an extract, transform, and load (ETL) job in Log Service. Log Service triggers must correspond to ETL jobs in Log Service. 1f7043ced683de1a4e3d8d70b5a412843d81****
    taskId For an ETL job, taskId is the identifier for a deterministic function invocation. c2691505-38da-4d1b-998a-f1d4bb8c****
    cursorTime The Unix timestamp of the time when the last log arrives at Log Service. Unit: seconds. 1529486425

Step 3: Write and test the function

After you create the Log Service trigger, you can write function code and test the function to verify whether the code is correct. Functions are triggered when Log Service collects incremental logs. Function Compute obtains the logs and prints the collected logs.

  1. On the function details page, click the Code tab, enter function code in the code editor, and then click Save.
    In this example, the function code is written in Python. The following code provides a template on how to extract most logical logs. You can specify accessKeyId and accessKey in the code or obtain them from context and creds.
    # -*- coding: utf-8 -*-
    import logging
    import json
    from aliyun.log import LogClient
    from time import time
    
    def logClient(endpoint, creds):
      logger = logging.getLogger()
      logger.info('creds info')
      logger.info(creds.access_key_id)
      logger.info(creds.access_key_secret)
      logger.info(creds.security_token)
      accessKeyId = 'your accessKeyId'
      accessKey = 'your accessKeyId scr'
      client = LogClient(endpoint, accessKeyId, accessKey)
      return client
    
    def handler(event, context):
      logger = logging.getLogger()
      logger.info('start deal SLS data')
      logger.info(event.decode().encode())
      info_arr = json.loads(event.decode())
      fetchdata(info_arr['source'],context)
      return 'hello world'
    
     def fetchdata(event,context):
      logger = logging.getLogger()
      endpoint = event['endpoint']
      creds = context.credentials
      client = logClient(endpoint, creds)
      if client == None :
          logger.info("client create failed")
          return False
      project = event['projectName']
      logstore = event['logstoreName']
      start_cursor = event['beginCursor']
      end_cursor = event['endCursor']
      loggroup_count = 10
      shard_id = event['shardId']
      while True:
          res = client.pull_logs(project, logstore, shard_id, start_cursor, loggroup_count, end_cursor)
          res.log_print()
          next_cursor = res.get_next_cursor()
          if next_cursor == start_cursor :
              break
          start_cursor = next_cursor
    
        #log_data =  res.get_loggroup_json_list()
      return True                        
  2. Click the Code tab and click Test Function.
    After the function is executed, you can view the result on the Code tab.

References

In addition to the Function Compute console, you can configure triggers by using the following methods:
  • Use Serverless Devs to configure triggers. For more information, see Serverless Devs.
  • Use SDKs to configure triggers. For more information, see Supported SDKs.

To modify or delete an existing trigger, see Manage triggers.