Configure a Log Service trigger

Scenarios

You can configure a Log Service trigger to obtain updated data and triggers function execution at a scheduled time to incrementally consume data from a Logstore. You can also use the trigger to perform a custom processing task in functions, such as a data cleansing task or a data processing task, and deliver data to a third-party service. This example only shows how to obtain and print log data.

Prerequisites

• Function Compute
  • Create a Service
  • Create a function
• Log Service
  • Create a Log Service project and a Logstore
  Note You must create a Log Service project and two Logstores. One Logstore is used to process logs and data sources, and the other Logstore is used to store logs that are generated by Function Compute.

Create a trigger

1. Log on to the Function Compute console.
2. In the left-side navigation pane, click Services and Functions.
3. In the top navigation bar, select the region where the service resides.
4. On the Services page, find the service that you want to manage and click Functions in the Actions column.
5. On the Functions page, click the name of the function that you want to manage.
   
6. On the function details page, click the Triggers tab, select the version or alias from the Version or Alias drop-down list, and then click Create Trigger.
   
7. In the Create Trigger panel, enter the relevant information. Click OK.
   

   Parameter	Description	Example
   Trigger Type	Select Log Service.	Log Service
   Name	Enter a custom trigger name.	log-trigger
   Version or Alias	The default value is LATEST. If you want to create a trigger for another version or alias, select a version or alias in the upper-right corner of the function details page. For more information about trigger versions, see Manage versions and Manage aliases.	LATEST
   Log Service Project	Select a Log Service project.	aliyun-fc-cn-chengdu-2238f0df-a742-524f-9f90-976ba457****
   Logstore	Select a Logstore. The current trigger subscribes to the data in the Logstore at a scheduled time and sends the data to Function Compute for processing.	function-log
   Trigger Interval	Enter the time interval at which the Log Service trigger triggers function execution. Valid values: [3,600]. Unit: seconds. Default value: 60.	60
   Retries	Enter the maximum number of retries that are allowed for a single trigger operation. Valid values: [0,100]. Default value: 3	3
   Trigger Log	Select a Logstore to which you want to store the logs that are generated when the Log Service trigger triggers function execution.	function-log2
   Invocation Parameters	You can configure a custom parameter in this field. This parameter is passed to the function as the parameter field of the event parameter. The value of this parameter must be a string in JSON format. By default, this parameter is empty.	None
   Role Name	Select AliyunLogETLRole. Note After you configure the preceding parameters, click OK. If this is the first time that you create a trigger of this type, click Authorize Now in the dialog box that appears.	AliyunLogETLRole

   After the trigger is created, the created trigger is displayed in the Trigger Name list.

Write a function

After you create the Log Service trigger, you can write the function code. The function is triggered when Log Service collects incremental logs. Function Compute obtains the logs and then prints the collected logs.

# -*- coding: utf-8 -*-
import logging
import json
from aliyun.log import LogClient
from time import time

def logClient(endpoint, creds):
  logger = logging.getLogger()
  logger.info('creds info')
  logger.info(creds.access_key_id)
  logger.info(creds.access_key_secret)
  logger.info(creds.security_token)
  accessKeyId = 'your accessKeyId'
  accessKey = 'your accessKeyId scr'
  client = LogClient(endpoint, accessKeyId, accessKey)
  return client

def handler(event, context):
  logger = logging.getLogger()
  logger.info('start deal SLS data')
  logger.info(event.decode().encode())
  info_arr = json.loads(event.decode())
  fetchdata(info_arr['source'],context)
  return 'hello world'

 def fetchdata(event,context):
  logger = logging.getLogger()
  endpoint = event['endpoint']
  creds = context.credentials
  client = logClient(endpoint, creds)
  if client == None :
      logger.info("client create failed")
      return False
  project = event['projectName']
  logstore = event['logstoreName']
  start_cursor = event['beginCursor']
  end_cursor = event['endCursor']
  loggroup_count = 10
  shard_id = event['shardId']
  while True:
      res = client.pull_logs(project, logstore, shard_id, start_cursor, loggroup_count, end_cursor)
      res.log_print()
      next_cursor = res.get_next_cursor()
      if next_cursor == start_cursor :
          break
      start_cursor = next_cursor

    #log_data =  res.get_loggroup_json_list()
  return True                        

Test the function

After you write the function, you must debug the function to verify whether the code is correct. Function execution is triggered when a new log is saved to a Logstore.

1. On the Function Details page, click the Function Code tab, and then click the icon. From the drop-down list, select Configure Test Parameters.
2. In the Configure Test Parameters dialog box, click the Create New Test Event or Edit Existing Test Event tab, and specify the Event Name and event content. Click OK.
   The event parameter is an input parameter of Function Compute. The following code shows the format of parameters:

   {
       "parameter": {},
       "source": {
           "endpoint": "http://cn-shanghai-intranet.log.aliyuncs.com",
           "projectName": "log-com",
           "logstoreName": "log-en",
           "shardId": 0,
           "beginCursor": "MTUyOTQ4MDIwOTY1NTk3ODQ2Mw==",
           "endCursor": "MTUyOTQ4MDIwOTY1NTk3ODQ2NA=="
       },
       "jobName": "1f7043ced683de1a4e3d8d70b5a412843d81****",
       "taskId": "c2691505-38da-4d1b-998a-f1d4bb8c****",
       "cursorTime": 1529486425
   }                       

   Parameter	Description	Example
   parameter	The value of Invocation Parameters.	None
   source	The log block information that you want the function to read from Log Service. endpoint: the Alibaba Cloud region in which the Log Service project is located. projectName: the name of the Log Service project. logstoreName: the name of the Logstore. shardId: the ID of a specified shard in the Logstore. beginCursor: the location where data consumption starts. endCursor: the location where data consumption ends.	{ "endpoint":"http://cn-shanghai-intranet.log.aliyuncs.com", "projectName":"log-com", "logstoreName":"log-en", "shardId":0, "beginCursor":"MTUyOTQ4MDIwOTY1NTk3ODQ2Mw==", "endCursor":"MTUyOTQ4MDIwOTY1NTk3ODQ2NA==" }
   jobName	The name of an extract, transform, and load (ETL) job in Log Service. The Log Service trigger that is configured for the function corresponds to an ETL job of Log Service.	1f7043ced683de1a4e3d8d70b5a412843d81****
   taskId	For an ETL job, taskId is the identifier for a deterministic function invocation.	c2691505-38da-4d1b-998a-f1d4bb8c****
   cursorTime	The Unix timestamp of the time when the last log arrives at Log Service.	1529486425

3. On the function details page, click the Function Code tab, and then click Test Function.

Check the result

On the Function Code tab, you can view a successful command.

References

To modify or delete a created trigger, see Manage triggers.