All Products
Search

This Product

    OSS trigger

    Document Center

    OSS trigger

    Last Updated: May 22, 2018

    Context

    In some cases, it is convinient that a function can be invoked automatically by events. For example, if OSS can automatically trigger functions upon new image upload or update, it is not necessary to develop and maintain services to poll OSS events and invoke FC (Function Compute) functions. This document introduces how OSS events can be configured to automatically invoke FC functions.

    Executions of a FC function are event-driven, currenly FC supports OSS, Log Service and Tablestore events, more event source support will be added. Unlike previous examples, FC functions are invoked by OSS event notifications instead of CLI commands. Similar to the database trigger concept, a FC trigger describes a set of rules. When an event matches these rules, the event source triggers the configured function. For example, OSS automatically invokes a configured function responding to OSS event related to objects with prefix source/ .

    In this example, you can learn about:

    • How to create a function by using a code package stored in OSS.
    • How to use an OSS trigger to automatically process files stored in OSS by a FC function.

    This example demostrates when an image with a source/ prefix is stored in OSS, a FC function is automatically triggered. The function scales the image and finally stored the output to OSS with a processed/ prefix. For example, an image source/serverless.png is automatically processed and stored to processed/serverless.png upon arriving at OSS.

    This example modifies the function in Example 3. The input object is not an image instead an OSS event, and the processed image is not returned as a function execution result instead is stored in OSS.

    This example assumes that:

    • An fc-code bucket stores the function code. When creating a function, FC copies the code from this bucket.
    • An image-album bucket. The prefix of source images is source/ and that of processed images is processed/.

    Note:

    In your own setup, you must replace the example buckets with your own buckets. You can also use a single bucket to perform the operations in this example. You can create a bucket in the OSS console, and replace the OSS endpoint with your own.

    Update a Service Role (ServiceRole)

    In Example 3, we have created an fc-logsrole, and permitted FC to assume this role and write logs to LogHub. In this example, this role is further granted with the following permissions:

    • Read objects from fc-code bucket. Then FC is allowed to copy code packages from this bucket to create functions.
    • Read/Write objects from/to image-album bucket which allows functions can read images from this bucket and write execution results to it.
    1. mkrp fc-copy-code-all -a '"oss:GetObject"' -r '"*"'
    2. attach -p /ram/policies/fc-copy-code-all -r /ram/roles/fc-logs
    3. mkrp fc-oss-rw -a '["oss:GetObject", "oss:PutObject"]' -r '"*"'
    4. attach -p /ram/policies/fc-oss-rw -r /ram/roles/fc-logs

    Note: If any problem occurs, click the last row of this page to view the complete demo shell operation video.

    Create a Function

    Upload Code to OSS

    In the code directory, create an “oss_image_process.js” file and implement a “resize” function:

    1. 'use strict';
    2. console.log('Loading function ...');
    3. var oss = require('ali-oss').Wrapper;
    4. var fs = require('fs');
    5. var jimp = require("jimp");
    7. module.exports.resize = function (eventBuf, ctx, callback) {
    8.     console.log('Received event:', eventBuf.toString());
    9.     var event = JSON.parse(eventBuf);
    10.     var ossEvent = event.events[0];
    11.     // Required by OSS sdk: OSS region is prefixed with "oss-", e.g. "oss-cn-shanghai"
    12.     var ossRegion = "oss-" + ossEvent.region;
    14.     // Create oss client.
    15.     var client = new oss({
    16.         region: ossRegion,
    17.         // Credentials can be retrieved from context
    18.         accessKeyId: ctx.credentials.accessKeyId,
    19.         accessKeySecret: ctx.credentials.accessKeySecret,
    20.         stsToken: ctx.credentials.securityToken
    21.     });
    22.     // Bucket name is from OSS event
    23.     client.useBucket(ossEvent.oss.bucket.name);
    25.     // Processed images are saved to processed/
    26.     var newKey = ossEvent.oss.object.key.replace("source/", "processed/");
    27.     var tmpFile = "/tmp/processed.png";
    29.     // Get object
    30.     console.log('Getting object: ', ossEvent.oss.object.key)
    31.     client.get(ossEvent.oss.object.key).then(function (val) {
    33.         // Read object from buffer
    34.         jimp.read(val.content, function (err, image) {
    35.             if (err) {
    36.                 console.error("Failed to read image");
    37.                 callback(err);
    38.                 return;
    39.             }
    40.             // Resize the image and save it to a tmp file
    41.             image.resize(128, 128).write(tmpFile, function (err) {
    42.                 if (err) {
    43.                     console.error("Failed to write image locally");
    44.                     callback(err);
    45.                     return;
    46.                 }
    47.                 // Putting object back to OSS with the new key
    48.                 console.log('Putting object: ', newKey);
    49.                 client.put(newKey, tmpFile).then(function (val) {
    50.                     console.log('Put object:', val);
    51.                     callback(null, val);
    52.                     return;
    53.                 }).catch(function (err) {
    54.                     console.error('Failed to put object: %j', err);
    55.                     callback(err);
    56.                     return
    57.                 });
    58.             });
    59.         });
    60.     }).catch(function (err) {
    61.         console.error('Failed to get object: %j', err);
    62.         callback(err);
    63.         return
    64.     });
    65. };
    1. # -*- coding: utf-8 -*-
    2. import oss2, json
    3. from wand.image import Image
    5. def resize(event, context):
    6.     evt = json.loads(event)
    7.     creds = context.credentials
    8.     # Required by OSS sdk
    9.     auth=oss2.StsAuth(
    10.         creds.access_key_id,
    11.         creds.access_key_secret,
    12.         creds.security_token)
    14.     evt = evt['events'][0]
    15.     bucket_name = evt['oss']['bucket']['name']
    16.     endpoint = 'oss-' +  evt['region'] + '.aliyuncs.com'
    17.     bucket = oss2.Bucket(auth, endpoint, bucket_name)
    19.     objectName = evt['oss']['object']['key']
    20.     # Processed images are saved to processed/
    21.     newKey = objectName.replace("source/", "processed/")
    23.     remote_stream = bucket.get_object(objectName)
    24.     if not remote_stream:
    25.         return
    27.     remote_stream = remote_stream.read()
    28.     with Image(blob=remote_stream)  as img:
    29.         with img.clone() as i:
    30.             i.resize(128, 128)
    31.             new_blob = i.make_blob()
    32.             bucket.put_object(newKey, new_blob)

    Code packaging: As jimp has been installed in Example 3, skip its installation in this example. The Alibaba Cloud OSS JavaScript SDK, and the nodejs6 runtime of FC already contains this package. Therefore, it does not need to be installed either.

    Upload the code package to OSS. The following uses ossutil as an example to describe how to upload the package by the CLI. If you have not installed ossutil in your system, go to the OSS console and upload code.zip to the fc-code bucket. (Note: The endpoint of this fc-code bucket must be the same as your preset endpoint.)

    1. cd code
    2. ls
    3. # hello_world.js  image_process.js  node_modules  oss_image_process.js
    4. zip -r code.zip .
    5. ossutil -e http://oss-cn-shanghai.aliyuncs.com cp code.zip oss://fc-code/code.zip

    Note: The previous commands are normal Linux commands rather than the commands in the shell mode of the FC CLI.

    Create a function.

    1. mkf demo/resize_oss_image -t nodejs6 -h oss_image_process.resize -b fc-code -o code.zip

    -b specifies the bucket to store the code.

    -o specifies the object key of the code package in the bucket.

    Create a Trigger

    Before creating a trigger, create an invocation role. Then OSS assumes this role to invoke functions.

    1. mkir fc-invoke-function
    2. mkrp fc-invoke-all -a '"fc:InvokeFunction"' -r '"*"'
    3. attach -p /ram/policies/fc-invoke-all -r /ram/roles/fc-invoke-function

    Create a trigger.

    1. cd demo/resize_oss_image
    2. mkt resize_image -t oss -r acs:ram::12345:role/fc-invoke-function -s acs:oss:cn-shanghai:12345:image-album -c resize_image_config.yaml

    -t specifies the trigger type. Only OSS is supported currently.

    -r specifies the role that OSS plays to call functions. Change 12345 to your Alibaba Cloud account ID.

    -s specifies the bucket ARN of the event source. Change 12345 to your Alibaba Cloud account ID and image-album to your bucket name.

    -c specifies the configuration of the OSS trigger. The information of resize_image_config.yaml is as follows:

    1. triggerConfig:
    2.     events:
    3.         - oss:ObjectCreated:PostObject
    4.         - oss:ObjectCreated:PutObject
    5.     filter:
    6.         key:
    7.             prefix: source/

    Test a Trigger

    Upload an image to image-album/source. If ossutil is not installed, go to the OSS console to upload the image.

    1. ossutil -e http://oss-cn-shanghai.aliyuncs.com cp data/serverless.png oss://image-album/source/serverless.png
    2. ossutil -e http://oss-cn-shanghai.aliyuncs.com ls oss://image-album/processed/

    Note that in the previous trigger configuration, OSS is configured to only trigger the FC function upon objects uploaded with the source prefix and store execution results to objects with the processed prefix. In is important that these two prefixes are different. For example, if you store the output object with the source prefix, the output object triggers invocation to the function again, resulting in an infinite loop and causing unnecessary costs.

    To view the complete demo operation video of example 4, click oss_image_process.

    Free Trial Free Trial