Object Storage Service:Process OSS file changes in real time using event notifications

Prerequisites

Ensure that Simple Message Queue (SMQ) (formerly MNS) is activated. To activate SMQ, visit the SMQ product page.

Notes

• Event notifications involve calls between OSS and SMQ. When an operation in OSS triggers a rule, OSS calls SMQ to send a message. This call from OSS to SMQ can fail. To check whether the call was successful, inspect the `x-oss-event-status` value in the response header. This value is Base64-encoded. If the decoded value is `{"Result": "Ok"}`, the call to SMQ was successful. Otherwise, the call failed.

• The event notification feature incurs fees for SMQ. For more information about billing, see Billing overview.

• For a list of regions that support event notifications for buckets, see Supported regions.

• You can configure a maximum of 10 event notification rules per region. If your business requires more rules, contact Technical Support.

• TS and M3U8 files generated by RTMP stream ingest do not trigger event notification rules. For more information about RTMP stream ingest, see LiveChannel operations.

How it works

After you create an event notification rule, when a request to OSS triggers the rule, SMQ sends a notification. The notification, which contains information about the operation, is sent to the HTTP server or SMQ queue that you configured. The following diagram shows this process:

Event types

Message notifications

{"events": [
      {
        "eventName": "",  // The event notification type.
        "eventSource": "", // The source of the event notification. The value is fixed to "acs:oss".
        "eventTime": "", // The event time in ISO 8601 format.
        "eventVersion": "", // The event notification version. The current version is "1.0".
        "oss": {
            "bucket": {
                "arn": "", // The unique identifier of the bucket. The format is "acs:oss:region:uid:bucketname".
                "name": "", // The name of the destination bucket.
                "ownerIdentity": "" // The owner of the bucket.
            }, 
            "object": {
                "deltaSize": "", // The change in the object size. For example, if a file is added, this value is the file size. If a file is overwritten, this value is the difference between the new file size and the old file size, so it can be negative.
                "eTag": "", // The ETag of the object.
                "key": "", // The object name.
                "position": "", // This parameter applies only to the ObjectCreated:AppendObject event. It indicates the position where the append operation starts for this request. The position for the first AppendObject request starts from byte 0.
                "readFrom": "", // This parameter applies only to the ObjectDownloaded:GetObject event. It indicates the position where the file read starts. For non-range requests, this value is 0. For range requests, this value is the starting byte of the request.
                "readTo": "", // This parameter applies only to the ObjectDownloaded:GetObject event. It indicates the position where the file read ends. For non-range requests, this value is the file size. For range requests, this value is the ending byte of the range request plus 1.
                "size": "" // The object size.
                }, 
        "ossSchemaVersion": "", // The version of this schema field. The current version is "1.0".
        "ruleId": "GetObject", // The ID of the rule that this event matches.
        "region": "", // The region where the bucket is located.
        "requestParameters": {
            "sourceIPAddress": "" // The source IP address of the request.
            }, 
        "responseElements": {
            "requestId": "" // The request ID.
            }, 
        "userIdentity": {
            "principalId": "" // The UID of the requester.
            }, 
        "xVars": {  // Custom parameters in the OSS Callback feature.
            "x:callback-var1":"value1",
            "x:vallback-var2":"value2"
            }
        }        
     }
  ]
}

{"events": [
      {
        "eventName": "ObjectDownloaded:GetObject",
        "eventSource": "acs:oss",
        "eventTime": "2016-07-01T11:17:30.000Z",
        "eventVersion": "1.0",
        "oss": {
            "bucket": {
                "arn": "acs:oss:cn-shenzhen:114895646818****:event-notification-test-shenzhen",
                "name": "event-notification-test-shenzhen",
                "ownerIdentity": "114895646818****"},
            "object": {
                "deltaSize": 0,
                "eTag": "0CC175B9C0F1B6468E1199E269772661",
                "key": "test",
                "readFrom": 0,
                "readTo": 1,
                "size": 1
            },
        "ossSchemaVersion": "1.0",
        "ruleId": "GetObjectRule",
        "region": "cn-shenzhen",
        "requestParameters": {
            "sourceIPAddress": "198.51.100.1"
            },
        "responseElements": {
            "requestId": "5FF16B65F05BC932307A3C3C"
            },
        "userIdentity": {
            "principalId": "114895646818****"
            },
        "xVars": {
            "x:callback-var1":"value1",
            "x:vallback-var2":"value2"
            }
        }        
     }
  ]
}

Procedure

1. Log on to the OSS console.

2. In the left-side navigation pane, click Buckets. On the Buckets page, find and click the desired bucket.

3. In the navigation pane on the left, choose Data Processing > Event Notification.

4. On the Event Notification page, click Create Rule.

5. In the Create Rule panel, configure the following parameters:

   Parameter	Description
   Rule Name	The name of the event notification rule. Rule names must be unique within the same region for the same account. The name must start with a letter and can contain uppercase letters, lowercase letters, digits, and hyphens (-). The length cannot exceed 85 characters.
   Event Type	The event type for the destination object. For example, if you want to receive event notifications when a destination object is created or overwritten by a copy operation, set the event type to `CopyObject`. You can configure one event notification rule for a destination object and specify multiple event types in the rule. You can also configure multiple event notification rules for a destination object. Note the following when you configure multiple rules: If multiple rules apply to the same destination object, the event types cannot be the same. For example, if Rule A is configured with the `CopyObject` event for the images prefix, Rule B cannot include the `CopyObject` event if it applies to any object with the images prefix. If multiple rules apply to different destination objects, the event types can be the same or different. For example, if Rule A is configured with the `PutObject` event for objects with the images prefix and .png suffix, and Rule B applies to objects with the log prefix and .jpg suffix, the event type for Rule B can be `PutObject` or `DeleteObject`. Important When you delete an object from a versioning-enabled bucket without specifying a version ID, a `DeleteObject` or `DeleteObjects` event notification is not triggered. This is because the default delete behavior does not delete any object version. Instead, it converts the current version of the object into a previous version and adds a delete marker. For more information about the object operations corresponding to event types, see Event types.
   Object Matching	The destination objects for which to trigger event notifications. Match destination objects by Full Name To match a destination object named exampleobject.txt in the root directory of the bucket, enter exampleobject.txt. To match a destination object named myphoto.jpg in the destdir directory of the bucket, enter destdir/myphoto.jpg. Match destination objects by Prefix/Suffix To match all objects in the bucket, leave the prefix and suffix empty. To match all objects in the examplefolder directory of the bucket, set the prefix to examplefolder/ and leave the suffix empty. To match all objects with the .jpg extension in the bucket, leave the prefix empty and set the suffix to .jpg. To match all objects with the .mp3 extension in the examplefolder directory of the bucket, set the prefix to examplefolder/ and the suffix to .mp3. Click the Add Condition button to create up to five Resource Description object matching rules.
   Subscription Endpoint	The endpoint that receives events. Subscription types such as HTTP and Queue are supported.

6. Click OK.

   After you complete the preceding steps, the event notification rule takes effect in about 10 minutes.

FAQ

Why is an event notification not triggered when I delete a file?

References

• You can specify objects to monitor in an event notification rule. When a specified event occurs for these objects, you receive a notification through the endpoint configured in SMQ. For more information, see Tutorial: Use SMQ to implement OSS event notifications.

• If you cannot retrieve the message body through the HttpEndpoint interface, it may be because you are using the POST method. For a solution, see What do I do if I cannot receive the message body when I use PHP to write an HttpEndpoint interface in Simple Message Queue (formerly MNS)?.

• You can also use the upload callback feature of OSS to implement upload callbacks. For more information, see Upload callback.