After you create a trail and specify or create an Object Storage Service (OSS) bucket as the delivery destination by using ActionTrail, events will be continuously delivered to and stored in an event log file in the OSS bucket. Then, you can use Data Lake Analytics (DLA) to query and analyze the events in visual mode.

Prerequisites

Background information

Adopting a serverless architecture, DLA is an interactive query and analytics service that allows you to use standard SQL statements for querying and analyzing log data in different formats and from different sources. For more information about DLA, see What is Data Lake Analytics?.

The following steps and figure show how to use DLA to query and analyze events delivered to OSS:

  1. You create a trail and configure it to deliver events to an OSS bucket by using ActionTrail.
  2. You synchronize events from the OSS bucket to DLA.
  3. DLA splits event logs stored in arrays in the OSS bucket into data entries and translates event logs stored in JSON format in the OSS bucket to structured tables. This simplifies the process of resolving log data in OSS buckets and enables standard SQL queries and analytics for the data.
Arcitecture

Procedure

  1. Create a schema in DLA.
    1. Log on to the DLA console.
    2. Select the region where your OSS bucket is located from the drop-down list in the upper-left corner.
    3. In the left-side navigation pane, choose Data Lake Management > Data into the lake.
    4. On the Data into the lake page, click Go to The Wizard in the ActionTrail Log Cleaning section.
    5. In the ActionTrail Log Cleaning wizard, set the parameters as required.
      Parameter Description
      ActionTrail File Root The path of the OSS folder to which ActionTrail delivers events. The default path specified by DLA ends with AliyunLogs/Actiontrail. You can set this parameter in one of the following ways:
      • Select Location: You specify a custom path to store events delivered by ActionTrail.
      • Auto Discovery: DLA automatically specifies a default path to store events delivered by ActionTrail.
      Schema Name The name of the mapped database of the OSS bucket in DLA.
      Data Storage Location After Cleaning The path of the OSS folder where cleansed log data is stored.
      • If you do no select the Custom check box, DLA automatically specifies a default path for storing cleansed log data.
      • If you select the Custom check box, you can specify a custom path.
      Data Cleaning Time The time when DLA starts to cleanse log data each day.

      The default value is 00:30. We recommend that you specify a time within the off-peak hours of your business to minimize the impact on your business.

    6. Click Create.
  2. Synchronize events from the OSS bucket to DLA.
    1. In the next step of the ActionTrail Log Cleaning wizard, click Sync Now.
    2. Click Return metadata management. On the Metadata management page, find the created schema and click Library table details in the Actions column.
    3. On the Table tab of the Metadata management page, view the information about the synchronization.
      To modify the configuration of a schema, perform the following steps: On the Data into the lake page, find the target schema and click Edit in the Operation column. In the Configuration step, reset the parameters as required. For more information about schemas, see Schemas.
  3. Use standard SQL statements to query and analyze events.
    1. In the left-side navigation pane, choose Serverless SQL > Execute.
    2. Find the target database that you want to analyze. If the database to be analyzed is not the current one, double click the target database.
    3. Enter the query statement in the SQL editor and click Sync Execute(F8). Then, DLA returns the execution result.
      SQL
      Note You can use any valid SQL query statement to query events in DLA.

Examples

Query events for a user identified by a specific AccessKey ID

  • Query statement: select * from `action_trail` where `user_identity_access_key_id` = 'target AccessKey ID' limit 20;
  • Result: DLA returns the first 20 events that has occurred under the user identified by the target AccessKey ID.

Query events related to ECS for a user identified by a specific AccessKey ID

  • Query statement: select * from `action_trail` where `user_identity_access_key_id` = 'target AccessKey ID' AND `service_name` = 'Ecs' limit 20;
  • Result: DLA returns the first 20 events related to Elastic Compute Service (ECS) that has occurred under the user identified by the target AccessKey ID.

Schemas

The following table describes the key fields of a schema.

Parameter Type Required Example Description
event_id String Yes F23A3DD5-7842-4EF9-9DA1-3776396A**** The ID of the event. ActionTrail generates a globally unique identifier (GUID) for each delivered event.
event_name String Yes CreateNetworkInterface The name of the event.
  • This field is set to the name of the API operation that was called if eventType is set to ApiCall.
  • This field is set to a string that indicates the action of the event if eventType is not set to ApiCall.
event_source String Yes ecs.aliyuncs.com The URL of the service that processed the event.
event_time String Yes 2020-01-09T12:12:14Z The time when the event occurred, in UTC.
event_type String Yes ApiCall The type of the event that generated the event log. Valid values:
  • ApiCall: indicates that an API was called. This is the most common event type. The userAgent field indicates whether the event was triggered by using the Alibaba Cloud Management Console or an SDK.
  • ConsoleOperation (ConsoleCall): indicates that a specific action was performed in the Alibaba Cloud Management Console. The name of this type of event can be the name of the API operation that was called or a string that indicates the action of the event.
  • AliyunServiceEvent: indicates that Alibaba Cloud performed a specific action on resources that you own, for example, releasing a subscription instance upon expiration.
  • PasswordReset: indicates that the password of your Alibaba Cloud account or a RAM user was reset.
  • ConsoleSignin: indicates a logon by using your Alibaba Cloud account or as a RAM user.
  • ConsoleSignout: indicates a logoff by using your Alibaba Cloud account or as a RAM user.
request_parameters Dictionary No N/A The request parameters that was sent with the API request.
response_elements Dictionary No N/A The response data that was returned.
service_name String Yes Ecs The name of the Alibaba Cloud service to which the request was sent.
source_ip_address String Yes 11.XX.XX.232 The IP address from which the request was sent.
Note If the API operation was called by a user in the console, this field is set to the user's IP address, rather than the IP address of the web server of the console.
user_agent String Yes Apache-HttpClient/4.5.7 (Java/1.8.0_152) The agent through which the API request was sent. Valid values:
  • AlibabaCloud (Linux 3.10.0-693.2.2.el7.x86_64;x86_64) Python/2.7.5 Core/2.13.16 python-requests/2.18.3
  • Apache-HttpClient/4.5.7 (Java/1.8.0_152)
user_identity_type String Yes ram-user The type of the identity. Valid values:
  • root-account: indicates an Alibaba Cloud account.
  • ram-user: indicates a RAM user.
  • assumed-role: indicates a RAM role.
  • system: indicates an Alibaba Cloud service.
user_identity_principal_id String Yes 28815334868278**** The ID of the requester.
  • This field is set to the ID of the Alibaba Cloud account if type is set to root-account.
  • This field is set to the ID of the RAM user if type is set to ram-user.
  • This field is set to a string in the RoleID:RoleSessionName format if type is set to assumed-role.
user_identity_account_id String Yes 112233445566**** The ID of the Alibaba Cloud account that owns the requester.
user_identity_accessKey_id String No 55nCtAwmPLkk**** The AccessKey ID used to make the API request. This field is required if the API request was made through the SDK, and is not required when the API request was made through the console.
user_name String No B** The name of the requester. This field is set to the name of the RAM user if type is set to ram-user. This field is set to a string in the RoleName:RoleSessionName format if type is set to assumed-role.