After you create a trail and specify or create an Object Storage Service (OSS) bucket as the delivery destination for the trail in ActionTrail, events are continuously delivered to and stored in an event log file in the OSS bucket. Then, you can use Data Lake Analytics (DLA) to query and analyze the events in a visualized manner.

Prerequisites

Background information

DLA adopts a serverless architecture and is an interactive query and analytics service. DLA allows you to use standard SQL statements to query and analyze log data in different formats and from different sources. For more information, see What is DLA?.

The following steps and figure show how to use DLA to query and analyze the events that are delivered to OSS:

  1. You create a trail and configure it to deliver events to an OSS bucket by using ActionTrail.
  2. You synchronize the delivered events from the OSS bucket to DLA.
  3. DLA splits the event logs stored in arrays in the OSS bucket into data entries and translates the event logs stored in JSON format in the OSS bucket to structured tables. This simplifies the process of resolving log data in OSS buckets and enables standard SQL queries and analytics of the data.
Arcitecture

Procedure

  1. Create a schema in DLA.
    1. Log on to the DLA console.
    2. Select the region where your OSS bucket resides from the drop-down list in the upper-left corner.
    3. In the left-side navigation pane, choose Data Lake Management > Data into the lake.
    4. On the Data into the lake page, click Go to The Wizard in the ActionTrail Log Cleaning section.
    5. In the ActionTrail Log Cleaning wizard, set the parameters as required.
      Parameter Description
      ActionTrail File Root The storage path where the logs of events that ActionTrail delivers to OSS are saved. The path must end with AliyunLogs/Actiontrail. You can set this parameter in one of the following ways:
      • Select Location: You specify a custom path to store the logs of events delivered by ActionTrail.
      • Auto Discovery: DLA automatically specifies a default path to store the logs of events delivered by ActionTrail.
      Schema Name The name of the schema. This parameter specifies the name of the DLA database that is mapped to the OSS bucket.
      Data Storage Location After Cleaning The path of the OSS folder to which the cleansed log data is written.
      • If you do no select Custom, DLA automatically specifies a default path for storing the cleansed log data.
      • If you select Custom, you can specify a custom path.
      Data Cleaning Time The time at which DLA cleanses OSS log data every day.

      By default, data cleansing is enabled at 00:30. To prevent your business from being affected during data cleansing, we recommend that you set this parameter to a time within off-peak hours based on your business requirements.

    6. Click Create.
  2. Synchronize events from the OSS bucket to DLA.
    1. In the next step of the ActionTrail Log Cleaning wizard, click Sync Now.
    2. Click Return metadata management. On the Metadata management page, find the created schema in the schema list and click Library table details in the Actions column.
    3. On the Table tab of the Metadata management page, view the information about the synchronization.
      For more information about schemas, see Schemas.
  3. Use standard SQL statements to query and analyze events.
    1. In the left-side navigation pane, choose Serverless Presto > Execute.
    2. Find the database based on which you want to analyze events and double-click the database name.
    3. Enter the query statement in the SQL editor and click Sync Execute(F8). Then, DLA returns the execution result.

Examples

Query events for a user identified by a specific AccessKey ID

  • Query statement: select * from `action_trail` where `user_identity_access_key_id` = 'User AccessKey ID' limit 20;
  • Results: DLA returns the first 20 events that occurred within the user account identified by the specified AccessKey ID.

Query ECS-related events for a user identified by a specific AccessKey ID

  • Query statement: select * from `action_trail` where `user_identity_access_key_id` = 'User AccessKey ID' AND `service_name` = 'Ecs' limit 20;
  • Results: DLA returns the first 20 events related to Elastic Compute Service (ECS) that occurred within the user account identified by the specified AccessKey ID.

Schemas

The following table describes the key fields of a schema.

Field Type Required Example Description
event_id String Yes F23A3DD5-7842-4EF9-9DA1-3776396A**** The ID of the event. ActionTrail generates a globally unique identifier (GUID) for each event.
event_name String Yes CreateNetworkInterface The name of the event.
  • This field is set to the name of the API operation that was called if the eventType field is set to ApiCall.
  • This field is set to a string that indicates the action of the event if the eventType field is not set to ApiCall.
event_source String Yes ecs.aliyuncs.com The URL of the service that processed the event.
event_time String Yes 2020-01-09T12:12:14Z The time when the event occurred, in UTC.
event_type String Yes ApiCall The type of the action that was recorded in the event log. Valid values:
  • ApiCall: indicates that an API operation was called. This is the most typical event type. The userAgent field indicates whether the event was triggered by using the Alibaba Cloud Management Console or an SDK.
  • ConsoleOperation (ConsoleCall): indicates that a specific action was performed in the Alibaba Cloud Management Console. The name of an event of this type can be the name of the API operation that was called or a string that indicates the action of the event.
  • AliyunServiceEvent: indicates that Alibaba Cloud performed a specific action on resources that you own, such as releasing a subscription instance upon expiration.
  • PasswordReset: indicates that your password was reset.
  • ConsoleSignin: indicates a logon to a console.
  • ConsoleSignout: indicates a logoff from a console.
request_parameters Dictionary No N/A The parameters specified in the API request.
response_elements Dictionary No N/A The response returned for the API request.
service_name String Yes Ecs The name of the Alibaba Cloud service with which the event is associated.
source_ip_address String Yes 11.32.XX.XX The IP address from which the event occurred.
Note If the API operation involved was called by a user in a console, this field is set to the IP address of the user, rather than the IP address of the web server of the console.
user_agent String Yes Apache-HttpClient/4.5.7 (Java/1.8.0_152) The user agent that sends the API request. Valid values:
  • AlibabaCloud (Linux 3.10.0-693.2.2.el7.x86_64;x86_64) Python/2.7.5 Core/2.13.16 python-requests/2.18.3
  • Apache-HttpClient/4.5.7 (Java/1.8.0_152)
user_identity_type String Yes ram-user The type of the entity that initiated the event. Valid values:
  • root-account: indicates an Alibaba Cloud account.
  • ram-user: indicates a RAM user.
  • assumed-role: indicates a RAM role.
  • system: indicates an Alibaba Cloud service.
user_identity_principal_id String Yes 28815334868278**** The ID of the requester.
  • This field is set to the ID of the Alibaba Cloud account involved if the user_identity_type field is set to root-account.
  • This field is set to the ID of the RAM user involved if the user_identity_type field is set to ram-user.
  • This field is set to a string in the RoleID:RoleSessionName format if the user_identity_type field is set to assumed-role.
user_identity_account_id String Yes 112233445566**** The ID of the Alibaba Cloud account that owns the requester.
user_identity_accessKey_id String No 55nCtAwmPLkk**** The AccessKey ID that is used by the requester. If the requester sent the API request by using an SDK, this field is recorded. If the requester logged on to the Alibaba Cloud Management Console, this field is not recorded.
user_name String No Alice The name of the requester. This field is set to the name of the RAM user involved if the user_identity_type field is set to ram-user. This field is set to a string in the RoleName:RoleSessionName format if the user_identity_type field is set to assumed-role.