After you create a trail and specify or create an Object Storage Service (OSS) bucket as the delivery destination for the trail in ActionTrail, events are continuously delivered to and stored in event log files in the OSS bucket. Then, you can use Data Lake Analytics (DLA) to query and analyze the events in a visualized manner.

Prerequisites

Background information

DLA adopts a serverless architecture and is an interactive query and analytics service. DLA allows you to use standard SQL statements to query and analyze log data in different formats and from different sources. For more information, see What is DLA?.

The following steps and figure show how to use DLA to query and analyze the events that are delivered to OSS:

  1. You create a trail and configure it to deliver events to an OSS bucket by using ActionTrail.
  2. You synchronize the delivered events from the OSS bucket to DLA.
  3. OSS stores the log data of multiple events as an array in the OSS bucket. DLA splits the log data by event and stores the log data in the JSON format. Then, DLA converts the log data for each event to a structured table. This simplifies the process of resolving log data in OSS buckets and allows standard SQL queries and analytics of data.
Arcitecture
Note You cannot use DLA to analyze events stored in Log Service. To analyze these events, you can use the analysis feature of Log Service.

Procedure

  1. Create a schema in DLA.
    1. Log on to the DLA console.
    2. Select the region where your OSS bucket resides from the drop-down list in the upper-left corner.
    3. In the left-side navigation pane, choose Data Lake Management > Data into the lake.
    4. On the Data into the lake page, click More and then click Go To The Wizard in the ActionTrail Log Cleaning section.
    5. In the ActionTrail Log Cleaning wizard, set the parameters as required.
      Parameter Description
      ActionTrail File Root The storage path where the logs of events that ActionTrail delivers to OSS are saved. The path must end with AliyunLogs/Actiontrail. You can set this parameter in one of the following ways:
      • Select Location: You specify a custom path to store the logs of events delivered by ActionTrail.
      • Auto Discovery: DLA automatically specifies a default path to store the logs of events delivered by ActionTrail.
      Schema Name The name of the schema. This parameter specifies the name of the DLA database that is mapped to the OSS bucket.
      Data Storage Location After Cleaning The path of the OSS folder to which the cleansed log data is written.
      • If you do no select Custom, DLA automatically specifies a default path for storing the cleansed log data.
      • If you select Custom, you can specify a custom path.
      Data Cleaning Time The time at which DLA cleanses OSS log data every day.

      By default, data cleansing is enabled at 00:30. To prevent your business from being affected during data cleansing, we recommend that you set this parameter to a time within off-peak hours based on your business requirements.

    6. Click Create.
  2. Synchronize events from the OSS bucket to DLA.
    1. In the next step of the ActionTrail Log Cleaning wizard, click Sync Now.
    2. In the left-side navigation pane, choose Data Lake Management > Metadata management. On the Metadata management page, find the created schema in the schema list and click Library table details in the Actions column.
    3. On the Table tab of the Metadata management page, view the information about the synchronization.
      For more information about schemas, see Schemas.
  3. Use standard SQL statements to query and analyze events.
    1. In the left-side navigation pane, choose Serverless Presto > Execute.
    2. Find the database based on which you want to analyze events and double-click the database name.
    3. Enter the query statement in the SQL editor and click Sync Execute(F8). Then, DLA returns the execution result.

Examples

Query events for a user identified by a specific AccessKey ID

  • Query statement: select * from `action_trail` where `user_identity_access_key_id` = 'User AccessKey ID' limit 20;
  • Results: DLA returns the first 20 events that occurred within the user account identified by the specified AccessKey ID.

Query ECS-related events for a user identified by a specific AccessKey ID

  • Query statement: select * from `action_trail` where `user_identity_access_key_id` = 'User AccessKey ID' AND `service_name` = 'Ecs' limit 20;
  • Results: DLA returns the first 20 events related to Elastic Compute Service (ECS) that occurred within the user account identified by the specified AccessKey ID.

Schemas

The following table describes the key fields of a schema.

Field Type Required Example Description
event_id String Yes F23A3DD5-7842-4EF9-9DA1-3776396A**** The ID of the event. ActionTrail generates a globally unique identifier (GUID) for each event.
event_name String Yes CreateNetworkInterface The name of the event.
  • If the event_type field is set to ApiCall, this field is set to the name of the API operation that was called.
  • If the event_type field is not set to ApiCall, this field is set to a string that indicates the action of the event.
event_source String Yes ecs.aliyuncs.com The URL of the service that processed the event.
event_time String Yes 2020-01-09T12:12:14Z The time when the event occurred, in UTC.
event_type String Yes ApiCall The type of the action that was recorded in the event log. Valid values:
  • ApiCall: indicates that an API operation was called. This is the most typical event type. The userAgent field indicates whether the event was triggered by using the Alibaba Cloud Management Console or an SDK.
  • ConsoleOperation (ConsoleCall): indicates that a specific action was performed in the Alibaba Cloud Management Console. The name of an event of this type can be the name of the API operation that was called or a string that indicates the action of the event.
  • AliyunServiceEvent: indicates that Alibaba Cloud performed a specific action on the resources that you own, such as releasing a subscription instance upon expiration.
  • PasswordReset: indicates that your password was reset.
  • ConsoleSignin: indicates a logon to the Alibaba Cloud Management Console.
  • ConsoleSignout: indicates a logoff from the Alibaba Cloud Management Console.
request_parameters Dictionary No N/A The parameters specified in the API request.
response_elements Dictionary No N/A The response returned for the API request.
service_name String Yes Ecs The name of the Alibaba Cloud service with which the event is associated.
source_ip_address String Yes 11.32.XX.XX The IP address from which the event occurred.
Note If the API operation involved was called by a user in the Alibaba Cloud Management Console, this field is set to the IP address of the user, rather than the IP address of the web server of the Alibaba Cloud Management Console.
user_agent String Yes Apache-HttpClient/4.5.7 (Java/1.8.0_152) The user agent that sent the API request. Examples:
  • AlibabaCloud (Linux 3.10.0-693.2.2.el7.x86_64;x86_64) Python/2.7.5 Core/2.13.16 python-requests/2.18.3
  • Apache-HttpClient/4.5.7 (Java/1.8.0_152)
user_identity_type String Yes ram-user The type of the entity that initiated the event. Valid values:
  • root-account: indicates an Alibaba Cloud account.
  • ram-user: indicates a RAM user.
  • assumed-role: indicates a RAM role.
  • system: indicates an Alibaba Cloud service.
user_identity_principal_id String Yes 28815334868278**** The ID of the requester.
  • If the user_identity_type field is set to root-account, this field is set to the ID of the Alibaba Cloud account involved.
  • If the user_identity_type field is set to ram-user, this field is set to the ID of the RAM user involved.
  • If the user_identity_type field is set to assumed-role, this field is set to a string in the RoleID:RoleSessionName format.
user_identity_account_id String Yes 112233445566**** The ID of the Alibaba Cloud account that owns the requester.
user_identity_accessKey_id String No 55nCtAwmPLkk**** The AccessKey ID that is used by the requester. If the requester sent the API request by using an SDK, this field is recorded. If the requester sent the API request by using the Alibaba Cloud Management Console, this field is not recorded.
user_name String No Alice The name of the requester. If the user_identity_type field is set to ram-user, this field is set to the name of the RAM user involved. If the user_identity_type field is set to assumed-role, this field is set to a string in the RoleName:RoleSessionName format.