After you create a trail and specify or create an Object Storage Service (OSS) bucket as the delivery destination for the trail in ActionTrail, events are continuously delivered to and stored in event log files in the OSS bucket. Then, you can use Data Lake Analytics (DLA) to query and analyze the events in a visualized manner.
DLA adopts a serverless architecture and is an interactive query and analytics service. DLA allows you to use standard SQL statements to query and analyze log data in different formats and from different sources. For more information, see What is DLA?.
The following steps and figure show how to use DLA to query and analyze the events that are delivered to OSS:
- You create a trail and configure it to deliver events to an OSS bucket by using ActionTrail.
- You synchronize the delivered events from the OSS bucket to DLA.
- OSS stores the log data of multiple events as an array in the OSS bucket. DLA splits the log data by event and stores the log data in the JSON format. Then, DLA converts the log data for each event to a structured table. This simplifies the process of resolving log data in OSS buckets and allows standard SQL queries and analytics of data.
- Create a schema in DLA.
- Log on to the DLA console.
- Select the region where your OSS bucket resides from the drop-down list in the upper-left corner.
- In the left-side navigation pane, choose .
- On the Data into the lake page, click More and then click Go To The Wizard in the ActionTrail Log Cleaning section.
- In the ActionTrail Log Cleaning wizard, set the parameters as required.
Parameter Description ActionTrail File Root The storage path where the logs of events that ActionTrail delivers to OSS are saved. The path must end with
AliyunLogs/Actiontrail. You can set this parameter in one of the following ways:
- Select Location: You specify a custom path to store the logs of events delivered by ActionTrail.
- Auto Discovery: DLA automatically specifies a default path to store the logs of events delivered by ActionTrail.
Schema Name The name of the schema. This parameter specifies the name of the DLA database that is mapped to the OSS bucket. Data Storage Location After Cleaning The path of the OSS folder to which the cleansed log data is written.
- If you do no select Custom, DLA automatically specifies a default path for storing the cleansed log data.
- If you select Custom, you can specify a custom path.
Data Cleaning Time The time at which DLA cleanses OSS log data every day.
By default, data cleansing is enabled at 00:30. To prevent your business from being affected during data cleansing, we recommend that you set this parameter to a time within off-peak hours based on your business requirements.
- Click Create.
- Synchronize events from the OSS bucket to DLA.
- In the next step of the ActionTrail Log Cleaning wizard, click Sync Now.
- In the left-side navigation pane, choose Library table details in the Actions column. . On the Metadata management page, find the created schema in the schema list and click
- On the Table tab of the Metadata management page, view the information about the synchronization.
For more information about schemas, see Schemas.
- Use standard SQL statements to query and analyze events.
- In the left-side navigation pane, choose .
- Find the database based on which you want to analyze events and double-click the database name.
- Enter the query statement in the SQL editor and click Sync Execute(F8). Then, DLA returns the execution result.
Query events for a user identified by a specific AccessKey ID
- Query statement:
select * from `action_trail` where `user_identity_access_key_id` = 'User AccessKey ID' limit 20;
- Results: DLA returns the first 20 events that occurred within the user account identified by the specified AccessKey ID.
Query ECS-related events for a user identified by a specific AccessKey ID
- Query statement:
select * from `action_trail` where `user_identity_access_key_id` = 'User AccessKey ID' AND `service_name` = 'Ecs' limit 20;
- Results: DLA returns the first 20 events related to Elastic Compute Service (ECS) that occurred within the user account identified by the specified AccessKey ID.
The following table describes the key fields of a schema.
|event_id||String||Yes||F23A3DD5-7842-4EF9-9DA1-3776396A****||The ID of the event. ActionTrail generates a globally unique identifier (GUID) for each event.|
|event_name||String||Yes||CreateNetworkInterface||The name of the event.
|event_source||String||Yes||ecs.aliyuncs.com||The URL of the service that processed the event.|
|event_time||String||Yes||2020-01-09T12:12:14Z||The time when the event occurred, in UTC.|
|event_type||String||Yes||ApiCall||The type of the action that was recorded in the event log. Valid values:
|request_parameters||Dictionary||No||N/A||The parameters specified in the API request.|
|response_elements||Dictionary||No||N/A||The response returned for the API request.|
|service_name||String||Yes||Ecs||The name of the Alibaba Cloud service with which the event is associated.|
|source_ip_address||String||Yes||11.32.XX.XX||The IP address from which the event occurred.
Note If the API operation involved was called by a user in the Alibaba Cloud Management Console, this field is set to the IP address of the user, rather than the IP address of the web server of the Alibaba Cloud Management Console.
|user_agent||String||Yes||Apache-HttpClient/4.5.7 (Java/1.8.0_152)||The user agent that sent the API request. Examples:
|user_identity_type||String||Yes||ram-user||The type of the entity that initiated the event. Valid values:
|user_identity_principal_id||String||Yes||28815334868278****||The ID of the requester.
|user_identity_account_id||String||Yes||112233445566****||The ID of the Alibaba Cloud account that owns the requester.|
|user_identity_accessKey_id||String||No||55nCtAwmPLkk****||The AccessKey ID that is used by the requester. If the requester sent the API request by using an SDK, this field is recorded. If the requester sent the API request by using the Alibaba Cloud Management Console, this field is not recorded.|
|user_name||String||No||Alice||The name of the requester. If the user_identity_type field is set to ram-user, this field is set to the name of the RAM user involved. If the user_identity_type field is set to assumed-role, this field is set to a string in the RoleName:RoleSessionName format.|