After you create a trail and specify or create an Object Storage Service (OSS) bucket as the delivery destination by using ActionTrail, events will be continuously delivered to and stored in an event log file in the OSS bucket. Then, you can use Data Lake Analytics (DLA) to query and analyze the events in visual mode.
Adopting a serverless architecture, DLA is an interactive query and analytics service that allows you to use standard SQL statements for querying and analyzing log data in different formats and from different sources. For more information about DLA, see What is Data Lake Analytics?.
The following steps and figure show how to use DLA to query and analyze events delivered to OSS:
- You create a trail and configure it to deliver events to an OSS bucket by using ActionTrail.
- You synchronize events from the OSS bucket to DLA.
- DLA splits event logs stored in arrays in the OSS bucket into data entries and translates event logs stored in JSON format in the OSS bucket to structured tables. This simplifies the process of resolving log data in OSS buckets and enables standard SQL queries and analytics for the data.
- Create a schema in DLA.
- Log on to the DLA console.
- Select the region where your OSS bucket is located from the drop-down list in the upper-left corner.
- In the left-side navigation pane, choose .
- On the Data into the lake page, click Go to The Wizard in the ActionTrail Log Cleaning section.
- In the ActionTrail Log Cleaning wizard, set the parameters as required.
Parameter Description ActionTrail File Root The path of the OSS folder to which ActionTrail delivers events. The default path specified by DLA ends with
AliyunLogs/Actiontrail. You can set this parameter in one of the following ways:
- Select Location: You specify a custom path to store events delivered by ActionTrail.
- Auto Discovery: DLA automatically specifies a default path to store events delivered by ActionTrail.
Schema Name The name of the mapped database of the OSS bucket in DLA. Data Storage Location After Cleaning The path of the OSS folder where cleansed log data is stored.
- If you do no select the Custom check box, DLA automatically specifies a default path for storing cleansed log data.
- If you select the Custom check box, you can specify a custom path.
Data Cleaning Time The time when DLA starts to cleanse log data each day.
The default value is 00:30. We recommend that you specify a time within the off-peak hours of your business to minimize the impact on your business.
- Click Create.
- Synchronize events from the OSS bucket to DLA.
- In the next step of the ActionTrail Log Cleaning wizard, click Sync Now.
- Click Return metadata management. On the Metadata management page, find the created schema and click Library table details in the Actions column.
- On the Table tab of the Metadata management page, view the information about the synchronization.To modify the configuration of a schema, perform the following steps: On the Data into the lake page, find the target schema and click Edit in the Operation column. In the Configuration step, reset the parameters as required. For more information about schemas, see Schemas.
- Use standard SQL statements to query and analyze events.
- In the left-side navigation pane, choose .
- Find the target database that you want to analyze. If the database to be analyzed is not the current one, double click the target database.
- Enter the query statement in the SQL editor and click Sync Execute(F8). Then, DLA returns the execution result.Note You can use any valid SQL query statement to query events in DLA.
Query events for a user identified by a specific AccessKey ID
- Query statement:
select * from `action_trail` where `user_identity_access_key_id` = 'target AccessKey ID' limit 20;
- Result: DLA returns the first 20 events that has occurred under the user identified by the target AccessKey ID.
Query events related to ECS for a user identified by a specific AccessKey ID
- Query statement:
select * from `action_trail` where `user_identity_access_key_id` = 'target AccessKey ID' AND `service_name` = 'Ecs' limit 20;
- Result: DLA returns the first 20 events related to Elastic Compute Service (ECS) that has occurred under the user identified by the target AccessKey ID.
The following table describes the key fields of a schema.
|event_id||String||Yes||F23A3DD5-7842-4EF9-9DA1-3776396A****||The ID of the event. ActionTrail generates a globally unique identifier (GUID) for each delivered event.|
|event_name||String||Yes||CreateNetworkInterface||The name of the event.
|event_source||String||Yes||ecs.aliyuncs.com||The URL of the service that processed the event.|
|event_time||String||Yes||2020-01-09T12:12:14Z||The time when the event occurred, in UTC.|
|event_type||String||Yes||ApiCall||The type of the event that generated the event log. Valid values:
|request_parameters||Dictionary||No||N/A||The request parameters that was sent with the API request.|
|response_elements||Dictionary||No||N/A||The response data that was returned.|
|service_name||String||Yes||Ecs||The name of the Alibaba Cloud service to which the request was sent.|
|source_ip_address||String||Yes||11.XX.XX.232||The IP address from which the request was sent.
Note If the API operation was called by a user in the console, this field is set to the user's IP address, rather than the IP address of the web server of the console.
|user_agent||String||Yes||Apache-HttpClient/4.5.7 (Java/1.8.0_152)||The agent through which the API request was sent. Valid values:
|user_identity_type||String||Yes||ram-user||The type of the identity. Valid values:
|user_identity_principal_id||String||Yes||28815334868278****||The ID of the requester.
|user_identity_account_id||String||Yes||112233445566****||The ID of the Alibaba Cloud account that owns the requester.|
|user_identity_accessKey_id||String||No||55nCtAwmPLkk****||The AccessKey ID used to make the API request. This field is required if the API request was made through the SDK, and is not required when the API request was made through the console.|
|user_name||String||No||B**||The name of the requester. This field is set to the name of the RAM user if type is set to ram-user. This field is set to a string in the RoleName:RoleSessionName format if type is set to assumed-role.|