For security compliance audits, performance analysis, and troubleshooting, enable SQL Explorer and Audit. This feature automatically records all SQL statements executed in the database kernel and their related information, such as the executing account, IP address, and execution details. It has a negligible impact on instance performance and provides reliable data for querying historical SQL records and performing subsequent analysis and auditing.
Prerequisites
You have purchased Database Autonomy Service (DAS) Enterprise Edition.
You can use the SQL Explorer and Audit feature only after you enable Database Autonomy Service (DAS) Enterprise Edition. In the console, you can enable only the highest edition of DAS Enterprise Edition that is supported in the current region. Different editions of DAS Enterprise Edition have different supported regions.
A RAM user must have the AliyunRDSReadOnlyWithSQLLogArchiveAccess permission to use the Audit feature. For more information about how to grant permissions to a RAM user, see Use RAM to control permissions on ApsaraDB RDS.
Alternatively, you can use a custom policy to grant a RAM user permission to use the Search (audit) feature, including exporting data. For more information, see Use a custom policy to authorize a RAM user to use the Search (including Export) function of SQL Explorer and Audit.
The SQL Explorer and Audit feature records all Data Query Language (DQL), Data Manipulation Language (DML), and Data Definition Language (DDL) operations. This information is output by the database kernel and consumes minimal CPU resources.
Feature overview
SQL Explorer and Audit provides features such as Search (audit), SQL Explorer, security audit, traffic playback and stress testing, and SQL analysis.
Search (audit): Query and export the execution history of SQL statements and their related information, such as the database, status, and execution time.
SQL Explorer: Diagnose the health of SQL statements, troubleshoot performance issues, and analyze business traffic within a specified time range.
Security audit (legacy): Automatically identifies risks such as high-risk SQL statements, SQL injection, and new access sources.
Traffic playback and stress testing: Provides traffic playback and stress testing features to help you verify whether your instance specifications need to be scaled up to handle peak business traffic.
SQL analysis: Analyzes SQL statements over a specified period to identify abnormal SQL statements and locate performance issues.
SQL Explorer and Audit is commonly used in the following scenarios:
Industries with strict data security requirements, such as finance, security, securities, government, and insurance.
Scenarios that require detailed investigation of database operations, such as issue troubleshooting, SQL statement performance investigation, and abnormal session investigation.
In cases of data loss or damage, use the SQL statements recorded by SQL Explorer to analyze issues and aid data recovery.
Supported regions
China (Hangzhou), China (Shanghai), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Shenzhen), China (Heyuan), China (Guangzhou), China (Chengdu), China (Hong Kong), Singapore, Japan (Tokyo), Malaysia (Kuala Lumpur), Indonesia (Jakarta), US (Silicon Valley), UK (London), US (Virginia), and Germany (Frankfurt)
Billing details
If you purchased an instance and enabled the SQL Explorer feature before it was upgraded to SQL Explorer and Audit: The feature fee is charged by RDS on an hourly basis. Prices vary based on the instance region.
USD 0.0015 per GB/hour: China (Hong Kong), US (Silicon Valley), and US (Virginia).
USD 0.0018 per GB/hour: Singapore, Japan (Tokyo), Germany (Frankfurt), UAE (Dubai), Malaysia (Kuala Lumpur), Indonesia (Jakarta), and UK (London).
USD 0.0012 per GB/hour: China (Hangzhou), China (Shanghai), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Shenzhen), China (Heyuan), China (Guangzhou), and China (Chengdu).
NoteYou can click One-click Upgrade in the upgrade dialog box that appears on the SQL Explorer tab to upgrade to SQL Explorer and Audit. After the upgrade, the fee is charged by DAS.
If you enable the SQL Explorer and Audit feature after the upgrade: The fee is charged by DAS. You can use the SQL Explorer and Audit feature only after you enable Database Autonomy Service (DAS) Enterprise Edition. The supported regions and billing methods vary based on the edition. For more information, see Databases and regions supported by each DAS Enterprise Edition and Product billing of each DAS Enterprise Edition.
NoteIn the console, you can enable only the latest version of DAS Enterprise Edition that is supported in the current region.
Usage notes
Online query:
Time range: A single online query can span a maximum of 24 hours. You can query data for any 24-hour period within the data retention period. Queries that span the maximum duration may time out. If you need to query SQL records over a longer time range, use Simple Log Service (SLS) to access the audit logs. For more information, see Collect audit logs of an ApsaraDB RDS for MySQL instance.
Query method: Online queries support combined searches but not fuzzy searches. Keywords must be at least four characters long.
SQL Explorer and Audit:
Maximum SQL statement length: The maximum length of an SQL statement that can be recorded is
8192bytes. This length is also affected by theloose_rds_audit_max_sql_sizeparameter (for MySQL 5.6/5.7) or theloose_rds_audit_log_event_buffer_sizeparameter (for MySQL 8.0). The effective limit is the minimum of these values. Due to a prefix added during data collection and processing, the actual recorded length of an SQL statement is slightly less than 8,192 bytes or the value of the parameter.Audit record query: You can query SQL statement records by thread ID and transaction ID. To query records by transaction ID, you must set the
loose_rds_audit_log_versionparameter toMYSQL_V3. Specific minor engine versions are required: the minor engine version of MySQL 8.0 must be 20210930 or later, and the minor engine version of MySQL 5.7 must be 20210630 or later. For more information, see Parameters of an ApsaraDB RDS for MySQL 8.0 instance and Update the minor engine version of an ApsaraDB RDS for MySQL instance.SQL Explorer (Trial Edition): If you are using the trial edition of SQL Explorer, you cannot call the DescribeSQLLogRecords and DescribeSQLLogFiles API operations to query audit logs.
Lock wait time: The audit logs contain the lock wait time, whereas the slow query log does not.
When an application uses the prepare method, two statements appear in SQL Explorer: one with a question mark and the other with a specific value.
If you connect to an instance by using the endpoint of a database proxy and the transaction-level connection pool is enabled for the proxy, connections may be reused. In this case, the IP address and port shown by the
show processlistcommand or in SQL Explorer may be different from the actual client IP address and port.When an SQL statement is executed on an ApsaraDB RDS for MySQL instance that is attached to a PolarDB-X 1.0 instance, multiple logs are generated in SQL Explorer and Audit on the ApsaraDB RDS for MySQL instance due to horizontal sharding.
Log integrity: In the following extreme scenarios, some SQL Explorer and Audit logs may be lost:
The instance is under high load and has performance bottlenecks.
The instance receives a high volume of SQL requests.
The audit log collection service is not running correctly.
Enable SQL Explorer and Audit
If you enable the audit log collection feature for your ApsaraDB RDS for MySQL instance in CloudLens for RDS of Simple Log Service (SLS), the SQL Explorer and Audit feature is automatically enabled for the instance.
If you have not enabled the SQL Explorer and Audit feature, you can view the binary log. The binary log allows you to query only the SQL statements for data additions, deletions, and modifications that are generated within the backup retention period. You cannot view the source IP address or account information.
Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
In the left-side navigation pane, choose .
Click Enable DAS Enterprise Edition V3.
Select the features that you want to enable and click Activate.
Modify data storage duration
When you reduce the data retention period for SQL Explorer and Audit, DAS immediately deletes any SQL audit logs older than the new period. Before reducing the retention period, export and save your logs.
Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
In the left-side navigation pane, choose .
Click Service Settings.
On the Service Settings page, modify the log storage duration for Audit Scenarios and click Submit.
Disable SQL Explorer and Audit
After you disable SQL Explorer and Audit, DAS clears all existing logs. Export and save the logs before disabling the feature. When you re-enable the feature, logs are recorded from the time of re-enabling.
Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
In the left-side navigation pane, choose .
On the Audit tab, in the Logs section, click Export.
In the dialog box that appears, select the fields and time range to export, and then click OK.
NoteYou can export a maximum of 10 million records from a seven-day period in a single task.
You can set the Export Time Range to export SQL records for a longer period.
After the export is complete, download the exported file and save it securely.
Click Service Settings to disable SQL Explorer and Audit.
If you have enabled DAS Enterprise Edition V3, clear the check boxes for all features of SQL Explorer and Audit, and click Submit.
NoteAbout one hour after you disable the SQL Explorer and Audit feature, the system releases the storage space used for its data.
Migrate data between Enterprise Editions
The migration operation cannot be terminated or rolled back. Read the migration instructions carefully before you proceed.
If your database instance supports Enterprise Edition V3, you can migrate data from Database Autonomy Service (DAS) Enterprise Edition V1 or V2 to Enterprise Edition V3 to benefit from more favorable pricing. For more information, see How do I migrate data between different versions of DAS Enterprise Edition?
Compared with Enterprise Edition V1, Enterprise Edition V2 uses a different underlying storage architecture that achieves cost-effectiveness through hybrid storage of hot and cold data. Enterprise Edition V3 is built on this hybrid storage architecture and provides more flexible billing by itemizing charges based on the features used.