KubeAPI operation audit in Service Mesh (ASM) records every Kubernetes API call made against your mesh, giving you a security-relevant, chronological trail of who did what, when, and from where. Use this feature to:
Trace configuration changes to specific users or system components
Identify unauthorized or suspicious API calls
Analyze operation patterns across namespaces and resource types
Support compliance and forensic investigations
Once enabled, ASM generates structured audit logs and provides built-in dashboards on the Mesh Audit page through the Overview, Operation Audit, Operation Overview, and Operation Details tabs.
Key concepts
In this document, resources refer to Istio resources managed through the Kubernetes API, including virtual services, Istio gateways, destination rules, Envoy filters, sidecar proxies, and service entries.
When you enable mesh audit, ASM automatically creates the following in Simple Log Service (SLS):
| Resource | Naming convention | Purpose |
|---|---|---|
| Project | mesh-log-${Mesh-ID} | Organizes all audit-related Logstores |
| Logstore | audit-${Mesh-ID} | Stores the audit log entries |
Indexes are preconfigured in the Logstore. Do not modify them -- modifying indexes may cause audit reports to fail.
Billing
Audit logs are stored in SLS, and standard SLS charges apply based on log volume. For pricing details, see Pay-by-feature.
Prerequisites
Before you begin, ensure that you have:
Enable KubeAPI operation audit
New ASM instances
Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click Create ASM Instance.
On the Create Service Mesh page, select Enable Mesh Audit in the Mesh Audit section, configure the remaining parameters, and then click Create Service Mesh. For parameter descriptions, see Create an ASM instance.
Existing ASM instances
Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance.
In the left-side navigation pane, choose Mesh Security Center > KubeAPI Operation Audit.
On the Mesh Audit page, select Enable Mesh Audit and click OK.
View audit reports
Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance.
In the left-side navigation pane, choose Mesh Security Center > KubeAPI Operation Audit.
On the Mesh Audit page, click one of the following tabs:
Mesh Audit Overview: Shows an aggregate view of all Kubernetes API operations in the ASM instance, including operation counts, trends, and top operators.
Operation Details: Shows detailed information about operations on specific resource types. Specify a resource type to see the total operation count, distribution by namespace, operation trends over time, and a detailed operation list.
More operations
Query audit logs in SLS
For advanced log queries and analysis beyond the built-in dashboards, query the audit logs directly in the SLS console.
Log on to the Simple Log Service console.
In the Projects section, click the project named
mesh-log-${Mesh-ID}.Click the
audit-${Mesh-ID}Logstore, then click Search & Analyze.
API server audit logs are retained for 30 days by default. To change the retention period, see Manage a Logstore.
Common query patterns
| Query goal | What to enter in the search box |
|---|---|
| Operations by a specific RAM user | The RAM user ID |
| Operations on a specific resource | The resource name |
| Operations by system components only | NOT user.username: node NOT user.username: serviceaccount NOT user.username: apiserver NOT user.username: kube-scheduler NOT user.username: kube-controller-manager |
For query syntax and analysis options, see Log search overview.
Set alerts
SLS supports real-time alerts based on audit log queries. Alerts can be delivered through SMS, DingTalk chatbots, email, custom webhooks, and Message Center in the Alibaba Cloud Management Console. For setup instructions, see Alerting.
The following examples show how to create alert rules using query statements from audit reports.
Example 1: Alert on container command execution
Scenario: A company enforces strict access control and prohibits users from logging on to or running commands on containers in ASM instances. An alert must fire immediately when someone attempts either action. The notification must include the target container, executed commands, operator, event ID, operation time, and source IP address.
Query statement:
verb : create and objectRef.subresource:exec and stage: ResponseStarted | SELECT auditID as "Event ID", date_format(from_unixtime(__time__), '%Y-%m-%d %T' ) as "Operation time", regexp_extract("requestURI", '([^\?]*)/exec\?.*', 1)as "Resource", regexp_extract("requestURI", '\?(.*)', 1)as "Command" ,"responseStatus.code" as "Status code",
CASE
WHEN "user.username" != 'kubernetes-admin' then "user.username"
WHEN "user.username" = 'kubernetes-admin' and regexp_like("annotations.authorization.k8s.io/reason", 'RoleBinding') then regexp_extract("annotations.authorization.k8s.io/reason", ' to User "(\w+)"', 1)
ELSE 'kubernetes-admin' END
as "Username",
CASE WHEN json_array_length(sourceIPs) = 1 then json_format(json_array_get(sourceIPs, 0)) ELSE sourceIPs END
as "Source IP address" limit 100Conditional expression:
Operation event =~ ".*"Example 2: Alert on failed internet access to the API server
Scenario: A company with an internet-facing ASM instance monitors connections from each source IP and their failure rate. An alert must fire when a source IP sends 10 or more requests and more than 5 of them fail. The notification must include the source IP, its geographic region, and whether it is flagged as risky.
Query statement:
* | select ip as "Source IP address", total as "Number of Internet connection requests", round(rate * 100, 2) as "Connection failure rate", failCount as "Number of unauthorized Internet connection requests", CASE when security_check_ip(ip) = 1 then 'yes' else 'no' end as "Whether the IP address is risky", ip_to_country(ip) as "Country", ip_to_province(ip) as "Province", ip_to_city(ip) as "City", ip_to_provider(ip) as "ISP" from (select CASE WHEN json_array_length(sourceIPs) = 1 then json_format(json_array_get(sourceIPs, 0)) ELSE sourceIPs END
as ip, count(1) as total,
sum(CASE WHEN "responseStatus.code" < 400 then 0
ELSE 1 END) * 1.0 / count(1) as rate,
count_if("responseStatus.code" = 403) as failCount
from log group by ip limit 10000) where ip_to_domain(ip) != 'intranet' having "Number of connections" > 10 and "Connection failure rate" > 50 ORDER by "Number of connections" desc limit 100Conditional expression:
Source IP address =~ ".*"Recreate a deleted audit project
If the SLS project used for mesh audit is accidentally deleted, recreate it from the ASM console.
Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance.
In the left-side navigation pane, choose Mesh Security Center > KubeAPI Operation Audit.
On the Mesh Audit page, click Recreate in the Rebuild Mesh Audit message.
The recreated project uses the original project name with a timestamp suffix.
What's next
Configure audit alerts for operations on ASM resources: Set up automated alert notifications for important resource changes.
Overview of zero trust security: Configure workload identity, peer authentication, request authentication, and authorization policies for fine-grained mesh security.