Enable Log Shipping for a Cloud-Native Gateway - MSE

Microservices Engine (MSE) cloud-native gateways generate access logs for every request they process. To centralize these logs for querying, failure tracing, traffic analysis, and understanding the geographical locations of client users, enable log shipping to send them to Simple Log Service (SLS).

Prerequisites

Before you begin, make sure that you have:

A cloud-native gateway. For more information, see Create a cloud-native gateway
Simple Log Service activated. To activate it, log on to the Simple Log Service console. For more information, see What is Simple Log Service?

Enable log shipping

Log on to the MSE console. In the top navigation bar, select a region.
In the left-side navigation pane, choose Cloud-native Gateway > Gateways. On the Gateways page, click the ID of the gateway.
In the left-side navigation pane, click Parameter Settings.
In the Observability Parameters section, click next to Log Shipping.
In the Log Shipping Settings dialog box, turn on Enable Log Shipping (Ship Gateway Access Logs to Log Service).
Note
- After you enable log shipping, an SLS project is created automatically. Alternatively, select an existing SLS project.
- If Managed Service for OpenTelemetry is activated, metric data for the gateway is available in the Managed Service for OpenTelemetry console. For more information, see Enable tracing analysis for a cloud-native gateway.

View and query logs

After log shipping is enabled, click the link next to Project in the Observability Parameters section to open the Logstore for the gateway in the SLS console.

To find requests that returned 5xx errors in the past hour, run the following query:

response_code >= 500 | SELECT request_id, method, path, response_code, upstream_host, duration ORDER BY duration DESC LIMIT 50

To trace a specific request, locate the x-request-id value from the client response header and search the logs:

request_id: <x-request-id-value>

For more query syntax, see Guide to log query and analysis.

Log fields

Each access log entry contains the fields listed in the following table. The following sample shows a typical entry.

Sample log entry

{
  "__time__": 1741593600,
  "cluster_id": "gw-abc123def456",
  "method": "POST",
  "path": "/api/v1/orders",
  "protocol": "HTTP/1.1",
  "authority": "api.example.com",
  "response_code": 200,
  "response_flags": "-",
  "bytes_received": 256,
  "bytes_sent": 1024,
  "duration": 85,
  "request_duration": 12,
  "response_tx_duration": 3,
  "upstream_service_time": 70,
  "upstream_host": "10.0.2.15:8080",
  "upstream_cluster": "outbound|8080||order-service.default.svc.cluster.local",
  "downstream_local_address": "10.0.1.5:443",
  "downstream_remote_address": "203.0.113.50:52341",
  "request_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "route_name": "order-api-route",
  "user_agent": "Mozilla/5.0 (compatible; MyApp/2.0)",
  "x_forwarded_for": "203.0.113.50",
  "start_time": "2025-03-10T08:00:00.000Z",
  "trace_id": "abc123def4567890abc123def4567890",
  "upstream_protocol": "HTTP/1.1",
  "requested_server_name": "api.example.com"
}

Field reference

Field name	Data type	Description
__time__	long	Timestamp when the log entry was generated.
cluster_id	string	ID of the gateway instance.
consumer	string	Consumer name. Populated when consumer authentication is enabled.
custom_log	json	Custom log data. Used with custom plug-ins.
authority	string	`Host` header value from the request.
bytes_received	long	Request body size in bytes. Excludes the request header.
bytes_sent	long	Response body size in bytes. Excludes the response header.
downstream_local_address	string	Address of the gateway pod that handled the request.
downstream_remote_address	string	Address of the client connected to the gateway.
downstream_transport_failure_reason	string	Reason for a downstream (client-side) transport failure.
duration	long	Total request processing time in milliseconds, measured from when the gateway receives the first byte from the client to when it sends the last byte of the response.
method	string	HTTP method (for example, `GET`, `POST`, `PUT`).
path	string	Request path (for example, `/api/v1/orders`).
protocol	string	HTTP protocol version (for example, `HTTP/1.1`).
request_id	string	Unique request ID. The gateway generates this value and adds it to the `x-request-id` header field. Backend servers can parse this ID from the header for troubleshooting.
requested_server_name	string	Server name used to establish the SSL/TLS connection (SNI).
request_duration	long	Time in milliseconds from when the gateway receives the first byte to the last byte of the client request.
response_code	long	HTTP status code returned to the client (for example, `200`, `502`, `504`).
response_code_details	string	Additional context for the status code, such as `via_upstream`, `route_not_found`, or `direct_response`.
response_flags	string	Response flag indicating the cause of a request failure. See the Response flags section for details.
response_tx_duration	long	Time in milliseconds from when the gateway receives the first byte from the upstream service to when it sends the last byte to the client.
route_name	string	Name of the matched route.
start_time	string	Request start time in UTC format.
trace_id	string	Distributed trace ID. Available when tracing is enabled.
upstream_cluster	string	Name of the upstream service cluster.
upstream_host	string	IP address and port of the upstream service that handled the request.
upstream_local_address	string	Local address used by the gateway to connect to the upstream service.
upstream_protocol	string	Protocol used for the request to the upstream service.
upstream_service_time	long	Time in milliseconds for the gateway to reach the upstream service and for the upstream service to process the request.
upstream_transport_failure_reason	string	Reason for a connection failure to the upstream service.
user_agent	string	`User-Agent` header value from the request.
x_forwarded_for	string	`X-Forwarded-For` header value, typically containing the client's real IP address.
ext_authz_status_code	long	Status code returned by an external authorization service.
ext_authz_duration	long	Response time of an external authorization service in milliseconds.

Response flags

The response_flags field identifies why a request failed. The following tables list all possible values, grouped by category.

Note

In this context, downstream refers to the client and upstream refers to the backend service.

Upstream failures

Flag	Description
UH	No healthy upstream host available in the cluster.
UF	Connection to the upstream service failed.
UT	Upstream request timed out.
UR	Upstream service reset the connection.
UC	Connection to the upstream service was terminated.
UPE	HTTP protocol error in the upstream service response.
UMSDR	Upstream request processing time exceeded the maximum allowed duration.

Routing and cluster errors

Flag	Description
NR	No route is configured for the request.
NC	The upstream cluster was not found.

Downstream failures

Flag	Description
DC	Connection from the client (downstream) was terminated.
DPE	HTTP protocol error in the client (downstream) request.

Timeout and duration limits

Flag	Description
DT	Request or connection duration exceeded `max_connection_duration` or `max_downstream_connection_duration`.
SI	Stream idle timeout.

Rate limiting and authorization

Flag	Description
RL	Request was rate-limited locally by the HTTP rate limiting filter. Does not apply to 429 status codes.
RLSE	Request was rejected due to a rate limiting service error.
UAEX	Request was rejected by an external authorization service.

Fault injection

Flag	Description
DI	Request processing was delayed by a fault injection rule.
FI	Request was aborted with a status code specified in a fault injection rule.

Other

Flag	Description
LH	Local service did not pass the health check.
LR	Connection was reset locally.
URX	Request was rejected because the upstream retry limit was reached or the maximum number of TCP connection attempts was exceeded.
IH	Request was rejected because a header contained an invalid value that failed strict validation.
OM	Request was terminated by the overload manager.
DF	Request was terminated due to a DNS resolution failure.

Billing

Log shipping does not incur additional fees in MSE. SLS charges apply based on the resources consumed. For details, see Pay-by-feature.