When you access Object Storage Service (OSS), large numbers of access logs are generated. After you enable and configure logging for a bucket, OSS generates logs every hour in accordance with predefined naming conventions and then stores the logs as objects in a specified bucket. You can use Alibaba Cloud Log Service or build a Spark cluster to analyze the logs.

Usage notes

  • The source bucket for which logs are generated and the destination bucket in which the logs are stored can be the same or different. However, the destination bucket must belong to the same Alibaba Cloud account in the same region as the source bucket.
  • OSS generates log objects on an hourly basis. However, requests in an hour may be recorded in log objects generated for the previous or subsequent hour.
  • Before you disable logging, OSS keeps generating log objects. Delete log objects that you no longer need to reduce your storage costs.

    You can configure lifecycle rules to regularly delete log objects. For more information, see Lifecycle rules based on the last modified time.

  • More fields may be added to OSS logs. We recommend that developers consider potential compatibility issues when they develop log processing tools.

Naming conventions of log objects

The following naming conventions apply to a log objects:

<TargetPrefix><SourceBucket>YYYY-mm-DD-HH-MM-SS-UniqueString
Parameter Description
TargetPrefix The prefix of the log object name.
SourceBucket The name of the source bucket for which the access logs are generated.
YYYY-mm-DD-HH-MM-SS The time when the log object was created. The items of this field indicate the year, month, day, hour, minute, and second in sequence.
UniqueString The string generated by OSS to uniquely identify the log object.

Log formats and examples

  • Log formats

    OSS access logs include the information about the requester and accessed resources in the following format:

    RemoteIP Reserved Reserved Time "RequestURL" HTTPStatus SentBytes RequestTime "Referer" "UserAgent" "HostName" "RequestID" "LoggingFlag" "RequesterAliyunID" "Operation" "BucketName" "ObjectName" ObjectSize ServerCostTime "ErrorCode RequestLength "UserID" DeltaDataSize "SyncRequest" "StorageClass" "TargetStorageClass" "TransmissionAccelerationAccessPoint" "AccessKeyID"
    Parameter Example Description
    RemoteIP 192.168.0.1 The IP address of the requester.
    Reserved - The reserved field. Retain the default value -.
    Reserved - The reserved field. Retain the default value -.
    Time 03/Jan/2021:14:59:49 +0800 The time when OSS received an access request.
    RequestURL GET /example.jpg HTTP/1.0 The request URL that contains a query string.

    OSS ignores the query string parameter that starts with x-. However, this parameter is recorded in logs. Therefore, you can tag a request by using a query string parameter that starts with x-. Then, you can use this tag to query the log that corresponds to the request.

    HTTPStatus 200 The HTTP status code that OSS returns.
    SentBytes 999131 The downstream traffic generated by the request. Unit: byte.
    RequestTime 127 The time consumed to complete the request. Unit: milliseconds.
    Referer http://www.aliyun.com/product/oss The Referer header in the HTTP request.
    UserAgent curl/7.15.5 The User-Agent header in the HTTP request.
    HostName examplebucket.oss-cn-hangzhou.aliyuncs.com The domain name that the request attempts to access.
    RequestID 5FF16B65F05BC932307A3C3C The ID of the request.
    LoggingFlag true Specifies whether logging is enabled. Valid values:
    • true: Logging is enabled.
    • false: Logging is disabled.
    RequesterAliyunID 16571836914537**** The user ID of the requester. The value of - indicates anonymous access.
    Operation GetObject The type of the request.
    BucketName examplebucket The name of the bucket that the request attempts to access.
    ObjectName example.jpg The name of the object that the request attempts to access.
    ObjectSize 999131 The size of the object that the request attempts to access. Unit: byte.
    ServerCostTime 88 The time that OSS takes to process the request. Unit: milliseconds.
    ErrorCode - The error code that OSS returns. The value of - indicates that no error code is returned.
    RequestLength 302 The length of the request. Unit: byte.
    UserID 16571836914537**** The ID of the bucket owner.
    DeltaDataSize - The size change of the object. The value of - indicates that this request does not involve write operations on the object.
    SyncRequest cdn Specifies whether the request is an Alibaba Cloud CDN back-to-origin request. Valid values:
    • cdn: The request is a CDN back-to-origin request.
    • -: The request is not a CDN back-to-origin request.
    StorageClass Standard The storage class of the object that the request attempts to access. Valid values:
    • Standard
    • IA
    • Archive
    • Cold Archive
    • -: The storage class of the object is not obtained.
    TargetStorageClass - The storage class of the object after the storage class is converted based on a lifecycle rule or the CopyObject operation. Valid values:
    • Standard
    • IA
    • Archive
    • Cold Archive
    • -: The request does not involve operations to convert the storage class of the object.
    TransmissionAccelerationAccessPoint - The accelerate endpoint that is used to access the bucket. For example, if the requester accesses the bucket by using an accelerate endpoint in the China (Hangzhou) region, the value of TransmissionAccelerationAccessPoint is cn-hangzhou.

    The value of - indicates that no accelerate endpoint is used or the accelerate endpoint is in the same region as the bucket that the request attempts to access.

    AccessKeyID LTAI4FrfJPUSoKm4JHb5**** The AccessKey ID of the requester. The value of - indicates anonymous requests.
  • Sample logs
    192.168.0.1 - - [03/Jan/2021:14:59:49 +0800] "GET /example.jpg HTTP/1.0" 200 999131 127 "http://www.aliyun.com/product/oss" "curl/7.15.5" "examplebucket.oss-cn-hangzhou.aliyuncs.com" "5FF16B65F05BC932307A3C3C" "true" "16571836914537****" "GetObject" "examplebucket" "example.jpg" 999131 88 "-" 302 "16571836914537****" - "cdn" "standard" "-" "-" "LTAI4FrfJPUSoKm4JHb5****"

    After the log objects are stored in the specified bucket in OSS, you can use Alibaba Cloud Log Service to analyze the log objects. Before you analyze the log objects, you must import the log objects to Alibaba Cloud Log Service. For more information about how to import data, see Import data from OSS to Log Service. For more information about the analysis feature of Log Service, see Log analysis overview.

Implementation methods

Implementation method Description
Console A user-friendly and intuitive web application.
ossutil A high-performance command-line tool.
Java SDK SDK demos for a variety of programming languages.
Python SDK
PHP SDK
Go SDK
C SDK
.NET SDK
Node.js SDK
Ruby SDK

FAQ

Can I query interrupted requests in OSS access logs?

No. OSS does not record interrupted requests in access logs. If you send a request by using an SDK, you can identify the cause of the request interruption based on the return value of the SDK.