All Products
Search
Document Center

Simple Log Service:PutLogs

Last Updated:Oct 26, 2023

Writes logs to a specified Logstore.

Description

  • When you call the PutLogs operation to write logs to Simple Log Service, Simple Log Service checks the format of the logs. If a log does not meet the format requirements, the request fails and no logs are written to Simple Log Service.

  • You can write logs only in the Protocol Buffers (Protobuf) format as log groups. For more information, see Data encoding.

  • You can write logs in one of the following modes:

    • LoadBalance: In this mode, logs are randomly written to a shard that is available in the specified Logstore. This mode delivers high availability for write operations and is suitable for data consumption scenarios in which you do not need to preserve the order of logs.

    • KeyHash: In this mode, a hash key is added to the request parameters. The shard to which logs are written is determined based on the hash key. The hash key is optional. If you do not configure the hash key, logs are written to shards in LoadBalance mode. For example, Simple Log Service can write data from a producer such as an instance to a shard based on the hash value of the instance name. This ensures that the data that is written to the shard is ordered and the data in the shard is consumed based on the order. This way, when a shard is split or when shards are merged, the data that contains the same hash key is included only in one shard at a time. For more information, see Shard.

  • The maximum size of raw logs that can be written to Simple Log Service when the PutLogs operation is called is 10 MB. We recommend that the total size of the values for each log in a log group does not exceed 1 MB. Historical versions of SDKs may have different limits. We recommend that you upgrade your SDK to the latest version.

  • The references for Simple Log Service SDK for Java and Simple Log Service SDK for Python provide examples on how to call the PutLogs operation. For more information, see Get started with Log Service SDK for Java and Get started with Log Service SDK for Python.

Protobuf log data

The following tables describe the fields of compressed log data in the Protobuf format. For more information, see Data model and Data encoding.

  • Log

    Parameter

    Type

    Required

    Description

    Time

    Integer

    Yes

    The timestamp at which logs are generated. The timestamp follows the UNIX time format. It is the number of seconds that have elapsed since 00:00:00 Thursday, January 1, 1970.

    Contents

    List

    Yes

    The list of log fields. The list must contain at least one element. For more information, see the Content table.

  • Content

    Parameter

    Type

    Required

    Description

    Key

    String

    Yes

    The name of the custom key.

    Value

    String

    Yes

    The value of the custom key.

  • LogTag

    Parameter

    Type

    Required

    Description

    Key

    String

    Yes

    The name of the custom key.

    Value

    String

    Yes

    The value of the custom key.

  • LogGroup

    Parameter

    Type

    Required

    Description

    Logs

    List

    Yes

    The list of logs. For more information, see the Log table.

    Topic

    String

    No

    The topic of logs. This field is user-defined and is used to distinguish between logs.

    Source

    String

    No

    The source of logs. For example, the IP address of the server in which the log is generated can be used as the source.

    LogTags

    List

    Yes

    The list of log tags. For more information, see the LogTag table.

Request syntax

  • LoadBalance mode

    POST /logstores/logstoreName/shards/lb HTTP/1.1
    Authorization: LOG yourAccessKeyId:yourSignature
    Content-Type: application/x-protobuf
    Content-Length: Content Length
    Content-MD5: Content MD5
    Date: GMT Date
    Host: ProjectName.Endpoint
    x-log-apiversion: 0.6.0
    x-log-bodyrawsize: BodyRawSize
    x-log-compresstype: lz4
    x-log-signaturemethod: hmac-sha1
    <Compressed log data in the Protobuf format>
  • KeyHash mode

    POST /logstores/logstoreName/shards/route?key=14d2f850ad6ea48e46e4547edbbb27e0
    Authorization: LOG yourAccessKeyId:yourSignature
    Content-Type: application/x-protobuf
    Content-Length: Content Length
    Content-MD5: Content MD5
    Date: GMT Date
    Host: ProjectName.Endpoint
    x-log-apiversion: 0.6.0
    x-log-bodyrawsize: BodyRawSize
    x-log-compresstype: lz4
    x-log-signaturemethod: hmac-sha1
    <Compressed log data in the Protobuf format>

Host consists of a project name and a Simple Log Service endpoint. You must specify a project in Host.

Request parameters

  • Request headers

    For more information about the common request headers of Simple Log Service API, see Common request headers.

  • Request parameters

    Parameter

    Type

    Required

    Example

    Description

    projectName

    String

    Yes

    ali-test-project

    The name of the project.

    logstoreName

    String

    Yes

    sample-logtail-config

    The name of the Logstore.

Response parameters

  • Response headers

    This operation returns only common response headers. For more information about the common response headers of Simple Log Service API, see Common response headers.

  • Response elements

    If the HTTP status code 200 is returned, the request is successful. If the request is successful, no response elements are returned.

Examples

  • Sample requests

    POST /logstores/sls-test-logstore/shards/lb
    {
        "Content-Length": 118,
        "Content-Type":"application/x-protobuf",
        "x-log-bodyrawsize":1356,
        "Host": "ali-test-project.cn-hangzhou-devcommon-intranet.sls.aliyuncs.com",
        "Content-MD5":"6554BD042149C844761C2C094A8FECCE",
        "Date":"Thu, 12 Nov 2015 06:54:26 GMT",
        "x-log-apiversion": "0.6.0",
        "x-log-compresstype":"lz4"
        "x-log-signaturemethod": "hmac-sha1",
        "Authorization":"LOG yourAccessKeyId:yourSignature"
    }
    <Binary data that is obtained after logs in the Protobuf format are compressed by using the LZ4 algorithm>
  • Sample success responses

    Header
    {   
        "Access-control-allow-origin" : "*", 
        "Date" : "Wed, 11 Nov 2015 08:28:20 GMT",
        "Server" : "Tengine",
        "Content-Length" : 0, 
        "x-log-requestid" : "5642FC2399248C8F7B0145FD",
        "Connection" : "close"
    }

Error codes

HTTP status code

Error code

Error message

Description

400

PostBodyInvalid

Protobuffer content cannot be parsed.

The error message returned because the logs in the Protobuf format are invalid and cannot be parsed.

400

InvalidTimestamp

Invalid timestamps are in logs.

The error message returned because the logs contain invalid timestamps.

400

InvalidEncoding

Non-UTF8 characters are in logs.

The error message returned because the logs contain non-UTF-8 characters.

400

InvalidKey

Invalid keys are in logs.

The error message returned because the logs contain invalid keys.

400

PostBodyTooLarge

Logs must be less than or equal to 3 MB and 4096 entries.

The error message returned because the size of logs is greater than or equal to 3 MB or the number of logs exceeds 4,096.

400

PostBodyUncompressError

Failed to decompress logs.

The error message returned because the logs failed to be decompressed. For more information, see Data compression.

400

PostBodyInvalid

The post data time is out of range

The error message returned because the log timestamp is not within the time range [-168 hours,+15 minutes].

404

LogStoreNotExist

logstore logstoreName does not exist.

The error message returned because the specified Logstore does not exist.

For a list of error codes, see Common error codes.