Log Service allows you to aggregate data from multiple source Logstores to a target Logstore by configuring a data transformation task for each source Logstore.

Note Currently, data transformation tasks can be dispatched only in the same region.
The following section describes an example of aggregating data from multiple source Logstores under different accounts.
  • Raw logs
    """
    Logs in the Logstore of account 1
    Project region: UK (London)
    Project name: Project_1
    Logstore name: Logstore_1
    """
    "Log 1"
    request_id: 1
    http_host:  m1.abcd.com
    http_status:  200
    request_method:  GET
    request_uri:  /pic/icon.jpg
    
    "Log 2"
    request_id: 2
    http_host:  m2.abcd.com
    http_status:  301
    request_method:  POST
    request_uri:  /data/data.php
    
    """
    Logs in the Logstore of account 2
    Project region: UK (London)
    Project name: Project_2
    Logstore name: Logstore_2
    """
    "Log 1"
    request_id: 3
    host:  m3.abcd.com
    status:  404
    request_method:  GET
    request_uri:  /category/abc/product_id
    
    "Log 2"
    request_id: 4
    host:  m4.abcd.com
    status:  200
    request_method:  GET
    request_uri:  /data/index.html
  • Aggregation objectives
    • Aggregate all log events whose http_status is 200 in Logstore_1 of account 1 and Logstore_2 of account 2 into Logstore_3 of account 3.
    • Unify the expressions of log event fields in the two Logstores. That is, use the http_host field instead of the host field and use the http_status field instead of the status field in the log events in both Logstores.
  • LOG DSL rules
    • Configure the following transformation rule in Logstore_1 of account 1:
      e_if(e_match("http_status", "200"), e_output("target_logstore"))
      In addition, configure Logstore_3 of account 3 as the target Logstore target_logstore in the task configuration item of the transformation rule.Transformation rule
    • Configure the following transformation rule in Logstore_2 of account 2:
      e_if(e_match("status", "200"), e_compose(e_rename("status", "http_status", "host", "http_host"), e_output("target_logstore")))
      With reference to the configuration of account 1, configure Logstore_3 of account 3 as the target Logstore target_logstore in the task configuration item of the transformation rule.
  • Transformation result
    """
    Logs in the Logstore of account 3
    Project region: UK (London)
    Project name: Project_3
    Logstore name: Logstore_3
    """
    "Log 1"
    request_id: 1
    http_host:  m1.abcd.com
    http_status:  200
    request_method:  GET
    request_uri:  /pic/icon.jpg
    
    "Log 2"
    request_id: 4
    http_host:  m4.abcd.com
    http_status:  200
    request_method:  GET
    request_uri:  /data/index.html