Simple Log Service:Quick preview

Procedure

1. Log on to the Simple Log Service console.

2. Go to the data transformation page.

   1. In the Projects section, click the project you want.

   2. On the Log Storage > Logstores tab, click the logstore you want.

   3. On the query and analysis page, click Data Transformation.

3. In the upper-right corner of the page, select a time range for the data.

   After you select a time range, verify that logs appear on the Raw Logs tab.

4. In the editing box, you can enter the data transformation statements.

   For more information about the syntax, see Data transformation syntax.

   Note

   The editor supports comments in transformation statements. You can use this feature to debug your script line by line.

5. You can preview the data.

   1. In the upper-right corner of the page, click Quick.

   2. At the bottom of the page, click Test Data.

   3. On the Test Data tab, you can enter test data.

      Test data includes base data and dimension table data.

      • On the Data tab, you can set the test data.

        On the Raw Logs tab, you can find a destination log and click Add to Test Data. You can also enter test data manually.

        Note

        • The size of test data for a single preview cannot exceed 1 MB.

        • You can separate multiple test data entries with a blank line.

        • For field values that span multiple lines, you can use the Markdown format. You can use ``` to identify the entire field.

        • Test data on the Data tab can be in key-value (KV) or JSON format. In KV format, you can use a colon (:) to connect a field name and its value.

        • Example 1: This example contains two test data entries. The first is in KV format and includes a multi-line field named `traceback`. The second is in JSON format.

          time_local: 25/May/2020:01:56:22
          user agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18
          "request:method": GET
          ```    
          traceback: Traceback (most recent call last):
            File "traceback_print_exc.py", line 20, in <module>
              produce_exception()
            File "/home/user/code/test.py", line 16, in produce_exception
              produce_exception(recursion_level-1)
            File "/home/user/code/test.py", line 18, in produce_exception
              raise RuntimeError()
          
          RuntimeError
          ```
          
          {
            "time_local": "25/May/2020:01:56:22",
            "user agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18",
            "request:method": "GET",
            "remote user": "john"
          }

        • Example 2: This example is a complete JSON instance that contains three test data entries.

          [
            {
              "time_local": "25/May/2020:01:56:22",
              "user agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18",
              "request:method": "GET",
              "remote user": "john"
            },
            {
              "time_local": "25/May/2020:01:56:22",
              "user agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18",
              "request:method": "GET",
              "remote user": "john"
            },
            {
              "time_local": "25/May/2020:01:56:22",
              "user agent": "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.18 (KHTML, like Gecko) Chrome/11.0.661.0 Safari/534.18",
              "request:method": "GET",
              "remote user": "john"
            }
          ]

      • On the Dimension Table tab, you can set the test data.

        Dimension tables are used to preview resources that resource functions use. You can enter sample dimension table data to test the data preview.

        Note

        On the Dimension Table tab, the data samples for the `res_rds_mysql` and `res_log_LogStore_pull` resource functions must be in CSV format. The data samples for the `res_oss_file` and `res_local` resource functions can be in CSV or JSON format.

        Example: This example contains two test data entries. The first is in CSV format, and the second is in JSON format.

        ip,country,province
        127.0.0.1,China,Shanghai
        192.168.0.0,China,Zhejiang
        
        [
          {
            "ip": "127.0.0.1",
            "country": "China",
            "province": "Shanghai"
          },
          {
            "ip": "192.168.0.0",
            "country": "China",
            "province": "Zhejiang"
          }
        ]

   4. Click Preview Data.

      Note

      A single preview returns a maximum of 100 transformation results.

      After the preview is configured, you can view the results on the Transformation Results tab.

      • If data transformation fails because of statement errors or incorrect permission configurations, you can follow the on-screen instructions to resolve the issue.

      • If the transformation results are correct, you can save the results. For more information, see Create a data transformation task.

Quick Preview Example

• Transformation statement

  You can enter the following transformation statement in the editing box.

  # e_set("insert_field", "test_value")
  e_table_map(
      res_rds_mysql(
          address="rm-uf6wjk5****.mysql.rds.aliyuncs.com",
          username="test_username",
          password="****",
          database="test_db",
          table="test_table",
      ),
      "ip",
      ["country", "province"],
  )

• Test data

  On the Test Data > Data tab, you can enter the following content.

  {
      "id": "1001", 
      "ip": "127.0.0.1"
  }

• Dimension table data

  On the Test Data > Dimension Table tab, you can enter the following content.

  ip,country,province
  127.0.0.1,China,Shanghai
  192.168.0.0,China,Zhejiang

• Preview results