This topic describes some common use cases where functions are called to process date and time.

Use case 1: data type conversions

Three types of data are involved in processing date and time in the LOG DSL syntax: UNIX timestamp, datetime string, and datetime object. They can be mutually converted to each other, as shown in the following figure.Data type conversions
1. Mutual conversion between datetime objects and UNIX timestamps
  • Convert a datetime object to a UNIX timestamp.
    • dt_parsetimestamp: a smart conversion function, which converts a datetime object or datetime string to a UNIX timestamp.
    • dt_totimestamp: converts only a datetime object to a UNIX timestamp.
  • Convert a UNIX timestamp to a datetime object.
    • dt_parse: a smart conversion function, which converts a UNIX timestamp or datetime string to a datetime object.
    • dt_fromtimestamp: converts only a UNIX timestamp to a datetime object.
2. Mutual conversion between datetime objects and datetime strings
  • Convert a datetime object to a datetime string.
    • dt_str: a smart conversion function, which converts a datetime object, UNIX timestamp, or datetime string to a datetime string in a specified format.
    • dt_strftime: converts only a datetime object to a datetime string.
  • Convert a datetime string to a datetime object.
    • dt_parse: a smart conversion function, which converts a datetime string or UNIX timestamp to a datetime object.
    • dt_strptime: converts only a datetime string to a datetime object.
3. Mutual conversion between datetime strings and UNIX timestamps
  • Convert a datetime string to a UNIX timestamp.

    dt_parsetimestamp: a smart conversion function, which converts a datetime string or datetime object to a UNIX timestamp.

  • Convert a UNIX timestamp to a datetime string.
    • dt_str: a smart conversion function, which converts a UNIX timestamp, datetime object, or datetime string to a datetime string in a specified format.
    • dt_strftimestamp: converts only a UNIX timestamp to a datetime string.

4. Use of conversion functions such as dt_parse

Most of the preceding conversions can be implemented by using either a smart conversion function or a dedicated conversion function. Smart conversion functions, such as dt_parse, can convert different types of field values, such as UNIX timestamps, datetime objects, and datetime strings, in a smart manner. The following is an example:
  • Raw log 1
    time1: 1562741899
    time2: 2019-07-10 06:58:19
  • LOG DSL orchestration
    e_set("time3", dt_parse(v("time1"), tz="Asia/Shanghai"))
    e_set("time4", dt_parse(v("time2"), tz="Asia/Shanghai")
  • Transformation result
    time1: 1562741899
    time2: 2019-07-10 06:58:19
    time3: 2019-07-10 06:58:19+08:00
    time4: 2019-07-10 06:58:19+08:00
However, smart conversion functions cannot meet your requirements in some scenarios. For example, smart conversion functions such as dt_parse cannot automatically parse date and time values in a custom format. In this case, you need to call dt_strptime to parse these values.
  • Raw log 2
    time1: 2019-07-10 06:58:19
    time2: 2019/07/10 06-58-19
  • LOG DSL orchestration
    e_set("time3", dt_parsetimestamp(v("time1")))
    e_set("time4", dt_parsetimestamp(dt_strptime(v("time2"), fmt="%Y/%m/%d %H-%M-%S")))
  • Transformation result
    time1: 2019-07-10 06:58:19
    time2: 2019/07/10 06-58-19
    time3: 1562741899
    time4: 1562741899
Note
  • We recommend that you call smart conversion functions by default because they can automatically convert different types of field values.
  • Smart conversion functions cannot automatically parse data and time values in a custom format. In this case, you need to call dt_strptime to parse these values.

Use case 2: conversions related to the time zone

  • 1. Different forms of datetime strings
    The datetime strings in the LOG DSL syntax are expressed in the following two forms:
    • A datetime string with a time zone, such as 2019-06-02 18:41:26+08:00.
    • A datetime string without a time zone, such as 2019-06-02 10:41:26.
    A time zone is appended to a datetime string to indicate the difference between the local time and the UTC time.
    • 2019-06-02 18:41:26+08:00 indicates that the time 2019-06-02 18:41:26 is in the UTC+8 time zone.
    • 2019-06-02 18:41:26-07:00 indicates that the time 2019-06-02 18:41:26 is in the UTC-7 time zone.
    The time indicated by a datetime string without a time zone varies depending on the time zone. If the time zone field of a time is not specified, the time is regarded as being in the UTC+0 time zone. Take 2019-06-02 18:41:26 as an example.
    • By default, the time is regarded as being in the UTC+0 time zone. That is, the time indicated by this time string is equivalent to 2019-06-02 18:41:26+00:00.
    • If the time zone is UTC+8, the time indicated by the time string is equivalent to 2019-06-02 18:41:26+08:00.
  • 2. Convert a datetime string to a UNIX timestamp
    • Convert a datetime string without a time zone

      To convert a datetime string without a time zone, such as 2019-06-02 18:41:26, to a UNIX timestamp, you must specify a time zone for the datetime string. The converted UNIX timestamp varies depending on the time zone.

      Raw log
      { 'time': '2019-06-02 18:41:26'}
      LOG DSL orchestration
      e_set("Shanghai_timestamp", dt_parsetimestamp(v("time"), tz="Asia/Shanghai"))
      e_set("Los_Angeles_timestamp", dt_parsetimestamp(v("time"), tz="America/Los_Angeles"))
      e_set("UTC_timestamp", dt_parsetimestamp(v("time")))
      • tz="Asia/Shanghai" indicates that the time indicated by the time field is in the time zone of Shanghai.
      • If no time zone is specified, the given time is regarded as being in the UTC+0 time zone.
      • For more information about all optional time zone strings for the tz field, see Time zone list.
      Transformation result
      {
        'Shanghai_timestamp': '1559472086',
        'Los_Angeles_timestamp': '1559526086',
        'UTC_timestamp': '1559500886'
      }
    • Convert a datetime string with a time zone

      You do not need to specify any time zone field for a datetime string with a time zone, such as 2019-06-02 18:41:26+08:00.

      Raw log
      { 'China_time': '2019-06-02 18:41:26+08:00',
        'America_time': '2019-06-02 3:41:26-07:00',
        'UTC_time': '2019-06-02 10:41:26+00:00'
      }
      LOG DSL orchestration
      e_set("timestamp1", dt_parsetimestamp(v("China_time")))
      e_set("timestamp2", dt_parsetimestamp(v("America_time")))
      e_set("timestamp3", dt_parsetimestamp(v("UTC_time")))
      Transformation result
      {
        "timestamp1": "1559472086",
        "timestamp2": "1559472086",
        "timestamp3": "1559472086"
      }
  • 3. Convert datetime strings in different time zones
    • Convert a datetime string without a time zone

      For a datetime string without a time zone, such as 2019-06-02 18:41:26, you can convert the datetime string to a UNIX timestamp and then convert the UNIX timestamp to a datetime string in another time zone.

      Transformation request: Convert a time in the time zone of Los Angeles to a time in the time zone of Shanghai.

      Raw log
      # Assume that the time is in the time zone of Los Angeles.
      {'time': '2019-06-04 2:41:26'}
      LOG DSL orchestration
      e_set("timestamp", dt_parsetimestamp(v("time"), tz="America/Los_Angeles"))
      e_set("Shanghai_time", dt_parse(v("timestamp"), tz="Asia/Shanghai"))
      Transformation result
      {
        'time': '2019-06-04 2:41:26',
        'Shanghai_time': '2019-06-04 17:41:26+08:00'
      }
    • Convert a datetime string with a time zone

      For a datetime string with a time zone, such as 2019-06-02 18:41:26+08:00, you can directly call dt_astimezone to convert the datetime string to a datetime string in another time zone.

      Raw log
      {'time': '2019-06-04 2:41:26+08:00'}
      LOG DSL orchestration
      e_set("new_time", dt_astimezone(v("time"), tz="America/Los_Angeles"))
      Transformation result
      {
        'time': '2019-06-04 2:41:26+08:00',
        'new_time': '2019-06-03 11:41:26-07:00'
      }

Use case 3: use of datetime strings, datetime objects, and UNIX timestamps

The date and time involved in LOG DSL orchestration are expressed in two forms: UNIX timestamp and datetime string or object.

Datetime string or object: facilitates data display and helps you read logs easily.

UNIX timestamp: the number of seconds that have elapsed since the midnight of January 1, 1970 at UTC or GMT. It can be used in the following scenarios:
  • Use a UNIX timestamp to express the system time.
    In event logs, the metadata field __time__ indicates the time when a log was generated and the field __receive_time__ indicates the time when a log was received. The values of these fields use UNIX timestamps to indicate the corresponding system time.
    __source__:  1.2.3.4
    __tag__:__receive_time__:  1562741899
    __topic__: 
    __time__: 1562731122
  • Use UNIX timestamps to perform time-related computations.
    The UNIX timestamp indicates the number of seconds that have elapsed since January 1, 1970. You can use it to directly perform time-related computations in many scenarios.
    • Raw log
      time1: 1562741899
      time2: 1562731122
    • LOG DSL orchestration
      e_set("time_diff", op_sub(v("time1"), v("time2")))
    • Transformation result
      time1: 1562741899
      time2: 1562731122
      time_diff: 10777

Use case 4: date offset

You can call the dt_add function to add a number to, subtract a number from, or overwrite a certain date part of the input datetime value. The fields in dt_add are as follows:
dt_add(field name, dt1=None, dt2=None, year(s)=None, month(s)=None, day(s)=None, hour(s)=None, minute(s)=None, second(s)=None, microsecond(s)=None, weeks(s)=None, weekday=None)
  • 1. Meanings of year(s) and month(s)

    Those fields with (s) in their names, such as year(s), month(s), and day(s) can be expressed in two forms, for example, year and years for year(s), and month and months for month(s).

    Take year and years as an example. If year is used, the value of year is substituted for the existing value at the granularity of year. If years is used, the value of years is added to the existing value at the granularity of year.

    • Raw log
      "time1": "2019-06-04 2:41:26"
    • LOG DSL orchestration 1
      e_set("time2", dt_add(v("time1"), year=2018))
    • Transformation result 1
      "time1": "2019-06-04 2:41:26"
      "time2": "2018-06-04 02:41:26"
    • LOG DSL orchestration 2
      e_set("time2", dt_add(v("time1"), years=2018))
    • Transformation result 2
      "time1": "2019-06-04 2:41:26"
      "time2": "4037-06-04 02:41:26"
  • 2. Use of the weekday field
    The weekday field is used together with related weekday functions, such as dt_MO and dt_TU, to indicate the offset to a specific day in a week. For more information, see dt_MO.
    • Raw log
      # June 4, 2019 is a Tuesday.
      
      "time1": "2019-06-04 2:41:26"
    • LOG DSL orchestration
      # Return the date of the next Monday since time1.
      e_set("nex_Monday", dt_add(v("time1"), weekday=dt_MO(1)))
      
      # Return the date of the last Tuesday since time1.
      e_set("previous_Tuesday", dt_add(v("time1"), weekday=dt_TU(op_neg(1))))
      
      # Return the date of the second Saturday since time1.
      e_set("nex_next_Saturday", dt_add(v("time1"), weekday=dt_SA(2)))
      
      # Return the date of the second to last Sunday since time1.
      e_set("previous_previous_Sunday", dt_add(v("time1"), weekday=dt_SU(op_neg(2))))
    • Transformation result
      "time1": "2019-06-04 2:41:26",
      "next_Monday": "2019-06-10 02:41:26",
      "previous_Tuesday": "2019-06-04 2:41:26",
      "next_next_Saturday": "2019-06-15 02:41:26",
      "previous_previous_Sunday": "2019-05-26 02:41:26"
    Note If the date corresponding to time1 is Tuesday, its last Tuesday and next Tuesday are both time1.