All Products
Search
Document Center

Modify a data export task

Last Updated: May 31, 2020

Overview

This topic describes how to use the Log Service console to modify the parameter settings of existing data export tasks.

Prerequisites

  • The Log Service and Time Series Database (TSDB) services are purchased.
  • The Log Service and TSDB services reside in the same region. Otherwise, they cannot exchange data.
  • A data export task is created in the Log Service console.

Modify task configurations

When you create a data export task in the Log Service console to export data from Log Service to TSDB, you can specify the task details. For example, you can specify the target TSDB instance, the time of the earliest data record to be exported, the Log Service topic, and the TSDB metric.

This topic describes how to modify task configurations.

Operations in the Log Service console

  1. Log on to the Log Service console, and select the project and the Logstore for which the data export task is created.
  2. Click the right arrow for the target Logstore to show the Logstore drop-down list. Then, click Export, click TSDB, and then click the name of the target TSDB data export task. On the page that appears, you can manage the data export task.

  3. Click the Modify LogShipper button. On the page that appears, modify the task configurations.

Modify the task configurations in the console

Before you modify the task configurations, you must enter the AccessKey ID and the AccessKey secret of the logon account to verify the identity and load task data.

After the AccessKey ID and AccessKey secret are verified, the system loads the existing task configurations. Then, you can modify the task configurations based on your business needs.

Parameters

Parameter
Description
Shipper Name The task name. The task name cannot be changed.
Shipper Description The task description.
TSDB Instance The TSDB instance to which the data is exported. You can select a TSDB instance from the drop-down list that shows the instances that you have purchased in the current region.
Metric The TSDB metric for which the data is exported.
AccessKey ID The AccessKey ID of the logon account. You must specify the AccessKey ID for identity verification when you export data from Log Service to TSDB.
AccessKey Secret The AccessKey secret of the logon account. You must specify the AccessKey secret for identity verification when you export data from Log Service to TSDB.
Time Point Mapping Specifies the mappings between the timestamps in log records and the TSDB timestamps. The default value of this parameter is ${logTag:__time__}. The default value indicates that the time points when logs were generated are considered as the TSDB timestamps. You can specify an expression for this parameter.
Field Mapping Specifies the mappings between the fields in log records and TSDB fields. In the first text box, enter the name of a field in the log records. In the second text box, enter the name of a TSDB field. If you specify only one field mapping, data is exported based on the single-value data model. In this case, the name of the TSDB field must be value. If you specify multiple field mappings, data is exported based on the multi-value data model.
Tag Mapping Specifies the TSDB tags for the exported data records. In the first text box, enter a tag name. In the second text box, enter a tag value. You can also use expressions to specify the tag names and the tag values.
Shipping Time Specifies the time when the earliest data record to be shipped was generated. Log Service does not ship the data that is generated earlier than the specified time.
  1. If no metrics exist, the system automatically creates a metric.

  2. When Log Service exports logs to TSDB, the setting of the Time Point Mapping parameter determines the Timestamp field values of TSDB data points. The default value of this parameter is ${logTag:time}. The default value indicates that the time points when log records were generated are considered as the TSDB timestamps. The timestamps of log records are specified by the time parameter and use the second time unit. Therefore, if multiple log records occur within one second, you must determine the log records based on which the TSDB timestamps are determined.

  3. You can use the ${logTag:xx} and ${logColumn:xx} expressions to specify the numeric values for the Time Point Mapping parameter. The values of the fields specified by the expressions must be able to be converted into numeric values. Each field value must be converted into the number of seconds or milliseconds that have elapsed since 00:00:00, January 1, 1970.

  4. If the values of the fields specified by the expressions fail to be converted into numeric values, the system uses the log record timestamps as the TSDB timestamps. The log record timestamps are specified by the ${logTag:time} element.

Use expressions to reference the context information

When you configure data export tasks, you can use expressions to reference the context information for the Time Point Mapping and Tag Mapping parameters. The expression syntax is described as follows:

${context:variable}

  • The logColumn prefix in an expression indicates that the expression references the column values in the original logs of Log Service as the target values.
  • The logTag prefix in an expression indicates that the expression references the tag values in the original logs of Log Service as the target values.

You can always find the following tags in the logs of Log Service:

  • __topic__: indicates the service tag of an entire log group. A log group may contain one or more log records. The expression format is ${logTag:__topic__}.
  • __source__: indicates the IP address of the machine that generates the log record. The expression format is ${__source__}.
  • __time__: indicates the timestamp of the log record. The unit of the timestamp is seconds. The expression format is ${__time__}.

For more information about Log Service context variables, see the Log Service documentation.

Submit the task

After you specify the required parameters, click Submit to modify the configurations of the data export task.

Considerations

  • The changes that are made to the task configurations do not take effect instantly. The changes take effect only after you log on to the Log Service console, stop the task, and then restart the task.
  • After you modify the configurations of a data export task, you can click the task name in the Log Service console. On the page that appears, you can stop the task or view the task status.