After you create a data transformation task, a dashboard is automatically created for the task. You can view the metrics of the task on the dashboard. This topic describes the metrics of a data transformation task and how to view the metrics on the dashboard.

View the dashboard

After you create a transformation task, you can view the task on the Data Transformation Overview page of the task.

  1. Log on to the Log Service console.
  2. In the Projects section, find the project to which the job belongs and click the project.
  3. In the left-side navigation pane, choose Jobs > Data Transformation.
  4. Click the target data transformation task. The Data Transformation Overview page of the task appears. In the Status section of the page, view the metrics of the task.

Overall metrics

The following figure shows the overall metrics.Overall metrics
  • Total logs read: the total number of log entries that are read from all shards of the source Logstore.
  • Total logs delivered: the total number of log entries that are read from all shards of the source Logstore and sent to the destination Logstore.
  • Total logs failed: the total number of log entries that are read from all shards of the source Logstore and failed to be transformed.
  • Log delivered ratio: the ratio of the log entries that are sent to the destination Logstore to the log entries that are read from the source Logstore.

Transform speed

This metric indicates the number of log entries that are transformed every minute.Transform speed
  • accept: the number of log entries that are read from all shards of the source Logstore.
  • dropped: the number of log entries that are read from all shards of the source Logstore and dropped as expected.
  • delivered: the number of log entries that are read from all shards of the source Logstore and sent to the destination Logstore.
  • failed: the number of log entries that are read from all shards of the source Logstore and failed to be transformed.

Shard consumption delay and shard consumption speed

These two metrics indicate the consumption delay and speed of each shard within a minute when the transformation task reads data from the source Logstore.Shard consumption delay and shard consumption speed
  • Consumption delay: the difference between the time when logs in a shard are consumed and the time when the latest logs are written to the shard. The two time points are calculated based on the log receiving time.
  • Shard consumption speed: the average number of log entries that are read by a shard per second within a minute.
Note The delay of real-time consumption of logs is about 1 second. When historical logs are transformed, the shard consumption delay is high in the beginning, but is eventually reduced to a low level.

Active shard

This metric indicates the number of accepted, dropped, delivered, and failed log entries per second in each shard within a specified period of time.Active shard

Exception detail

In the reason field, you can view the code issues that may cause errors.Exception detail

You can also access the internal-etl-log Logstore in the current project and use the ERROR or WARNING keyword to view complete error logs.