All Products
Search
Document Center

Simple Log Service:Manage a data transformation job

Last Updated:Jan 16, 2024

This topic describes how to manage a data transformation job in the Simple Log Service console. You can view the details and status of a job in the console. You can also start, stop, modify, and delete the job and configure alert rules.

View the details of a data transformation job

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

  3. In the left-side navigation pane, choose Job Management > Data Transformation.

  4. In the list of data transformation jobs, find and click the data transformation job that you want to manage.

  5. On the Data Transformation Overview page, view the details of the data transformation job.

    加工任务

    Parameter

    Description

    Source Logstore

    The name of the Logstore from which data is read for transformation.

    Status

    The status of the data transformation job. For more information, see View the status of a data transformation job.

    Job ID

    The ID of the data transformation job.

    Time Range

    The time range within which the data transformation job is run.

    Job Created At

    The time when the data transformation job was created.

    Job Modified At

    The time when the data transformation job was last modified.

    Job Ended At

    The time when the data transformation job ends.

    Note

    If you do not configure the Processing End Time parameter when you create a data transformation job, this parameter is left empty.

    Storage Target

    The name of the storage destination, the name of the Logstore in which the transformed data is stored, and the name of the project to which the Logstore belongs.

    Consumption Progress

    The data consumption progress of the data transformation job.

    Status

    The dashboard that is provided by Simple Log Service to display the statistics about the execution of the data transformation job. The statistics include the overall information, transformation rate, consumption latency, consumption rate, active shards, and exceptions. For more information, see Data transformation dashboard.

View the status of a data transformation job

You can view the status of a data transformation job on the Data Transformation Overview page. The following table lists the job states and the operations that are supported in each state.

State\Operation

Stop the job

Start the job

Rerun the job

Modify the job

Delete the job

Starting

Not supported

Not supported

Not supported

Supported

Supported

Running

Supported

Not supported

Supported

Supported

Supported

Stopping

Not supported

Not supported

Not supported

Supported

Supported

Terminated

Not supported

Supported

Supported

Supported

Supported

Success

Not supported

Not supported

Supported

Supported

Supported

Failed

Not supported

Not supported

Supported

Supported

Supported

Stop a data transformation job

If a data transformation job is in the Running state, choose Actions > Stop on the Data Transformation Overview page of the job.

Note

If a data transformation job is stopped, the system records the checkpoint at which the job is stopped. If the job is started again, data transformation is resumed from this checkpoint. If you want to resume data transformation from the beginning, you must rerun the job. For more information, see Rerun a data transformation job.

Start a data transformation job

If a data transformation job is in the Terminated state, click Start on the Data Transformation Overview page of the job.

Rerun a data transformation job

You can rerun a data transformation job regardless of the job status. To rerun a data transformation job, choose Actions > Rerun on the Data Transformation Overview page of the job.

Note

If you rerun a data transformation job, the job is run from the beginning. If you want to continue running a data transformation job from the checkpoint at which the job was stopped, you must stop the job and then start the job.

Modify the transformation rule of a data transformation job

Note

After you modify the transformation rule of a data transformation job, you can perform the following operations:

  • If you want to use the new transformation rule to continue running the job, you must stop the job and then start the job.

  • If you want to use the new transformation rule to rerun the job, you must run the job from the beginning. For more information, see Rerun a data transformation job.

  1. On the Data Transformation Overview page, click Edit Rule.

  2. Modify the transformation rule based on your business requirements.

    For more information about the rule syntax, see Language introduction.

  3. Preview data.

    1. Select Quick.

      You can select Quick or Advanced. For more information, see Preview mode overview.

    2. Click Preview Data.

    3. View the results.

      • If data fails to be transformed because the syntax of the transformation rule or configured permissions are invalid, troubleshoot the failure as prompted.

      • If the transformed data is returned as expected, go to Step 4.

  4. Click Modify Transformation Settings.

  5. Modify the parameter settings and click OK.

    For more information about the parameters, see Create a data transformation job.

Delete a data transformation job

Warning

After you delete a data transformation job, the job cannot be restored. Proceed with caution.

You can delete a data transformation job by using one of the following methods:

  • In the list of data transformation jobs, move the pointer over the data transformation job that you want to delete and choose 更多操作 > Delete.

  • On the Data Transformation Overview page, choose Actions > Delete.

Configure alert rules

In the Data Transformation Troubleshooting dashboard of the Data Transformation Overview page, you can view the metrics of the data transformation job, subscribe to the dashboard, or enable monitoring and configure alert rules for the metrics. For more information, see Enable monitoring for data transformation jobs.