Dataphin:Manage modeling tasks

Background information

• Modeling tasks include logical dimension table tasks, logical fact table tasks, and logical aggregate table tasks. Each logical table task corresponds to a logical table node. The system automatically configures the schedule based on parameters that you set, such as the statistical period. You can perform O&M on logical table tasks from a table or field perspective.

  Note

  You can view logical table tasks only in projects that support standardized modeling.

• Modeling tasks automatically generate multiple materialized nodes based on the materialization logic of standardized modeling. Each logical table node is a combination of these materialized nodes. Multiple fields or modeling task nodes can belong to the same materialized node.

Usage notes

Standardized modeling uses a dynamic materialization logic. Therefore, you can perform O&M from a physical or logical perspective:

• Physical perspective: You may need to view the latest code or operational logs of a materialized node to understand how an operation on the node affects the associated fields or logical table task nodes. For example, to backfill data for a modeling task node, you must first identify the materialized nodes to which the logical table task node belongs. Then, you must backfill data for each materialized node one by one. This process is complex and cannot ensure data consistency.

• Logical perspective: You only need to select the fields or logical table task nodes that require O&M. The system automatically calculates the corresponding materialized nodes based on the current materialization method. Based on the principles of data consistency and correctness, the system prompts you about related fields that require associated operations. You do not need to be concerned about the physical implementation. This helps you shorten the operational path, reduce O&M costs, and improve O&M efficiency.

Access the modeling task page

1. On the Dataphin home page, choose Develop > Task O&M from the top menu bar.

2. In the navigation pane on the left, choose Task O&M > Recurring Task.

3. In the top menu bar, select the production or development environment.

4. On the Recurring Task page, click the Modeling Task tab.

Operations in the modeling task list

After a modeling task is submitted to the Operation Center for scheduling, it appears in the Recurring Task > Modeling Task list. The list displays the task object, schedule, priority, owner, related baselines, project, last update time, tags, and supported operations.

• Recurrence: The time when the recurring task is scheduled to run in the specified time zone.

• Priority: The priority of the task. For a logical aggregate table, the priorities of all metrics in the table are displayed. Therefore, a logical aggregate table can have multiple priorities.

• Operation Owner: The O&M owner of the logical table task. For a logical aggregate table, the owners of all metrics in the table are displayed. Therefore, a logical aggregate table can have multiple owners.

• Related Baselines: The baselines to which the node belongs as a guaranteed leaf node and the related baselines that include the node as an ancestor node.

  Note

  This column is not displayed if the baseline feature is disabled.

• Project: The project to which the task belongs. The project is displayed in the Project English Name (Project Chinese Name) format. If a logical aggregate table spans multiple projects, the names of the projects are displayed and separated by commas (,).

The following operations are available in the modeling task list.

Operations for modeling task DAG nodes

The Directed Acyclic Graph (DAG) shows the dependencies between the task node and its ancestor and descendant nodes. You can also perform O&M on the ancestor and descendant nodes. By default, the DAG shows the main node (the selected node) and its first-level ancestor and descendant nodes. You can select a modeling task node to perform O&M operations on the task.

Dataphin supports cross-project O&M. To perform O&M operations on a cross-project modeling task node, you must have view and operation permissions for the project to which the metric belongs. For an aggregate table task, you can perform operations only on metrics for which you have the required permissions.

• Operations in the DAG

  Operation	Description
  Expand Parent Nodes	Expand the dependency nodes at different levels of the main node in the DAG.
  Expand Child Nodes
  Expand Materialized Nodes	Expand the materialized nodes of the current logical table task node to view the names of the materialized nodes, the fields in the materialized nodes, and the materialization code.
  View Operation Logs	View the operation logs of the current node.

• Operations for modeling nodes

  Hover over a DAG node to view its name, type, schedule, owner, and description. The operations for modeling DAG nodes are the same as the operations in the modeling task list. For more information, see Operations in the modeling task list.

  Note

  Because an aggregate table task does not belong to a specific project, you cannot configure monitoring and alerting for cross-project metrics and their aggregate tables within the current project.

Batch operations for modeling tasks

Modeling tasks support the following batch operations: