The Dataphin system automatically analyzes data lineage and validates the change type and content of specification modeling tasks upon submission. It also conducts a preliminary check to ensure compliance with submission standards, thereby minimizing the risk of errors.
Data lineage parsing description
The system analyzes data lineage during task submission in the development environment and upon publishing in the production environment. It handles up to 100,000 entries; entries beyond this limit will not be recorded or displayed in the asset directory.
Dimension/fact logical table task
Submission details description
For dimension/fact logical table tasks, you can review the submission details, pre-check results, and add submission notes in the Submit dialog box.
Virtual and enumeration dimension logical tables do not support detailed submission views.
Submission Content
The submission content includes the task object's name, type, change type, and details. Change details encompass basic information, object information, table structure, physical information, table relationships, computing logic, scheduling configuration, dependency settings, and runtime configuration.
Pre-check
The system performs several pre-checks when submitting a dimension/fact logical table task. If any checks fail, the submission will not proceed. The following table, using the dimension logical table submission dialog box as an example, details these checks.
Check Item
Description
Table Structure
Verifies the configuration of the table structure.
Computing Logic
Checks the configuration of the computing logic.
Schedule Dependency
Ensures at least one upstream dependency is set, excluding self-dependency. For more information, see Configure logical table scheduling properties.
Runtime Parameters
Checks for assigned values to local variables. If unassigned, submission is not supported. For more information, see Logical table runtime parameter configuration.
Cross-node Parameters
Passes through variable parameters to direct downstream nodes. For more information, see Cross-node parameters.
Submission Remarks
Allows for the inclusion of remarks with the task submission, limited to 128 characters.
Check item description
Post-submission, the Submit dialog box displays the results of the task checks, as shown in the dimension logical table Submit dialog box example below.
Check item | Description |
Computing logic | Verify the configuration of the logical table's computing logic for accuracy and completeness. |
Parameter configuration check | The system lists all parsed variables from the code, including names, types, and current values, to ensure local variables are properly assigned. |
Permission verification | The system checks each object in the logical table for operation permissions, detailing object names, types, permission status, and request operations.
|
Change compatibility check | Assess whether modifications to the logical table model will impact the availability of downstream objects or historical data, including renamed references and compatibility of altered fields. Note If compatibility alerts arise and no action is needed, select Ignore Alert And Continue Submission to proceed. |
Schedule dependency check | Examine the task's dependency and output configurations for accuracy.
|
Operation execution | This check initiates the task submission process, which cannot be cancelled once operation execution begins. |
to initiate a permission request. For detailed instructions, see Atomic metric/business filter task
Submission details description
When submitting an atomic metric/business filter task, you can review the task's submission details and enter remarks in the Submit dialog box. Below is an example using the business filter Submit dialog box.
Submission Content
The submission content includes the task object's name, type, change type, and details of the change, which encompass basic information, object information, and computing logic.
Submission Remarks
You can provide remarks for the task submission, limited to 128 characters.
Check item description
After submitting the task, you can view the results of the task's check items in the Submitting Log dialog box. The image below shows the business filter Submitting Log dialog box as an example.
Check Item | Description |
Repetitiveness Check | This check verifies the uniqueness of the computing logic, ensuring no repetition occurs, based on the active timeliness settings. |
Type Verification | This process confirms that the output type of the computing logic aligns with the predefined type, in accordance with the active timeliness settings. |
Operation Execution | Variable configuration examines the variables within the code of a computing task, ensuring that local variables are properly set and parameter names are valid. For more information, see Offline mode variable configuration. |