Mapping rules define the association between data standards and asset objects. Adherence monitoring then checks whether these objects comply with the mapped standards. This topic describes the basic concepts and logic of adherence monitoring.
Background information
After you create a data standard, you can apply it by associating it with an asset object. Dataphin maps data standards to asset objects by defining the mapping between the properties of the data standard and the metadata fields of the asset object. This mapping is a prerequisite for monitoring the standard adherence of asset objects. For more information, see Create and manage standard adherence mapping rules.
Lost bid monitoring
Adherence monitoring checks whether mapped objects comply with the associated data standards. It includes metadata monitoring and content quality monitoring:
Metadata monitoring assesses whether the field metadata of an asset object is consistent with the property values defined in the data standard.
Content quality monitoring assesses whether the content of a mapped asset object, such as a field or metric, complies with the data standard's constraints, such as value range and uniqueness.
NoteContent quality monitoring requires the quality monitoring feature. To use this feature, you must purchase the asset quality module.
Metadata monitoring
To configure metadata monitoring, you must specify the field metadata to monitor and its corresponding data standard property. Then, you can configure the validation logic, such as rules for value equality and case sensitivity. Note: If a value cannot be retrieved from the data source for the specified field metadata, the adherence assessment cannot be performed.
For example, you want to use a data standard to constrain the data type in field metadata. The "member registration channel" data standard requires the "data type" to be STRING. However, the mapped field "registration_channel" has a data type of INT in the database. This does not comply with the data standard, and the monitoring check fails.
Dataphin provides built-in public system properties for common metadata monitoring. When you create a data standard set, you can reference these system properties. Data standards created in the set then inherit the monitoring configurations from the system properties by default. For more information, see Create and manage public properties for data standard sets.
Content quality monitoring
There are two ways to add content quality monitoring to asset objects based on a data standard:
The monitoring rule is linked to a data standard's property value, and the rule configuration is derived from this value. For example, if you are monitoring a field's range, the rule's range is derived from the value of the data standard's "range" property. With this method, the Rule configuration is automatically generated, and you only need to define the Validation configuration.
The content of the monitoring rule is independent of the content of the data standard. However, if a rule configuration is added when the data standard is created, all asset objects mapped to that data standard can reference the pre-configured monitoring rule. This allows for rapid batch configuration. With this method, both the Rule configuration and Validation configuration are defined manually. For more information, see Step 2: Configure quality rules.