Registers a data lineage relationship in DataWorks Data Map. You can use this operation to establish lineage relationships between metadata entities managed by DataWorks, including table-to-table, column-to-column, table-to-column, and dataset-to-table scenarios. You can also establish lineage relationships between managed entities and custom entity objects registered by users. This operation is compatible with non-managed custom objects, but this approach is no longer recommended. Before calling this operation, make sure that the managed entities involved in the lineage registration already exist on the DataWorks platform.
Operation description
DataWorks Professional Edition or a higher edition is required.
Try it now
Test
RAM authorization
Request parameters
|
Parameter |
Type |
Required |
Description |
Example |
| SrcEntity | LineageEntity |
No |
The source entity. |
|
| DstEntity | LineageEntity |
No |
The destination entity. |
|
| Task | LineageTask |
No |
The task information. |
Entity object (SrcEntity / DstEntity)
Describes the source and destination of a lineage relationship. You can register lineage relationships between the following types of entities:
| Entity type | Pre-registration required | How to obtain the ID | Description |
| DataWorks collected table | No | Call ListTables or GetTable and use the Id in the response. | Applicable to tables already collected by the DataWorks platform or extended tables registered by users. |
| DataWorks collected column | No | Call ListColumns or GetColumn and use the Id in the response. | Columns corresponding to collected tables. |
| Dataset | No, if the dataset already exists. To create a dataset, call CreateDataset first. | Call CreateDataset, ListDatasets, or GetDataset and use the Id in the response. | Applicable to DataWorks dataset entities. |
| Dataset version | No, if the version already exists. To create a version, call CreateDatasetVersion first. | Call CreateDatasetVersion, ListDatasetVersions, or GetDatasetVersion and use the Id in the response. | Applicable to specific versions of a dataset. |
| User-registered custom entity object | Yes | Call CreateMetaEntityDef to create a custom entity definition, then call BatchCreateMetaEntities to create entity objects, and use the Id in the response. | Applicable to custom entities that need to be managed and queried in DataWorks. |
| Non-managed custom object (not recommended) | No | The caller specifies an Id prefixed with custom- in the format custom-{Type}:{Identifier}. | Compatible with legacy usage but no longer recommended. For external reports, third-party system objects, and other custom objects, register them as DataWorks custom entity objects first and then register the lineage. |
Additional notes:
For tables, columns, datasets, and dataset versions already collected by DataWorks, do not manually construct the
Id. Use theIdreturned by the correspondingGetorListAPI.- For custom objects, use the user-registered custom entity object approach. Complete the entity definition and entity object registration first:
Call
CreateMetaEntityDefto create a custom entity definition.Call
BatchCreateMetaEntitiesto create specific entity objects.Use the entity
Idreturned byBatchCreateMetaEntitiesasSrcEntity.IdorDstEntity.Id.
For non-managed custom objects, you do not need to call
CreateMetaEntityDeforBatchCreateMetaEntities. Simply pass an entityIdprefixed withcustom-and specify theName. This approach is mainly for backward compatibility with legacy scenarios and is no longer recommended. For new integrations, use the registered custom entity object approach for easier querying, management, and custom attribute extension.SrcEntity.AttributesandDstEntity.Attributesrepresent extended attributes of the entity endpoints. Place attributes of the lineage relationship itself inTask.Attributes.
Task information (Task)
Describes the job or task that produces the lineage relationship.
| Parameter | Type | Required | Description |
| Id | String | No | The unique identifier of the task. We recommend that you pass a stable value for idempotent registration, querying, and deletion. Only letters, digits, and underscores are supported. Maximum length: 64 characters. |
| Type | String | Yes | The task type. Must start with custom-. Only letters, digits, underscores, and hyphens are supported. Maximum length: 64 characters. |
| Attributes | Map<String,String> | No | The task attributes. A maximum of 20 attributes are supported. Complex values must be serialized as JSON strings. • Key: the attribute name (letters, digits, and underscores, ≤ 64 characters). • Value: the attribute value (≤ 128 characters). |
Request examples.
DataWorks collected table and custom entity object
{
"SrcEntity": {
"Id": "custom_entity-customer_api:api_001",
"Name": "api_001"
},
"DstEntity": {
"Id": "mysql-table:rm-xxx::demo_db::demo_table",
"Name": "demo_table"
},
"Task": {
"Id": "api_to_table_001",
"Type": "custom-lineage-task",
"Attributes": {
"scene": "api_to_table",
"owner": "zhangsan"
}
}
}
```.
### Non-managed custom object and DataWorks collected table
> This approach is mainly for backward compatibility with legacy usage and is no longer recommended. For new integrations, register custom entity objects by using `CreateMetaEntityDef` and `BatchCreateMetaEntities` first, and then use the returned entity `Id` to register the lineage.
```json
{
"SrcEntity": {
"Id": "custom-report:report_001",
"Name": "report_001"
},
"DstEntity": {
"Id": "maxcompute-table:::demo_project:default:demo_table",
"Name": "demo_table"
},
"Task": {
"Id": "report_to_table_001",
"Type": "custom-lineage-task",
"Attributes": {
"scene": "report_to_table"
}
}
}
```.
Response elements
|
Element |
Type |
Description |
Example |
|
object |
The response. |
||
| RequestId |
string |
The request ID. A unique identifier for the request. |
C99E2BE6-9DEA-5C2E-8F51-1DDCFEADE490 |
| Success |
boolean |
Indicates whether the request was successful. |
true |
| Id |
string |
The lineage relationship ID. |
4as3dasf654a |
Examples
Success response
JSON format
{
"RequestId": "C99E2BE6-9DEA-5C2E-8F51-1DDCFEADE490",
"Success": true,
"Id": "4as3dasf654a"
}
Error codes
See Error Codes for a complete list.
Release notes
See Release Notes for a complete list.