The Dataphin metadata warehouse manages business metadata and compute engine metadata within a Dataphin project space in the metadata warehouse tenant (OPS tenant). It consists of periodic data integration nodes, SQL script nodes, and Shell nodes. Metadata warehouse initialization configures the compute engine type and initializes metadata. You can use Databricks as the compute engine for this initialization.
Limits
Only accounts with the metadata warehouse tenant super administrator or system administrator role can perform initialization.
Keep the account and password of the metadata warehouse tenant super administrator or system administrator secure. Exercise caution when operating with the metadata warehouse tenant super administrator account.
Procedure
-
On the Dataphin home page, select Management Center > System Settings from the top menu bar.
-
In the navigation pane on the left, select System Operations And Maintenance > Metadata Warehouse Settings.
-
On the Metadata Warehouse Settings configuration wizard page, click Start.
-
When selecting the initialization engine type, choose the Databricks engine type.
ImportantIf the metadata warehouse has already been initialized, it will default to the last successfully initialized engine. Switching to an incompatible compute engine may render the administration feature unavailable.
-
Click Next.
-
On the Parameter Checking page, configure the parameters for the Databricks compute engine.
Parameter
Description
Authentication Type
Select Service Principal (M2M) or Personal Access Token (PAT).
-
Service Principal (M2M): Authenticates with a Service Principal. Requires the Service Principal and Secret.
-
Personal Access Token (PAT): Authenticates with a personal token. Requires the token of the personal account.
Server hostname
Enter the workspace URL in the format
{workspace-host-name}.cloud.databricks.com.Service Principal
Enter the Service Principal (Client ID).
NoteThis option is available only when the authentication method is Service Principal (M2M).
Secret
Enter the Client Secret.
NoteThis option is available only when the authentication method is Service Principal (M2M).
token
Enter the personal access token for the account.
NoteThis option is available only when the authentication method is Personal Access Token (PAT).
HTTP path
Select the HTTP path. The list displays options based on the authentication information you entered.
Catalog
Select the catalog. The list displays options based on the authentication information you entered.
Schema
Select the Databricks schema. The list displays options based on the authentication information you entered.
Meta Project
The project space for metadata production and processing logic. We recommend that you set it to dataphin_meta. Do not change this name during reinitialization, or the initialization will fail.
-
-
Click Test Connection. After a successful connection test, click Next.
-
On the initialization page, click Start.
NoteThe initialization process takes approximately 15 minutes.
-
When the page indicates that the execution is successful, click Finish to complete the configuration.
What to do next
After you initialize the system metadata, you can set the compute engine for the Dataphin instance. If the metadata warehouse compute engine is Databricks, you can set the compute engine for a business tenant to any engine type except MaxCompute. For more information, see Compute settings.