This topic describes how to create a crawler to collect metadata from an AnalyticDB for PostgreSQL data source to DataWorks. You can manage the tables that store the collected AnalyticDB for PostgreSQL metadata on the Data Map page.

Prerequisites

An EMR cluster is associated with your workspace as a compute engine instance. For more information, see Associate an EMR compute engine instance with a workspace.

Limits

  • You cannot collect metadata across regions. You must create a crawler in the region where the source metadata resides to collect the metadata.
  • You must collect metadata over the Internet.

Procedure

  1. In the left-side navigation pane, click AnalyticDB for PostgreSQL.
  2. On the AnalyticDB for PostgreSQLMetadata Crawler page, click Create Crawler.
  3. In the Create Crawler dialog box, complete the wizard.
    1. In the Basic Information step, set the parameters.
      Basic Information step
      Parameter Description
      Crawler Name Required. The name of the crawler. You must set a unique name.
      Crawler Description The description of the crawler.
      Workspace The workspace of the data source from which you want to collect metadata.
      Data Source Type The type of the data source from which you want to collect metadata. The default value is AnalyticDB for PostgreSQL and cannot be changed.
    2. In the Select Collection Object step, select a data source from the Data Source drop-down list.
      If no data source is available, click Create to go to the Data Source page and create an AnalyticDB for PostgreSQL data source. For more information, see Configure an AnalyticDB for PostgreSQL connection.
    3. Click Start Testing next to Test Crawler Connectivity. If the message The connectivity test is successful appears, the DataWorks metadata service can connect to the AnalyticDB for PostgreSQL data source.
    4. Click Next.
  4. On the AnalyticDB for PostgreSQLMetadata Crawler page, find the created crawler and click Run in the Actions column.
    After the crawler is run, click the number in the Updated Tables in Last Run or Added Tables in Last Run column to view the details of the updated or created tables.
    Notice The Run button is available only in the Actions column of a crawler that needs to be manually triggered.
    You can also perform the following operations on the AnalyticDB for PostgreSQLMetadata Crawler page:
    • Click Details in the Actions column of a crawler. In the Crawler Details dialog box, view the detailed information about the crawler.
    • Click Edit in the Actions column of a crawler. In the Edit Crawler dialog box, modify the configurations of the crawler.
    • Click Delete in the Actions column of a crawler. In the Confirm message, click Ok to delete the crawler.
    • Click Stop in the Actions column of a crawler that is running to stop the crawler.