All Products
Search

This Product

    AnalyticDB:Use DolphinScheduler to schedule XIHE SQL jobs

    Document Center

    AnalyticDB:Use DolphinScheduler to schedule XIHE SQL jobs

    Last Updated:Sep 10, 2024

    DolphinScheduler is a distributed and extensible open source workflow orchestration platform with powerful Directed Acyclic Graph (DAG) visual interfaces. DolphinScheduler can help you efficiently run and manage workflows for large amounts of data. You can use DolphinScheduler to schedule XIHE SQL jobs of AnalyticDB for MySQL to improve data processing efficiency.

    Prerequisites

    • DolphinScheduler installed. For more information, see DolphinScheduler documentation.

    • A MySQL driver of V8.0.0 or later is configured in the installation directory of DolphinScheduler. For information about the download link, see mysql-connector-java.

    • The IP address of the server that runs DolphinScheduler is added to an IP address whitelist of an AnalyticDB for MySQL cluster. For more information, see IP address whitelists.

    Procedure

    1. Create a data source.

      1. Access the DolphinScheduler web interface. In the top navigation bar, click Datasource.

      2. Click Create DataSource. In the Choose DataSource Type dialog box, select MYSQL.

      3. In the CreateDataSource dialog box, configure the parameters that are described in the following table.

        Parameter

        Description

        DataSource

        Select MYSQL.

        Datasource Name

        The name of the data source.

        IP

        The endpoint of the AnalyticDB for MySQL cluster. You can log on to the AnalyticDB for MySQL console and go to the Cluster Information page to view the endpoint.

        Port

        The port number of the AnalyticDB for MySQL cluster. Set the value to 3306.

        User Name

        The name of the database account of the AnalyticDB for MySQL cluster.

        Database Name

        The name of the database of the AnalyticDB for MySQL cluster.

        Note

        Other parameters are optional. For more information, see MySQL.

      4. Click Test Connect. After the data source passes the connection test, click Confirm.

    2. Create a project.

      1. In the top navigation bar, click Project.

      2. Click Create Project.

      3. In the Create Project dialog box, configure the parameters such as Project Name and Owned Users. For more information, see Project.

    3. Create a workflow.

      1. Click the name of the created project. In the left-side navigation pane, choose Workflow > Workflow Definition to go to the Workflow Definition page.

      2. Click Create Workflow to go to the workflow DAG edit page.

      3. In the left-side list of the page, select SQL and drag it to the right-side canvas.

      4. In the Current node settings dialog box, configure the parameters that are described in the following table.

        Parameter

        Description

        Datasource types

        Select MYSQL.

        Datasource instances

        Select the data source that is created in Step 1.

        SQL Type

        The type of the SQL job. Valid values: Query and Non Query.

        SQL Statement

        The SQL statement.

      5. Click Confirm.

      6. In the upper-right corner of the page, click Save. In the Basic Information dialog box, configure the parameters such as Workflow Name.

      7. Click Confirm.

    4. Run the workflow.

      1. Find the created workflow and click the image icon in the Operation column to publish the workflow.

      2. Click the image icon in the Operation column.

      3. In the Please set the parameters before starting dialog box, configure the parameters.

      4. Click Confirm to run the workflow.

    5. View the details about the workflow.

      1. In the left-side navigation pane, choose Task > Task Instance.

      2. Find the tasks of the workflow and click the image icon in the Operation column to view the execution results and logs of the workflow.