Scheduling XIHE SQL jobs manually is error-prone and difficult to scale as pipeline complexity grows. Use Apache DolphinScheduler — a distributed, open source workflow orchestration platform with a visual Directed Acyclic Graph (DAG) editor — to automate XIHE SQL job execution and manage large-scale data processing workflows against AnalyticDB for MySQL.
Prerequisites
Before you begin, ensure that you have:
-
DolphinScheduler installed. See DolphinScheduler documentation.
-
A MySQL driver (v8.0.0 or later) placed in the DolphinScheduler installation directory. Download it from mysql-connector-java.
-
The IP address of the DolphinScheduler server added to the IP address whitelist of your AnalyticDB for MySQL cluster. See IP address whitelists.
Schedule XIHE SQL jobs
Complete the following steps in order: create a data source, create a project, define and run a workflow, then verify execution results.
Step 1: Create a data source
-
Open the DolphinScheduler web interface. In the top navigation bar, click Datasource.
-
Click Create DataSource. In the Choose DataSource Type dialog box, select MYSQL.
-
In the CreateDataSource dialog box, configure the following parameters.
Parameter Required Description DataSource Required Select MYSQL. Datasource Name Required A name for the data source. IP Required The endpoint of the AnalyticDB for MySQL cluster. Log in to the AnalyticDB for MySQL console and go to the Cluster Information page to find the endpoint. Port Required The port number of the AnalyticDB for MySQL cluster. Set to 3306.User Name Required The database account name of the AnalyticDB for MySQL cluster. Database Name Required The database name of the AnalyticDB for MySQL cluster. For the full list of optional parameters, see MySQL datasource.
-
Click Test Connect. After the connection test passes, click Confirm.
Step 2: Create a project
-
In the top navigation bar, click Project.
-
Click Create Project.
-
In the Create Project dialog box, configure Project Name, Owned Users, and any other parameters. For details, see Project.
Step 3: Create a workflow
-
Click the name of the project you created. In the left navigation pane, choose Workflow > Workflow Definition.
-
Click Create Workflow to open the workflow DAG edit page.
-
In the node list on the left, select SQL and drag it onto the canvas.
-
In the Current node settings dialog box, configure the following parameters.
Parameter Required Description Datasource types Required Select MYSQL. Datasource instances Required Select the data source you created in Step 1. SQL Type Required The type of the SQL job. Valid values: Query and Non Query. SQL Statement Required The SQL statement to execute. -
Click Confirm.
-
In the upper-right corner of the page, click Save. In the Basic Information dialog box, set Workflow Name and any other parameters, then click Confirm.
Step 4: Run the workflow
-
Locate the workflow in the list. In the Operation column, click the
icon to publish the workflow. -
Click the
icon in the Operation column. -
In the Please set the parameters before starting dialog box, configure the run parameters.
-
Click Confirm to start the workflow.
Step 5: View execution results
-
In the left navigation pane, choose Task > Task Instance.
-
Locate the tasks for your workflow. In the Operation column, click the
icon to view the execution results and logs.