Azkaban is an open-source batch workflow job scheduler that can be used to create, execute, and manage workflows that contain complex dependencies. Use it to run XIHE SQL jobs against AnalyticDB for MySQL on a recurring schedule, with dependency tracking between jobs.
Prerequisites
Before you begin, ensure that you have:
A MySQL client installed on the server where Azkaban runs. See MySQL documentation.
Azkaban installed and running. See Azkaban documentation.
The IP address of the Azkaban server added to the IP whitelist of your AnalyticDB for MySQL cluster. See Configure a whitelist.
Prepare test data
In your AnalyticDB for MySQL cluster, create a test database and table:
-- Create a database.
CREATE DATABASE azkaban_test;
-- Create a table.
CREATE TABLE azkaban_test.names (
id BIGINT,
name STRING
);
-- Insert data into the table.
INSERT INTO azkaban_test.names VALUES(1, 'Li'), (2, 'Yang');Step 1: Write a workflow file
An Azkaban workflow consists of .job files, each defining a single job. Each job declares its type, the shell command to run, and which jobs it depends on. You package all .job files into a ZIP file for upload.
This example defines three jobs that run in sequence: start → exec_query → end.
Job files
start.job — entry point of the workflow:
#start
type=command
command=echo 'job start'exec_query.job — runs the SQL statement against AnalyticDB for MySQL after start completes:
type=command
dependencies=start
command=mysql -h<endpoint> -u<username> -p<password> -P3306 -e "create table azkaban_test.duplicated_names as select * from azkaban_test.names"end.job — signals workflow completion after exec_query completes:
#end
type=command
dependencies=exec_query
command=echo 'job end'The key fields in each .job file are:
| Field | Description |
|---|---|
type=command | Runs the value of command as a shell command |
dependencies | Comma-separated list of job names that must complete before this job starts |
command | The shell command to run |
The MySQL command in exec_query.job uses the following parameters:
| Parameter | Description |
|---|---|
-h | Endpoint of the AnalyticDB for MySQL cluster. Find it on the Cluster Information page in the AnalyticDB for MySQL console. |
-u | Database account name for the cluster |
-p | Password for the database account |
-P | Port number. Set to 3306. |
-e | SQL statement to run |
Package the workflow
Create a folder, place all .job files in it, and compress the folder as a ZIP file.
Step 2: Create a project and upload the workflow file
Log in to the Azkaban console. In the top navigation bar, click Projects.
In the upper-right corner, click Create Project.
In the Create Project dialog box, enter a Name and Description, then click Create Project.
In the upper-right corner, click Upload.
In the Upload Project Files dialog box, select the ZIP file you created, then click Upload.
Step 3: Execute the workflow
On the Projects page, click the Flows tab.
Click Execute Flow.
In the dialog box, click Execute.
In the Flow submitted notification, click Continue.
Step 4: View execution details
In the top navigation bar, click Executing.
Click the Recently Finished tab to see completed executions.
Click an execution ID to open the execution detail page.
On the Job List tab, review the status and duration of each job.
Click Logs next to a job to view its full execution log.