Job management - - Alibaba Cloud Documentation Center

This document describes the SQL syntax for job management in the Lindorm Stream Engine, including creating, listing, viewing the details of, and terminating streaming jobs.

Prerequisites

Before you begin, ensure that you have:

Stream engine version 3.1.8 or later. CREATE JOB applies only to the stream engine. To check the version or perform a minor version upgrade, go to the Lindorm console.

CREATE JOB

CREATE JOB packages one or more Flink SQL statements into a named job and submits it to the Lindorm stream engine for execution. Use it to implement stream processing logic such as filtering, transformation, enrichment, and aggregation, with results written to Lindorm.

Syntax

delimiter $$
CREATE JOB job_name
(
    flink_sqls
)
$$
delimiter ;

flink_sqls is one or more semicolon-terminated Flink SQL statements (such as SET, CREATE TABLE, and INSERT INTO) that define the computation logic. All statements are executed together as a single job.

Parameters

Parameter	Required	Type	Constraints
`job_name`	Yes	String	Letters, numbers, periods (`.`), hyphens (`-`), and underscores (`_`) only. Cannot start with `.` or `-`. Length: 1–255 characters.
`flink_sqls`	Yes	Flink SQL	One or more semicolon-terminated Flink SQL statements. For syntax details, see the Flink community documentation.

Example

The following job generates random data using the datagen connector and prints it to the console using the print connector.

delimiter $$
CREATE JOB datagen_job (
  SET 'parallelism.default' = '6';
  CREATE TABLE datagen (
    f_sequence INT,
    f_random INT,
    f_random_str STRING,
    ts AS localtimestamp,
    WATERMARK FOR ts AS ts
  ) WITH (
    'connector' = 'datagen',
    -- optional options --
    'rows-per-second'='5',
    'fields.f_sequence.kind'='sequence',
    'fields.f_sequence.start'='1',
    'fields.f_sequence.end'='50000000',
    'fields.f_random.min'='1',
    'fields.f_random.max'='500',
    'fields.f_random_str.length'='10'
  );

  CREATE TABLE print_table (
    f_sequence INT,
    f_random INT,
    f_random_str STRING
    ) WITH (
    'connector' = 'print'
  );

  INSERT INTO print_table select f_sequence,f_random,f_random_str from datagen;
)
$$
delimiter ;

Verify the job

Run SHOW JOBS; to confirm the job was created.

SHOW JOBS

SHOW JOBS lists all stream jobs in the current resource group. Use it to check job status after submitting a stream job, or to retrieve a Query ID for use in other job management statements.

Syntax

SHOW JOBS

Example

List all stream jobs in the current resource group:

SHOW JOBS;

Output:

+-----------------+------------+-----------+
| Query ID        | Job Status | Job Start |
+-----------------+------------+-----------+
| test_datagen2   | RUNNING    | 310029    |
| filter1         | RUNNING    | 1732491   |
| test_datagen    | RUNNING    | 540327    |
| sync_task       | RUNNING    | 47336395  |
| datagen_job     | RUNNING    | 87726     |
| filter2         | CANCELED   | 38779     |
+-----------------+------------+-----------+

The output includes the following columns:

Column	Description
`Query ID`	The unique identifier of the stream job.
`Job Status`	The current state of the job.
`Job Start`	The elapsed time since the job started.

DESCRIBE

Use DESCRIBE (or DESC) to display the details of a Flink job, including its status, runtime duration, and full SQL definition.

Syntax

{ DESCRIBE | DESC } name

name is the name of the Flink job. To list all jobs in the current resource group, run SHOW JOBS.

Example

Display the details of the datagen_job job:

DESC datagen_job \G;

Output:

*************************** 1. row ***************************
       Query ID: datagen_job
         Job ID: 0021dcbf7633ed1b4c5a64a59591****
         Status: RUNNING
   Is Stoppable: false
     Start time: 1753952195254
       End time: -1
       Duration: 61348039
Max Parallelism: -1
        DelayMs: -1.0
            Sql:
  SET 'parallelism.default' = '6';
  CREATE TABLE datagen (
    f_sequence INT,
    f_random INT,
    f_random_str STRING,
    ts AS localtimestamp,
    WATERMARK FOR ts AS ts
  ) WITH (
    'connector' = 'datagen',

    'rows-per-second'='5',
    'fields.f_sequence.kind'='sequence',
    'fields.f_sequence.start'='1',
    'fields.f_sequence.end'='50000000',
    'fields.f_random.min'='1',
    'fields.f_random.max'='500',
    'fields.f_random_str.length'='10'
  );

  CREATE TABLE print_table (
    f_sequence INT,
    f_random INT,
    f_random_str STRING
    ) WITH (
    'connector' = 'print'
  );

  INSERT INTO print_table select f_sequence,f_random,f_random_str from datagen;

Append \G to display results in vertical format for easier reading.

Output fields

Field	Description
Query ID	The name of the Flink job, as specified in the `DESCRIBE` statement.
Job ID	The unique identifier assigned to the job by the stream engine.
Status	The current job status. Common values: `RUNNING`, `FINISHED`, `FAILED`, `CANCELED`.
Is Stoppable	Whether the job can be stopped with a savepoint. `false` means the job does not support savepoint-based stops.
Start time	The time the job started, as a Unix timestamp in milliseconds.
End time	The time the job ended, as a Unix timestamp in milliseconds. `-1` means the job is still running.
Duration	How long the job has been running, in milliseconds.
Max Parallelism	The maximum parallelism configured for the job. `-1` means no explicit limit is set.
DelayMs	The processing delay in milliseconds. `-1.0` means no delay data is available.
Sql	The full SQL definition submitted with the job.

What's next

To view job details without SQL, navigate to the Jobs page in the Lindorm console.
To list all jobs in the current resource group, use SHOW JOBS.

TERMINATE

The TERMINATE JOB statement stops a running stream job. Stream jobs run continuously until explicitly stopped, so use this statement to stop a job when it is no longer needed.

Syntax

TERMINATE JOB job_name

Usage notes

job_name is the name of the stream job to stop. The job must be in RUNNING status. Use the DESCRIBE statement to confirm the job status before running TERMINATE JOB.

Example

Stop the stream job test_datagen2:

TERMINATE JOB test_datagen2;

Verify the result

Run SHOW JOBS; to check the job status. After the statement executes successfully, Job Status for test_datagen2 is CANCELED.

job_name      | Job Status | ...
--------------+------------+----
test_datagen2 | CANCELED   | ...