This topic describes how to use the interval join.
Background information
The interval join allows two data streams to be joined on a common key. It connects elements from each stream when their timestamps fall within a specified relative time interval. After the two data streams are joined, timestamp columns in the input data streams are retained, which allows for further event-time processing.
Syntax
SELECT column-names
FROM table1 [AS <alias1>]
[INNER | LEFT | RIGHT |FULL ] JOIN table2
ON table1.column-name1 = table2.key-name1 AND TIMEBOUND_EXPRESSIONINNER JOIN, LEFT JOIN, RIGHT JOIN, and FULL JOIN are supported. If you use JOIN, INNER JOIN is executed by default.
SEMI JOIN and ANTI JOIN are not supported.
TIMEBOUND_EXPRESSION is an interval conditional expression on the columns of the time attributes of the two data streams. The following conditional expressions are supported:
ltime = rtime
ltime >= rtime AND ltime < rtime + INTERVAL '10' MINUTE
ltime BETWEEN rtime - INTERVAL '10' SECOND AND rtime + INTERVAL '5' SECOND
Example
This example shows items shipped within 4 hours of order placement.
Test data
Order table (orders)
id
productName
orderTime
1
phone
2024-04-01 10:00:00.0
2
laptop
2024-04-01 10:02:00.0
3
watch
2024-04-01 10:03:00.0
4
tablet
2024-04-01 10:05:00.0
Logistics table (Shipments)
shipId
orderId
status
shiptime
0
1
shipped
2024-04-01 11:00:00.0
1
2
delivered
2024-04-01 17:00:00.0
2
3
shipped
2024-04-01 12:00:00.0
3
4
shipped
2024-04-01 11:30:00.0
Test statements
CREATE TEMPORARY TABLE Orders( id BIGINT, productName VARCHAR, ordertime TIMESTAMP(3), WATERMARK wk FOR ordertime as withOffset(ordertime, 2000) -- Define the ordertime column as the event time attribute of the table and use a 2 seconds delayed watermark strategy. ) WITH ( 'connector' = 'kafka', 'topic' = '<yourTopic>', 'properties.bootstrap.servers' = '<yourBrokers>', 'scan.startup.mode' = 'earliest-offset', 'format' = 'csv' ); CREATE TEMPORARY TABLE Shipments( shipId BIGINT, orderId BIGINT, status VARCHAR, shiptime TIMESTAMP(3), WATERMARK wk FOR shiptime as withOffset(shiptime, 2000) -- Define the ordertime column as the event time attribute of the table and use a 2 seconds delayed watermark strategy. ) WITH ( 'connector' = 'kafka', 'topic' = '<yourTopic>', 'properties.bootstrap.servers' = '<yourBrokers>', 'scan.startup.mode' = 'earliest-offset', 'format' = 'csv' ); -- Create a MySQL sink. CREATE TEMPORARY TABLE rds_output( id BIGINT, productName VARCHAR, status VARCHAR ) WITH ( 'connector' = 'mysql', 'hostname' = '<yourHostname>', 'port' = '3306', 'username' = '<yourUsername>', 'password' = '<yourPassword>', 'database-name' = '<yourDatabaseName>', 'table-name' = '<yourTableName>' ); INSERT INTO rds_output SELECT id, productName, status FROM Orders AS o JOIN Shipments AS s on o.id = s.orderId AND o.ordertime BETWEEN s.shiptime - INTERVAL '4' HOUR AND s.shiptime;Test result
id(bigint)
productName(varchar)
status(varchar)
1
phone
shipped
3
watch
shipped
4
tablet
shipped