All Products
Search

This Product

    MaxCompute:JOIN

    Document Center

    MaxCompute:JOIN

    Last Updated:Dec 12, 2024

    MaxCompute allows you to use JOIN operations to join tables and return the data that meets join and query conditions. This topic describes the following JOIN operations: LEFT OUTER JOIN, RIGHT OUTER JOIN, FULL OUTER JOIN, INNER JOIN, NATURAL JOIN, implicit JOIN, and multiple JOIN operations.

    Overview

    MaxCompute supports the following types of JOIN operations:

    • LEFT OUTER JOIN

      It is also called LEFT JOIN. LEFT OUTER JOIN returns all rows in the left table, including the rows that do not match any rows in the right table.

      Note

      In a JOIN operation, the left table is a large table, and the right table is a small table in most cases. If the values in some rows of the right table are duplicate, we recommend that you do not perform multiple consecutive LEFT JOIN operations. If you perform multiple consecutive LEFT JOIN operations, data bloat may occur, and your jobs are interrupted.

    • RIGHT OUTER JOIN

      It is also called RIGHT JOIN. RIGHT OUTER JOIN returns all rows in the right table, including the rows that do not match any rows in the left table.

    • FULL OUTER JOIN

      It is also called FULL JOIN. FULL OUTER JOIN returns all rows in both the left and right tables.

    • INNER JOIN

      The INNER keyword can be omitted. INNER JOIN returns data rows if a match exists between the left and right tables.

    • NATURAL JOIN

      In a NATURAL JOIN operation, the fields that are used to join two tables are determined based on the common fields of the two tables. MaxCompute supports OUTER NATURAL JOIN. If you use the USING clause, the NATURAL JOIN operation returns common fields only once.

    • Implicit JOIN operation

      You can perform an implicit JOIN operation without the need to specify the JOIN keyword.

    • Multiple JOIN operations

      MaxCompute supports multiple JOIN operations. You can use parentheses () to specify the priorities of JOIN operations. A JOIN operation that is enclosed in parentheses () has a higher priority.

    Note

    • If an SQL statement contains the WHERE clause and you use the JOIN clause before the WHERE clause, the JOIN operation is performed first. Then, the results obtained from the JOIN operation are filtered based on the conditions specified by the WHERE clause. The final result is the intersection of two tables, not all rows in a table.

    • You can use the odps.task.sql.outerjoin.ppd parameter to control whether to use a non-JOIN condition in the OUTER JOIN ON clause as the input data of the JOIN operation. You can configure this parameter at the project or session level.

      • If you set this parameter to false, the non-JOIN condition in the ON clause is considered as the condition in the WHERE clause for the subquery of the JOIN operation. This is non-standard behavior. We recommend that you specify the non-JOIN condition in the WHERE clause.

      • If you set this parameter to false, the following SQL statements are equivalent. If you set this parameter to true, the two SQL statements are not equivalent. 

      SELECT A.*, B.* FROM A LEFT JOIN B ON A.c1 = B.c1 and A.c2='xxx';

SELECT A.*, B.* FROM (SELECT * FROM A WHERE c2='xxx') A LEFT JOIN B ON A.c1 = B.c1;

    Usage notes

    In a JOIN operation, the filter condition key is not null of the JOIN operation is automatically added for calculation. The row whose value of the join key is null is filtered out after the JOIN operation.

    Limits

    When you perform a JOIN operation, take note of the following limits:

    • MaxCompute does not support CROSS JOIN. A CROSS JOIN operation joins two tables without requiring you to specify conditions in the ON clause.

    • You must use equi-joins and combine conditions by using AND. You can use non-equi joins or combine multiple conditions by using OR in a MAPJOIN operation. For more information, see MAPJOIN.

    Syntax

    <table_reference> JOIN <table_factor> [<join_condition>]
| <table_reference> {LEFT OUTER|RIGHT OUTER|FULL OUTER|INNER|NATURAL} JOIN <table_reference> <join_condition>

    • table_reference: required. The query statement for the left table on which the JOIN operation is performed. The value of this parameter is in the table_name [alias] | table_query [alias] |... format.

    • table_factor: required. The query statement for the right table or a table on which the JOIN operation is performed. The value of this parameter is in the table_name [alias] | table_subquery [alias] |... format.

    • join_condition: optional. A JOIN condition is a combination of one or more equality expressions. The value of this parameter is in the on equality_expression [and equality_expression]... format. equality_expression is an equality expression.

    Note

    If partition pruning conditions are specified in the WHERE clause, partition pruning takes effect on both the parent and child tables. If partition pruning conditions are specified in the ON clause, partition pruning takes effect only on the child table. As a result, a full table scan is run for the parent table. For more information, see Check whether partition pruning is effective.

    Sample data

    Sample source data is provided for you to better understand the examples in this topic. The following statements show how to create the sale_detail and sale_detail_jt tables and insert data into the tables.

    -- Create two partitioned tables named sale_detail and sale_detail_jt. 
CREATE TABLE if NOT EXISTS sale_detail
(
shop_name     STRING,
customer_id   STRING,
total_price   DOUBLE
)
PARTITIONED BY (sale_date STRING, region STRING);

CREATE TABLE if NOT EXISTS sale_detail_jt
(
shop_name     STRING,
customer_id   STRING,
total_price   DOUBLE
)
PARTITIONED BY (sale_date STRING, region STRING);

-- Add partitions to the two tables. 
ALTER TABLE sale_detail ADD PARTITION (sale_date='2013', region='china') PARTITION (sale_date='2014', region='shanghai');
ALTER TABLE sale_detail_jt ADD PARTITION (sale_date='2013', region='china');

-- Insert data into the tables. 
INSERT INTO sale_detail PARTITION (sale_date='2013', region='china') VALUES ('s1','c1',100.1),('s2','c2',100.2),('s3','c3',100.3);
INSERT INTO sale_detail PARTITION (sale_date='2014', region='shanghai') VALUES ('null','c5',null),('s6','c6',100.4),('s7','c7',100.5);
INSERT INTO sale_detail_jt PARTITION (sale_date='2013', region='china') VALUES ('s1','c1',100.1),('s2','c2',100.2),('s5','c2',100.2);

-- Query data from the sale_detail and sale_detail_jt tables. Sample statements:
SET odps.sql.allow.fullscan=true;
SELECT * FROM sale_detail;
-- The following result is returned:
+------------+-------------+-------------+------------+------------+
| shop_name  | customer_id | total_price | sale_date  | region     |
+------------+-------------+-------------+------------+------------+
| s1         | c1          | 100.1       | 2013       | china      |
| s2         | c2          | 100.2       | 2013       | china      |
| s3         | c3          | 100.3       | 2013       | china      |
| null       | c5          | NULL        | 2014       | shanghai   |
| s6         | c6          | 100.4       | 2014       | shanghai   |
| s7         | c7          | 100.5       | 2014       | shanghai   |
+------------+-------------+-------------+------------+------------+

SET odps.sql.allow.fullscan=true;
SELECT * FROM sale_detail_jt;
-- The following result is returned:
+------------+-------------+-------------+------------+------------+
| shop_name  | customer_id | total_price | sale_date  | region     |
+------------+-------------+-------------+------------+------------+
| s1         | c1          | 100.1       | 2013       | china      |
| s2         | c2          | 100.2       | 2013       | china      |
| s5         | c2          | 100.2       | 2013       | china      |
+------------+-------------+-------------+------------+------------+

-- Create a table for the JOIN operation. 
SET odps.sql.allow.fullscan=true;
CREATE TABLE shop AS SELECT shop_name, customer_id, total_price FROM sale_detail;

    Examples

    The following examples shows the usage of JOIN based on the Sample data.

    • Example 1: LEFT OUTER JOIN. Sample statements:

      -- The full table scan feature must be enabled for partitioned tables. Otherwise, the JOIN operation fails. 
SET odps.sql.allow.fullscan=true;
-- Both the sale_detail_jt and sale_detail tables have the shop_name column. You must use aliases to distinguish between the columns in the SELECT statement. 
SELECT a.shop_name AS ashop, b.shop_name AS bshop FROM sale_detail_jt a 
       LEFT OUTER JOIN sale_detail b ON a.shop_name=b.shop_name;

      The following result is returned:

      +------------+------------+
| ashop      | bshop      |
+------------+------------+
| s2         | s2         |
| s1         | s1         |
| s5         | NULL       |
+------------+------------+

    • Example 2: RIGHT OUTER JOIN. Sample statements:

      -- The full table scan feature must be enabled for partitioned tables. Otherwise, the JOIN operation fails. 
SET odps.sql.allow.fullscan=true;
-- Both the sale_detail_jt and sale_detail tables have the shop_name column. You must use aliases to distinguish between the columns in the SELECT statement. 
SELECT a.shop_name AS ashop, b.shop_name AS bshop FROM sale_detail_jt a 
       RIGHT OUTER JOIN sale_detail b ON a.shop_name=b.shop_name;

      The following result is returned:

      +------------+------------+
| ashop      | bshop      |
+------------+------------+
| s1         | s1         |
| s2         | s2         |
| NULL       | s3         |
| NULL       | null       |
| NULL       | s6         |
| NULL       | s7         |
+------------+------------+

    • Example 3: FULL OUTER JOIN. Sample statements:

      -- The full table scan feature must be enabled for partitioned tables. Otherwise, the JOIN operation fails. 
SET odps.sql.allow.fullscan=true;
-- Both the sale_detail_jt and sale_detail tables have the shop_name column. You must use aliases to distinguish between the columns in the SELECT statement. 
SELECT a.shop_name AS ashop, b.shop_name AS bshop FROM sale_detail_jt a 
       FULL OUTER JOIN sale_detail b ON a.shop_name=b.shop_name;

      The following result is returned:

      +------------+------------+
| ashop      | bshop      |
+------------+------------+
| NULL       | s3         |
| NULL       | s6         |
| s2         | s2         |
| NULL       | null       |
| NULL       | s7         |
| s1         | s1         |
| s5         | NULL       |
+------------+------------+

    • Example 4: INNER JOIN. Sample statements:

      -- The full table scan feature must be enabled for partitioned tables. Otherwise, the JOIN operation fails. 
SET odps.sql.allow.fullscan=true;
-- Both the sale_detail_jt and sale_detail tables have the shop_name column. You must use aliases to distinguish between the columns in the SELECT statement. 
SELECT a.shop_name AS ashop, b.shop_name AS bshop FROM sale_detail_jt a 
       INNER JOIN sale_detail b ON a.shop_name=b.shop_name;

      The following result is returned:

      +------------+------------+
| ashop      | bshop      |
+------------+------------+
| s2         | s2         |
| s1         | s1         |
+------------+------------+

    • Example 5: NATURAL JOIN. Sample statements:

      -- The full table scan feature must be enabled for partitioned tables. Otherwise, the JOIN operation fails. 
SET odps.sql.allow.fullscan=true;
-- Perform a NATURAL JOIN operation. 
SELECT * FROM sale_detail_jt NATURAL JOIN sale_detail;
-- The preceding statement is equivalent to the following statement: 
SELECT sale_detail_jt.shop_name AS shop_name, sale_detail_jt.customer_id AS customer_id, sale_detail_jt.total_price AS total_price, sale_detail_jt.sale_date as sale_date, sale_detail_jt.region as region from sale_detail_jt 
INNER JOIN sale_detail 
ON sale_detail_jt.shop_name=sale_detail.shop_name AND sale_detail_jt.customer_id=sale_detail.customer_id and sale_detail_jt.total_price=sale_detail.total_price AND sale_detail_jt.sale_date=sale_detail.sale_date AND sale_detail_jt.region=sale_detail.region;

      The following result is returned:

      +------------+-------------+-------------+------------+------------+
| shop_name  | customer_id | total_price | sale_date  | region     |
+------------+-------------+-------------+------------+------------+
| s1         | c1          | 100.1       | 2013       | china      |
| s2         | c2          | 100.2       | 2013       | china      |
+------------+-------------+-------------+------------+------------+

    • Example 6: implicit JOIN. Sample statements:

      -- The full table scan feature must be enabled for partitioned tables. Otherwise, the JOIN operation fails. 
SET odps.sql.allow.fullscan=true;
-- Perform an implicit JOIN operation. 
SELECT * FROM sale_detail_jt, sale_detail WHERE sale_detail_jt.shop_name = sale_detail.shop_name;
-- The preceding statement is equivalent to the following statement: 
SELECT * FROM sale_detail_jt JOIN sale_detail ON sale_detail_jt.shop_name = sale_detail.shop_name;

      The following result is returned:

      +------------+-------------+-------------+------------+------------+------------+--------------+--------------+------------+------------+
| shop_name  | customer_id | total_price | sale_date  | region     | shop_name2 | customer_id2 | total_price2 | sale_date2 | region2    |
+------------+-------------+-------------+------------+------------+------------+--------------+--------------+------------+------------+
| s2         | c2          | 100.2       | 2013       | china      | s2         | c2           | 100.2        | 2013       | china      |
| s1         | c1          | 100.1       | 2013       | china      | s1         | c1           | 100.1        | 2013       | china      |
+------------+-------------+-------------+------------+------------+------------+--------------+--------------+------------+------------+

    • Example 7: Multiple JOIN operations. No priority is specified. Sample statements:

      -- The full table scan feature must be enabled for partitioned tables. Otherwise, the JOIN operation fails. 
SET odps.sql.allow.fullscan=true;
-- Both the sale_detail_jt and sale_detail tables have the shop_name column. You must use aliases to distinguish between the columns in the SELECT statement. 
SELECT a.* FROM sale_detail_jt a FULL OUTER JOIN sale_detail b ON a.shop_name=b.shop_name
        FULL OUTER JOIN sale_detail c ON a.shop_name=c.shop_name;

      The following result is returned:

      +------------+-------------+-------------+------------+------------+
| shop_name  | customer_id | total_price | sale_date  | region     |
+------------+-------------+-------------+------------+------------+
| s5         | c2          | 100.2       | 2013       | china      |
| NULL       | NULL        | NULL        | NULL       | NULL       |
| NULL       | NULL        | NULL        | NULL       | NULL       |
| NULL       | NULL        | NULL        | NULL       | NULL       |
| NULL       | NULL        | NULL        | NULL       | NULL       |
| NULL       | NULL        | NULL        | NULL       | NULL       |
| NULL       | NULL        | NULL        | NULL       | NULL       |
| s1         | c1          | 100.1       | 2013       | china      |
| NULL       | NULL        | NULL        | NULL       | NULL       |
| s2         | c2          | 100.2       | 2013       | china      |
| NULL       | NULL        | NULL        | NULL       | NULL       |
+------------+-------------+-------------+------------+------------+

    • Example 8: Multiple JOIN operations. Use parentheses () to specify the priorities of JOIN operations. Sample statements:

      -- The full table scan feature must be enabled for partitioned tables. Otherwise, the JOIN operation fails. 
SET odps.sql.allow.fullscan=true;
-- Perform multiple JOIN operations. Use parentheses () to specify the priority. 
SELECT * FROM shop JOIN (sale_detail_jt JOIN sale_detail ON sale_detail_jt.shop_name = sale_detail.shop_name) ON shop.shop_name=sale_detail_jt.shop_name;

      The following result is returned:

      +------------+-------------+-------------+------------+------------+------------+--------------+--------------+------------+------------+------------+--------------+--------------+------------+------------+
| shop_name  | customer_id | total_price | sale_date  | region     | shop_name2 | customer_id2 | total_price2 | sale_date2 | region2    | shop_name3 | customer_id3 | total_price3 | sale_date3 | region3    |
+------------+-------------+-------------+------------+------------+------------+--------------+--------------+------------+------------+------------+--------------+--------------+------------+------------+
| s2         | c2          | 100.2       | 2013       | china      | s2         | c2           | 100.2        | 2013       | china      | s2         | c2           | 100.2        | 2013       | china      |
| s1         | c1          | 100.1       | 2013       | china      | s1         | c1           | 100.1        | 2013       | china      | s1         | c1           | 100.1        | 2013       | china      |
+------------+-------------+-------------+------------+------------+------------+--------------+--------------+------------+------------+------------+--------------+--------------+------------+------------+

    • Example 9: Use JOIN and WHERE to query the number of records whose region is china and whose shop_name field has the same value in the two tables. All records in the sale_detail table are retained. Sample statements:

      -- The full table scan feature must be enabled for partitioned tables. Otherwise, the JOIN operation fails. 
SET odps.sql.allow.fullscan=true;
-- Execute the following SQL statement: 
SELECT a.shop_name
    ,a.customer_id
    ,a.total_price
        ,b.total_price 
FROM  (SELECT * FROM sale_detail WHERE region = "china") a 
LEFT JOIN (SELECT * FROM sale_detail_jt WHERE region = "china") b 
ON   a.shop_name = b.shop_name;

      The following result is returned:

      +------------+-------------+-------------+--------------+
| shop_name  | customer_id | total_price | total_price2 |
+------------+-------------+-------------+--------------+
| s1         | c1          | 100.1       | 100.1        |
| s2         | c2          | 100.2       | 100.2        |
| s3         | c3          | 100.3       | NULL         |
+------------+-------------+-------------+--------------+

      Sample statement of incorrect usage:

      SELECT a.shop_name
    ,a.customer_id
    ,a.total_price
        ,b.total_price 
FROM  sale_detail a 
LEFT JOIN sale_detail_jt b 
ON   a.shop_name = b.shop_name 
WHERE  a.region = "china" AND b.region = "china";

      The following result is returned:

      +------------+-------------+-------------+--------------+
| shop_name  | customer_id | total_price | total_price2 |
+------------+-------------+-------------+--------------+
| s1         | c1          | 100.1       | 100.1        |
| s2         | c2          | 100.2       | 100.2        |
+------------+-------------+-------------+--------------+

      The returned result is the intersection of the two tables, not all rows in the sale_detail table.