您可以使用ROLLUP语法实现只借助一条查询语句,就计算出数据在不同维度上划分组之后得到的统计结果以及所有数据的总体值。本文将介绍如何使用ROLLUP语法。
前提条件
PolarDB集群版本需为PolarDB MySQL版 8.0且Revision version为8.0.1.1.0或以上,您可以参见查询版本号确认集群版本。
语法
ROLLUP语法可以看作是GROUP BY语法的拓展,您只需要在原本的GROUP BY列之后加上WITH ROLLUP即可,例如:
SELECT year, country, product, SUM(profit) AS profit
FROM sales
GROUP BY year, country, product WITH ROLLUP;除了产生按照GROUP BY所指定的列聚合产生的结果外,ROLLUP还会从右至左依次计算更高层次的聚合结果,直至所有数据的聚合。例如上述语句就会先按照GROUP BY year, country, product计算总收益,再按照GROUP BY year, country以及GROUP BY year计算总收益,最后按照不带任何GROUP BY条件计算整个sales表的总收益。
使用ROLLUP语法主要有如下优势:
方便对数据进行多维度的统计分析,简化原本对不同维度各自进行SQL查询的编程复杂度。
实现更快更高效的查询处理。
ROLLUP可以将所有的聚合工作转移到服务端,只用读一次数据就能够完成原本多次查询才能完成的统计工作,减轻了客户端的处理负载和网络流量。
并行查询ROLLUP增强性能测试
PolarDB还实现了针对ROLLUP语法的并行查询(Parallel Query)增强,支持多个线程并发计算聚合结果并输出汇总后的结果,大大提升语句执行效率。
使用TPCH进行测试,以Q1为例,在原始的语句上加入ROLLUP语法:
本文的TPC-H的实现基于TPC-H的基准测试,并不能与已发布的TPC-H基准测试结果相比较,本文中的测试并不符合TPC-H基准测试的所有要求。
select
l_returnflag,
l_linestatus,
sum(l_quantity) as sum_qty,
sum(l_extendedprice) as sum_base_price,
sum(l_extendedprice * (1 - l_discount)) as sum_disc_price,
sum(l_extendedprice * (1 - l_discount) * (1 + l_tax)) as sum_charge,
avg(l_quantity) as avg_qty,
avg(l_extendedprice) as avg_price,
avg(l_discount) as avg_disc,
count(*) as count_order
from
lineitem
where
l_shipdate <= date_sub('1998-12-01', interval ':1' day)
group by
l_returnflag,
l_linestatus
with rollup
order by
l_returnflag,
l_linestatus;不开启并行查询时,耗时318.73秒。
root@localhost:dbt3 8.0.13-rds-dev> select -> l_returnflag, -> l_linestatus, -> sum(l_quantity) as sum_qty, -> sum(l_extendedprice) as sum_base_price, -> sum(l_extendedprice * (1 - l_discount)) as sum_disc_price, -> sum(l_extendedprice * (1 - l_discount) * (1 + l_tax)) as sum_charge, -> avg(l_quantity) as avg_qty, -> avg(l_extendedprice) as avg_price, -> avg(l_discount) as avg_disc, -> count(*) as count_order -> from -> lineitem -> where -> l_shipdate <= date_sub('1998-12-01', interval ':1' day) -> group by -> l_returnflag, -> l_linestatus -> with rollup -> order by -> l_returnflag, -> l_linestatus; +-------------+--------------+----------------+-------------------+-------------------+---------------------+----------+--------------+----------+-------------+ | l_returnflag | l_linestatus | sum_qty | sum_base_price | sum_disc_price | sum_charge | avg_qty | avg_price | avg_disc | count_order | +-------------+--------------+----------------+-------------------+-------------------+---------------------+----------+--------------+----------+-------------+ | NULL | NULL | 1526902118.00 | 2289567239008.24 | 2175081814581.5635 | 2262103960452.919838 | 25.501098 | 38238.521048 | 0.050000 | 59875936 | | A | NULL | 37683718.00 | 56503738343x.68 | 53672882989857.2197 | 55826126394x.603553 | 25.500331 | 38236.069808 | 0.050005 | 14777601 | | A | F | 37683718.00 | 56503738343x.68 | 53672882989857.2197 | 55826126394x.603553 | 25.500331 | 38236.069808 | 0.050005 | 14777601 | | N | NULL | 77204634.00 | 115915498514.15 | 110131315345.0758 | 114529893971.143253 | 25.497843 | 38237.208251 | xxx | 3617xxx | | N | F | 9839917.00 | 14735826224.79 | 13998714160.5500 | 14559295139.031004 | 25.516888 | 38247.950877 | 0.049977 | 385521 | | N | O | 76321271x.00 | 1144418224289.36 | 108719254040.2584 | 113063160227x.113229 | 25.497601 | 38234.013094 | xxx | 29932xxx | | R | NULL | 37702476x.00 | 56537580506.41 | 53710767035x.2680 | 55853917990x.170252 | 25.508535 | 38251.886895 | 0.049997 | 14780xxx | | R | F | 37702476x.00 | 56537580506.41 | 53710767035x.2680 | 55853917990x.170252 | 25.508535 | 38251.886895 | 0.049997 | 14780xxx | +-------------+--------------+----------------+-------------------+-------------------+---------------------+----------+--------------+----------+-------------+ 8 rows in set, 3 warnings (5 min 18.73 sec)开启并行查询后,耗时22.30秒,语句执行效率提升14倍以上。
root@localhost:dbt3 8.0.13-rds-dev> select /*+ SET_VAR(max_parallel_degree=32) */ -> l_returnflag, -> l_linestatus, -> sum(l_quantity) as sum_qty, -> sum(l_extendedprice) as sum_base_price, -> sum(l_extendedprice * (1 - l_discount)) as sum_disc_price, -> sum(l_extendedprice * (1 - l_discount) * (1 + l_tax)) as sum_charge, -> avg(l_quantity) as avg_qty, -> avg(l_extendedprice) as avg_price, -> avg(l_discount) as avg_disc, -> count(*) as count_order -> from -> lineitem -> where -> l_shipdate <= date_sub('1998-12-01', interval '1' day) -> group by -> l_returnflag, -> l_linestatus -> with rollup -> order by -> l_returnflag, -> l_linestatus; | l_returnflag | l_linestatus | sum_qty | sum_base_price | sum_disc_price | sum_charge | avg_qty | avg_price | avg_disc | count_order | | NULL | NULL | 1526902118.00 | 22839567239008.24 | 2175081814581.563 | 2262103960452.919838 | 25.501098 | 38238.521048 | 0.050008 | 59785936 | | A | NULL | 376633718.00 | 5650037383347.68 | 536782889857.2197 | 558261263942.604353 | 25.500331 | 38236.693008 | 0.050005 | 14777601 | | A | F | 376633718.00 | 565003738347.68 | 536782889857.2197 | 558261263942.604353 | 25.500331 | 38236.693008 | 0.050005 | 14777601 | | N | NULL | 775804634.00 | 11591540955214.15 | 11001912541965.18 | 11445508937148.143233 | 25.497847 | 38233.208251 | 0.050008 | 30433757 | | N | F | 9439317.00 | 14174538224.79 | 13399171416810.5908 | 14550093539.110493 | 25.514642 | 38284.468099 | 0.049935 | 369914 | | N | O | 763212717.00 | 11444182242889.36 | 10871925402084.36 | 11306316022279.113229 | 25.49766 | 38233.013698 | 0.050009 | 29932993 | | R | NULL | 377024766.00 | 56535804956.41 | 53710676703959.2680 | 55854919770932.170252 | 25.505586 | 38241.865898 | 0.050009 | 14777338 | | R | F | 377024766.00 | 56535804956.41 | 53710676703959.2680 | 55854919770932.170252 | 25.505586 | 38241.865898 | 0.049997 | 14803338 | 8 rows in set, 3 warnings (22.30 sec)