The GROUP BY clause is typically used with aggregate functions to group the rows of a query result. Each group is based on the values of one or more columns, and each distinct combination of values produces one summary row. GROUP BY ALL derives the grouping keys from the SELECT list automatically, and the ROLLUP, CUBE, and GROUPING SETS clauses extend GROUP BY to additional grouping patterns.
Syntax
Simple Log Service supports the following grouping forms. Choose one based on the grouping keys and subtotals that your query requires.
Clause | Use it when |
| You group rows by columns that you list explicitly. This is the standard grouping form. |
| You have many grouping fields, or you adjust analysis dimensions often, and you want the grouping keys derived from the |
| You need a subtotal for each group and a grand total for all groups. |
| You need every possible combination of the specified columns. |
| You need only the groupings that you specify. |
In every grouping form, the SELECT list can contain only grouping keys and aggregate results. For the full rule and a corrected query, see the note in Examples.
GROUP BY
Groups the analysis results by one or more specified columns. For examples, see Example 1, Example 2, and Example 3.
SELECT
key1,
...,
aggregate_function
GROUP BY
key1, ...Parameters:
key1: The column to group rows by. You can group rows by log field names or by result columns calculated by aggregate functions. TheGROUP BYclause supports one or more columns.aggregate_function: The aggregate function applied to each group, such ascount,min,max,avg, orsum.
GROUP BY ALL
GROUP BY ALL automatically uses the expressions in the SELECT list that contain no aggregate function and no window function as grouping keys, so you do not need to repeat those expressions in the GROUP BY clause. This form is useful when you have many grouping fields or when you adjust analysis dimensions frequently. For examples, see Example 7, Example 8, and Example 9.
SELECT
key1,
key2,
aggregate_function
GROUP BY ALLParameters:
key1andkey2: Grouping fields or expressions, such asstatus,request_method,substr(request_method, 1, 1), or aCASEexpression. An expression is grouped by its complete calculated result.GROUP BY ALLtakes no field list, so you do not repeat an alias or an expression after it.aggregate_function: The aggregate function applied to each group, such ascount(*),sum(request_time), oravg(request_time).
For example, the following two query statements are equivalent:
* | SELECT request_method, status, count(*) AS PV GROUP BY ALL* | SELECT request_method, status, count(*) AS PV GROUP BY request_method, statusRules and limits:
Automatic grouping is based on the current
SELECTlist, not on all fields in the logs. Adding or removing a non-aggregate, non-window expression changes the grouping dimensions accordingly.Wildcard selection is not supported at the same query level: you cannot combine
GROUP BY ALLwithSELECT *,SELECT table_alias.*, orSELECT *, count(*). List the fields explicitly. This limit applies to the wildcard selection itself, not to aggregate functions, socount(*)works as usual next to explicitly listed fields.An entire expression that contains an aggregate function or window function is not automatically added as a grouping key. For example,
sum(request_time) / count(*)is not used for grouping, and the fields inside the aggregate calls do not become grouping keys. Any non-aggregate field that you select alongside such an expression must still comply with the standardGROUP BYgrouping rules. InSELECT request_method, status, count(*) AS PV GROUP BY ALL, bothrequest_methodandstatusare grouping keys.If the
SELECTlist contains only aggregate expressions, all data that meets the query conditions is aggregated globally. For example,* | SELECT count(*) AS PV, sum(request_time) AS total_time GROUP BY ALLis equivalent to omittingGROUP BY ALL.You can use
GROUP BY ALLwithWHERE,HAVING,ORDER BY, and window functions. Window functions operate on the grouped and aggregated results, and the fields that they reference must still comply with the grouping rules.To derive grouping keys automatically, use
GROUP BY ALLby itself. InGROUP BY ALL statusorGROUP BY ALL ROLLUP (...),ALLmodifies the grouping specification that follows it, so no grouping keys are derived automatically.To group by a field or alias named
all, enclose the name in double quotation marks, as inGROUP BY "all". By default, a standaloneGROUP BY ALLderives grouping keys automatically even if a field namedallexists.
GROUP BY ROLLUP
The GROUP BY ROLLUP clause groups data into hierarchical summaries. It returns a subtotal for each group and a grand total for all groups. For example, GROUP BY ROLLUP (a, b) produces a result set for the grouping sets (a, b), (a, null), and (null, null). A null in a grouping set means that the column is aggregated in that set, so the row is a subtotal. For an example, see Example 6.
SELECT
key1,
...,
aggregate_function
GROUP BY ROLLUP (key1, ...)Parameters
key1: The column to group rows by. You can group by log field names or by the result columns of an aggregate function. TheGROUP BYclause supports grouping by single or multiple columns.aggregate_function: The Aggregate functionsapplied to each group, such ascount,min,max,avg, orsum.
GROUP BY CUBE
The GROUP BY CUBE clause groups data by all possible combinations of the specified columns. For example, GROUP BY CUBE (a, b) produces a result set for the grouping sets (a, b), (null, b), (a, null), and (null, null). A null in a grouping set means that the column is aggregated in that set, so the row is a subtotal. For an example, see Example 5.
SELECT
key1,
...,
aggregate_function
GROUP BY CUBE (key1, ...)Parameters
key1: The column to group rows by. You can group by log field names or by the result columns of an aggregate function. TheGROUP BYclause supports grouping by single or multiple columns.aggregate_function: The Aggregate functionsapplied to each group, such ascount,min,max,avg, orsum.
GROUP BY GROUPING SETS
The GROUP BY GROUPING SETS clause lets you specify multiple distinct groupings in a single query. For example, GROUP BY GROUPING SETS (a, b) is equivalent to combining the results of GROUP BY a and GROUP BY b, and produces a result set for the grouping sets (a, null) and (null, b). A column that is not part of a grouping set is null in the rows for that set. For an example, see Example 4.
SELECT
key1,
...,
aggregate_function
GROUP BY GROUPING SETS (key1, ...)Parameters
key1: The column to group rows by. You can group by log field names or by the result columns of an aggregate function. TheGROUP BYclause supports grouping by single or multiple columns.aggregate_function: The Aggregate functionsapplied to each group, such ascount,min,max,avg, orsum.
Examples
In an SQL statement, the SELECT list of a grouping query can contain only the following items:
Columns or expressions specified in the
GROUP BYclause.Columns or expressions that
GROUP BY ALLderives automatically as grouping keys from theSELECTlist.Results of aggregate calculations on columns, such as
COUNT()orSUM().
You can also use window functions on grouped results. For more information, see Example 9.
You cannot directly select a column that is not included in the GROUP BY clause, because the column may have multiple values after grouping and its meaning would be ambiguous. For example, the following query statement is invalid:
* | SELECT status, request_time, COUNT(*) AS PV GROUP BY statusThe query statement is invalid because request_time is not included in the GROUP BY clause and is not processed by an aggregate function. You can rewrite the query statement as follows:
* | SELECT status, arbitrary(request_time), COUNT(*) AS PV GROUP BY statusHere, arbitrary(request_time) is an aggregate operation on request_time that selects an arbitrary request_time value from each group. This allows you to include the column in your query.
Example 1: Calculate the page views (PV) for each status code
Calculate the PVs for each status code by using GROUP BY.
Query statement
* | SELECT status, count(*) AS PV GROUP BY statusQuery results: A table with the status and PV columns, which shows each HTTP status code and its PV.
Example 2: Calculate the hourly PVs for a website
Calculate the PVs for a website at an hourly time granularity by using GROUP BY.
Query statement
The __time__ field is a reserved field in Simple Log Service that represents the time column. In this query, time is an alias for date_trunc('hour', __time__). For more information about the date_trunc function, see date_trunc function.
* |
SELECT
count(*) AS PV,
date_trunc('hour', __time__) AS time
GROUP BY
time
ORDER BY
time
LIMIT
1000Query results: The query returns a result set with the PV and time columns. In the sample data, PV values 1202, 10159, and 28001 correspond to 2021-08-10 00:00:00.000, 2021-08-10 01:00:00.000, and 2021-08-10 02:00:00.000, respectively.
Example 3: Calculate PVs at a 5-minute time granularity
Calculate the PVs at a 5-minute time granularity by using GROUP BY.
Query statement
The date_trunc function can only group data by a fixed interval. To group data by a custom time interval, use the modulo operation. For example, % 300 aligns timestamps to a 5-minute (300 seconds) time granularity.
* |
SELECT
count(*) AS PV,
__time__-__time__ % 300 AS time
GROUP BY
time
LIMIT
1000Query results: The query returns three records: PV=143 for time=1628525100, PV=31 for time=1628526600, and PV=44 for time=1628526900.
Example 4: Group by request method and status independently (GROUPING SETS)
Group data by request method and by status independently, and calculate the PVs for each request method and for each status code.
Query statement
* |
SELECT
request_method,
status,
count(*) AS PV
GROUP BY
GROUPING SETS (request_method, status)Query results: When the data is grouped by
request_method,statusis null, and the PV is 189 for GET, 47 for POST, 39 for PUT, 9 for DELETE, and 1 for HEAD. When the data is grouped bystatus,request_methodis null, and the PV is, for example, 259 for 200, 2 for 201, and 2 for 202.
Example 5: Group by all combinations of request method and status (CUBE)
Group data by request method and request status. The grouping sets include (null, null), (request_method, null), (null, status), and (request_method, status). Calculate the PVs for each grouping set.
Query statement
* |
SELECT
request_method,
status,
count(*) AS PV
GROUP BY
CUBE (request_method, status)Query results: The result set contains the request_method, status, and PV columns and has 43 records. In the aggregate rows generated by CUBE, a null value for
request_methodorstatusindicates a subtotal for that dimension. For example, the PV is 285 for null/null (the grand total), 189 for GET/null, 47 for POST/null, 39 for PUT/null, 9 for DELETE/null, and 1 for HEAD/null. The PV is 259 for null/200, and 2 for each of null/201 and null/202.
Example 6: Generate hierarchical subtotals for request method and status (ROLLUP)
Group data by request method and request status. The grouping sets include (request_method, status), (request_method, null), and (null, null). Calculate the PVs for each grouping set.
Query statement
* |
SELECT
request_method,
status,
count(*) AS PV
GROUP BY
ROLLUP (request_method, status)Query results: The result set contains the request_method, status, and PV columns and has 33 records. The ROLLUP function generates summary rows at each grouping level, where
nullindicates that the column has been aggregated. For example,GET / null / 189indicates that the total PV for the GET method across all status codes is 189, andnull / null / 285indicates that the total PV for all methods and status codes is 285.
Example 7: Automatically group by multiple fields (GROUP BY ALL)
Calculate the PVs by request method and status code, and retain only the groups with more than 100 requests.
Query statement
* |
SELECT
request_method,
status,
count(*) AS PV
GROUP BY ALL
HAVING count(*) > 100
ORDER BY PV DESCQuery results: The result set contains the request_method, status, and PV columns. Only the combinations with more than 100 requests are returned, sorted by PV in descending order. This grouping is equivalent to
GROUP BY request_method, status.
Example 8: Automatically group by an expression (GROUP BY ALL)
Calculate the PVs by request method at a 5-minute time granularity without repeating the time expression or its alias.
Query statement
* |
SELECT
__time__ - __time__ % 300 AS time,
request_method,
count(*) AS PV
GROUP BY ALL
ORDER BY time, request_method
LIMIT 1000Query results: The result set contains the time, request_method, and PV columns. This grouping is equivalent to
GROUP BY __time__ - __time__ % 300, request_method.
Example 9: Rank grouped results (GROUP BY ALL)
Calculate the PVs for each status code, and then rank the status codes by request count in descending order.
Query statement
* |
SELECT
status,
count(*) AS PV,
rank() OVER (ORDER BY count(*) DESC) AS ranking
GROUP BY ALL
ORDER BY ranking, statusQuery results: The result set contains the status, PV, and ranking columns. Only
statusis used as a grouping key.count(*)performs the aggregation, andrank()ranks the aggregated results without participating in grouping. Status codes with the same request count have the same ranking.