The Percentile component computes 101 quantile values (0th through 100th percentile) for one or more columns in a MaxCompute table. Use it to understand data distribution before training, detect outliers in feature columns, or prepare data for downstream normalization steps.
Limitations
Supported column types: BIGINT, DOUBLE, and DATETIME only.
Null values in a column are skipped during calculation.
If all of the columns contain only null values, an error is returned.
Configure the component
Method 1: Configure on the pipeline canvas
Open the Percentile component in Machine Learning Designer (formerly Machine Learning Studio) and set the following parameters.
Parameters Setting tab
| Parameter | Description |
|---|---|
| Input Columns | Click Select Column to choose the columns to compute percentiles for. |
Tuning tab
| Parameter | Description |
|---|---|
| Number of Cores | The number of cores allocated to the job. |
| Memory Size per Core | The memory allocated to each core. |
Method 2: Use PAI commands
Run the following command using the SQL Script component.
PAI -name Percentile
-project algo_public
-DinputTableName=maple_test_percentile_3col_input
-DcolName=col0,col1,col2 -DoutputTableName=maple_test_percentile_3col_output;Required parameters
| Parameter | Description |
|---|---|
inputTableName | Name of the input table. |
outputTableName | Name of the output table. |
Optional parameters
| Parameter | Description |
|---|---|
colName | Names of the columns to compute. Default: all columns. Separate multiple column names with commas (,). |
inputPartitions | Partitions in the input table. Default: all partitions. Format: partition_name=value for a single partition, name1=value1,name2=value2 for multiple partitions, or name1=value1/name2=value2 for multi-level partitions. |
predictInputTableName | Name of the prediction table. When set, the component generates a prediction result. |
predictInputTablePartitions | Partitions in the prediction table. |
predictSelectedColNames | Column names to use from the prediction table. Default: all columns. Column names must match those in the training table. |
predictSelectedOriginalColNames | Column names whose data to retain in the output. Default: all columns. Separate multiple names with commas (,). |
predictOutputTableName | Name of the output prediction table. Used with predictInputTableName. |
lifecycle | Lifecycle of the output table. Default: no lifecycle. Must be a positive integer. |
coreNum | Number of cores. Valid values: 1–9999. Must be a positive integer. Used with memSizePerCore. |
memSizePerCore | Memory per core, in MB. Valid values: 1024–65536. Must be a positive integer. |
Example
Input table: maple_test_percentile_3col_input
| col0:double (1000 rows) | col1:bigint (100 rows) | col2:bigint (300 rows) |
|---|---|---|
| 962 | 88 | Tue Oct 15 00:26:40 CST 1974 |
| 218 | 99 | Thu Jan 04 20:53:20 CST 1973 |
| 565 | 44 | Sat Mar 09 02:40:00 CST 1974 |
| 314 | 68 | Mon Aug 11 22:40:00 CST 1975 |
| 583 | 13 | Sat Aug 23 12:26:40 CST 1975 |
| 615 | 87 | Tue May 25 14:13:20 CST 1971 |
| 70 | 53 | Fri Mar 23 09:20:00 CST 1979 |
| 929 | 63 | Mon Jul 03 16:26:40 CST 1972 |
| 249 | 48 | Thu Mar 15 07:33:20 CST 1973 |
| 428 | 62 | Wed Mar 17 03:33:20 CST 1971 |
| 119 | 1 | Thu Jun 26 15:33:20 CST 1975 |
| 756 | 27 | Mon Jan 30 17:20:00 CST 1978 |
| 490 | 75 | Wed Dec 11 21:20:00 CST 1974 |
| 957 | 12 | Sun Jul 05 12:26:40 CST 1970 |
| 80 | 22 | Wed Oct 04 06:40:00 CST 1972 |
| 681 | 57 | Wed Nov 03 15:06:40 CST 1971 |
| 13 | 95 | Sat Sep 12 23:06:40 CST 1970 |
PAI command
PAI -name Percentile
-project algo_public
-DinputTableName=maple_test_percentile_3col_input
-DcolName=col0,col1,col2 -DoutputTableName=maple_test_percentile_3col_output;Output table: maple_test_percentile_3col_output
The output table has one row per percentile rank (0–100). The quantile column holds the rank; each input column holds the corresponding percentile value. To find the 95th percentile of col0, locate the row where quantile = 95.
| quantile:bigint | col0:double | col1:bigint | col2:datetime |
|---|---|---|---|
| 0 | 0.0 | 0 | Thu Jan 01 08:00:00 CST 1970 |
| 1 | 9.0 | 0 | Sat Jan 24 11:33:20 CST 1970 |
| 2 | 19.0 | 1 | Sat Feb 28 04:53:20 CST 1970 |
| 3 | 29.0 | 2 | Fri Apr 03 22:13:20 CST 1970 |
| 4 | 39.0 | 3 | Fri May 08 15:33:20 CST 1970 |
| 5 | 49.0 | 4 | Fri Jun 12 08:53:20 CST 1970 |
| 6 | 59.0 | 5 | Fri Jul 17 02:13:20 CST 1970 |
| 7 | 69.0 | 6 | Thu Aug 20 19:33:20 CST 1970 |
| 8 | 79.0 | 7 | Thu Sep 24 12:53:20 CST 1970 |
| 9 | 89.0 | 8 | Thu Oct 29 06:13:20 CST 1970 |
| 10 | 99.0 | 9 | Wed Dec 02 23:33:20 CST 1970 |
| 11 | 109.0 | 10 | Wed Jan 06 16:53:20 CST 1971 |
| 12 | 119.0 | 11 | Wed Feb 10 10:13:20 CST 1971 |
| 13 | 129.0 | 12 | Wed Mar 17 03:33:20 CST 1971 |
| 14 | 139.0 | 13 | Tue Apr 20 20:53:20 CST 1971 |
| 15 | 149.0 | 14 | Tue May 25 14:13:20 CST 1971 |
| 16 | 159.0 | 15 | Tue Jun 29 07:33:20 CST 1971 |
| ... | ... | ... | ... |
| 84 | 839.0 | 83 | Thu Dec 15 10:13:20 CST 1977 |
| 85 | 849.0 | 84 | Thu Jan 19 03:33:20 CST 1978 |
| 86 | 859.0 | 85 | Wed Feb 22 20:53:20 CST 1978 |
| 87 | 869.0 | 86 | Wed Mar 29 14:13:20 CST 1978 |
| 88 | 879.0 | 87 | Wed May 03 07:33:20 CST 1978 |
| 89 | 889.0 | 88 | Wed Jun 07 00:53:20 CST 1978 |
| 90 | 899.0 | 89 | Tue Jul 11 18:13:20 CST 1978 |
| 91 | 909.0 | 90 | Tue Aug 15 11:33:20 CST 1978 |
| 92 | 919.0 | 91 | Tue Sep 19 04:53:20 CST 1978 |
| 93 | 929.0 | 92 | Mon Oct 23 22:13:20 CST 1978 |
| 94 | 939.0 | 93 | Mon Nov 27 15:33:20 CST 1978 |
| 95 | 949.0 | 94 | Mon Jan 01 08:53:20 CST 1979 |
| 96 | 959.0 | 95 | Mon Feb 05 02:13:20 CST 1979 |
| 97 | 969.0 | 96 | Sun Mar 11 19:33:20 CST 1979 |
| 98 | 979.0 | 97 | Sun Apr 15 12:53:20 CST 1979 |
| 99 | 989.0 | 98 | Sun May 20 06:13:20 CST 1979 |
| 100 | 999.0 | 99 | Sat Jun 23 23:33:20 CST 1979 |