Time series

x13_arima

• Autoregressive Integrated Moving Average Model (ARIMA) is a famous time series prediction method defined by Box and Jenkins in the early 1970s. Therefore, this model is also called the Box-Jenkins model or the Box-Jenkins approach.

• x13-arima is an ARIMA algorithm based on the open-source X-13ARIMA-SEATS seasonal adjustment.

• For more information about X-13ARIMA-SEATS Seasonal Adjustment Program, visit wiki.

• For more information about ARIMA, visit wiki.

PAI command

PAI -name x13_arima
    -project algo_public
    -DinputTableName=pai_ft_x13_arima_input
    -DseqColName=id 
    -DvalueColName=number
    -Dorder=3,1,1
    -Dstart=1949.1
    -Dfrequency=12
    -Dseasonal=0,1,1
    -Dperiod=12
    -DpredictStep=12
    -DoutputPredictTableName=pai_ft_x13_arima_out_predict
    -DoutputDetailTableName=pai_ft_x13_arima_out_detail

Parameter description

Time series format

The start and frequency parameters specify the two time dimensions of data (valueColName), TS1 and TS2. The frequency parameter represents the data frequency within a cycle, namely, the frequency of TS2 in each TS1. The start parameter is in the format of n1.n2, which indicates that the start date is the N2 TS2 in the N1 TS1.

For example, value=[1,2,3,5,6,7,8,9,10,11,12,13,14,15]

• start=1949.3, frequency=12 indicates that the data frequency is monthly per year, and the prediction start date is 1950.06.

• start=1949.3, frequency=4 indicates that the data frequency is quarterly per year, and the prediction start date is 1953.02.

• start=1949.3, frequency=7 indicates that the data frequency is daily per week, and the prediction start date is 1951.04.

• start=1949.1, frequency=1 can represent any time unit, and the prediction start date is 1963.00.

Example

Test data

Used data: AirPassengers.

This data set is the number of passengers for international airlines each month from 1949 to 1960, as shown in the following table.

Upload data by tuunel. The command is as follows.

create table pai_ft_x13_arima_input(id bigint,number bigint);
tunnel upload data/airpassengers.csv pai_ft_x13_arima_input -h true;

PAI command

PAI -name x13_arima
    -project algo_public
    -DinputTableName=pai_ft_x13_arima_input
    -DseqColName=id 
    -DvalueColName=number
    -Dorder=3,1,1
    -Dseasonal=0,1,1
    -Dstart=1949.1
    -Dfrequency=12
    -Dperiod=12
    -DpredictStep=12
    -DoutputPredictTableName=pai_ft_x13_arima_out_predict
    -DoutputDetailTableName=pai_ft_x13_arima_out_detail

Output description

• Output table: outputPredictTableName.
  The fields are:

  column name	comment
  pdate	Prediction date
  forecast	Prediction conclusion
  lower	Lower threshold of the prediction conclusion when the confidence level is confidenceLevel (default: 0.95)
  upper	Upper threshold of the prediction conclusion when the confidence level is confidenceLevel (default: 0.95)

  Data display:
  

• Output table: outputDetailTableName.
  The fields are:

  column name	comment
  key	model: indicates the model. evaluation: indicates the evaluation result. parameters: indicates training parameters. log: indicates the training log.
  summary	Storage details

  Data display:
  

  Display on the PAI web - Model factor (key=model)
  

  Display on the PAI web - Evaluation index (key=evaluation)
  

Algorithm scale

• Supported scale

  • Rows: a maximum of 1200 rows in a single group

  • Columns: 1 data column

• Resource calculation method

  • Default calculation method used when groupColNames is not set
    coreNum = 1
    memSizePerCore = 4096

  • Default calculation method used when groupColNames is set
    coreNum = floor (total number of data rows / 120,000)
    memSizePerCore = 4096

x13_auto_arima

x13-auto-arima includes an automatic ARIMA model selection procedure based largely on the procedure of Gomez and Maravall (1998) as implemented in TRAMO (1996) and subsequent revisions.

The x13-auto-arima selection process is as follows:

1. Default model estimation.

   • When frequency is 1, the default model is (0,1,1).
   • When frequency is greater than 1, the default model is (0,1,1)(0,1,1).

2. Identication of dierencing orders.

   • Skip this step if you set diff and seasonalDiff.
   • Determine the difference d and the seasonal difference D by using the Unit root test (wiki ).

3. Identication of ARMA model orders.
   

   Select the optimal model based on BIC(wiki), and the maxOrder and maxSeasonalOrder parameters are used in this step.

4. Comparison of identified model with default model.
   

   Compare models by using Ljung-Box Q statistic(wiki). If both models are unacceptable, use the (3,d,1)(0,D,1) model.

5. Final model checks.

PAI command line

PAI -name x13_auto_arima
    -project algo_public
    -DinputTableName=pai_ft_x13_arima_input
    -DseqColName=id 
    -DvalueColName=number
    -Dstart=1949.1
    -Dfrequency=12
    -DpredictStep=12
    -DoutputPredictTableName=pai_ft_x13_arima_out_predict2
    -DoutputDetailTableName=pai_ft_x13_arima_out_detail2

Parameter description

Time series format

The start and frequency parameters specify the two time dimensions of data (valueColName), TS1 and TS2.The frequency parameter represents the data frequency within a cycle, namely, the frequency of TS2 in each TS1. The start parameter is in the format of n1.n2, which indicates that the start date is the N2 TS2 in the N1 TS1.

For example, value=[1,2,3,5,6,7,8,9,10,11,12,13,14,15]

• start=1949.3, frequency=12 indicates that the data frequency is monthly per year, and the prediction start date is 1950.06.

• start=1949.3, frequency=4 indicates that the data frequency is quarterly per year, and the prediction start date is 1953.02.

• start=1949.3, frequency=7 indicates that the data frequency is daily per week, and the prediction start date is 1951.04.

• start=1949.1, frequency=1 can represent any time unit, and the prediction start date is 1963.00.

Example

Test data

Used data: AirPassengers

This data set is the number of passengers for international airlines each month from 1949 to 1960, as shown in the following table.

Upload data by tuunel. The command is as follows.

create table pai_ft_x13_arima_input(id bigint,number bigint);
tunnel upload data/airpassengers.csv pai_ft_x13_arima_input -h true;

PAI command

PAI -name x13_auto_arima
    -project algo_public
    -DinputTableName=pai_ft_x13_arima_input
    -DseqColName=id 
    -DvalueColName=number
    -Dstart=1949.1
    -Dfrequency=12
    -DmaxOrder=4
    -DmaxSeasonalOrder=2
    -DmaxDiff=2
    -DmaxSeasonalDiff=1
    -DpredictStep=12
    -DoutputPredictTableName=pai_ft_x13_arima_auto_out_predict
    -DoutputDetailTableName=pai_ft_x13_arima_auto_out_detail

Output description

• Output table: outputPredictTableName.
  The fields are:

  column name	comment
  pdate	Prediction date
  forecast	Prediction conclusion
  lower	Lower threshold of the prediction conclusion when the confidence level is confidenceLevel (default: 0.95)
  upper	Upper threshold of the prediction conclusion when the confidence level is confidenceLevel (default: 0.95)

  Data display:
  

• Output table: outputDetailTableName.
  

The fields are:

Display on the PAI web - Model factor (key=model)

Display on the PAI web - Evaluation index (key=evaluation)

Algorithm scale

• Supported scale

  • Rows: a maximum of 1200 rows in a single group

  • Columns: 1 data column

• Resource calculation method

  • Default calculation method used when groupColNames is not set
    coreNum = 1
    memSizePerCore = 4096

  • Default calculation method used when groupColNames is set
    coreNum = floor (total number of data rows / 120,000)
    memSizePerCore = 4096

Important notes

Why are the prediction results the same?

When an exception occurs during model training, the mean model is called, and all prediction results are the mean of the training data.

Common exceptions include “Not stationary after timing differential diff”, “training does not converge”, “variance is 0”. You can view the stderr file of individual nodes in the logview to obtain exception details.