Automatic ARIMA Order Selection

Automatic ARIMA order selection in MIDAS runs in two stages: it first sets the differencing order with a test, then chooses the remaining orders by an information criterion with the differencing held fixed. This page uses the non-seasonal ARIMA(p,d,q) model to explain each stage and the theory behind selecting the differencing order $d$ with a test rather than an information criterion. The seasonal differencing order $D$ , which appears when a seasonal period is specified, is covered in the final section.

ARIMA runs in the ARIMA tab, opened from Analysis > ARIMA... in the menu bar; for how to operate the tab and read its results, see The ARIMA Tab. To run it through the Agent API, see the ARIMA section of the Agent API.

The selection procedure

The first stage sets the differencing order $d$ . Starting at $d = 0$ , MIDAS applies the KPSS test to the series, differences it while the statistic exceeds a decision threshold, and stops once the statistic no longer exceeds it or $d$ reaches the limit $\text{maxD}$ . A series whose variance is essentially zero after differencing is treated as stationary, and the procedure stops. The test itself and the meaning of the threshold are covered in the sections below.

The second stage holds that $d$ fixed and fits ARIMA(p,d,q) for every combination of $p$ from $0$ to $\text{maxP}$ and $q$ from $0$ to $\text{maxQ}$ , selecting the candidate with the smallest AIC or BIC. Candidates whose estimation fails to converge are excluded from the selection; if no candidate converges, MIDAS fits and returns ARIMA(0,d,0).

When the order is specified manually, this procedure is skipped and the given order is used directly. For an automatic selection, MIDAS reports only the selected $d$ and the operational fact of whether the statistic exceeded the decision threshold, in a note next to the results. It does not display p-values or significant/not-significant verdicts.

Why d alone is selected by a test

If AIC/BIC selects $p, q$ , it seems natural to put $d$ into the same search and select all three by the information criterion at once. That comparison does not hold up.

Differencing swaps out the data the model explains. The log-likelihood of an ARIMA(p,d,q) model is computed on the $d$ -times differenced series $\Delta^d y$ . A candidate with $d=0$ assigns a probability density to the sequence of $N$ levels, while a candidate with $d=1$ assigns one to the $N-1$ period-to-period changes $\Delta y$ . The level series is the initial value plus the accumulated changes ( $y_t = y_1 + \sum_{i=2}^{t} \Delta y_i$ ), so even a completely specified joint distribution of the changes does not determine the distribution of the levels unless the starting level is supplied. A candidate with $d=1$ is a model that describes only how the series moves and says nothing about the initial level.

The likelihood is the probability density the model assigns to the observed data. The two candidates' likelihoods therefore answer different questions: how well the model explained all $N$ levels, and how well it explained only the $N-1$ changes. Ranking two numbers that do not answer the same question says nothing about which model is better.

AIC and BIC inherit this limitation as is. AIC is

\text{AIC} = -2\,\ell(\hat\theta) + 2k

where $\ell(\hat\theta)$ is the maximized log-likelihood and $k$ is the number of parameters. AIC estimates the Kullback-Leibler divergence between the fitted model and the data-generating process, up to a constant that depends only on the true distribution. Taking AIC differences between candidates cancels that constant and leaves an estimate of the difference in divergences — but only when every candidate assigns a probability density to the same observed data.

Superficial transformations of the response do not break this condition: a model written for a smoothly and invertibly transformed response can be re-expressed as a density for the original data by adding the change-of-variables Jacobian term to its log-likelihood. For a model whose response is merely shifted by a constant, the Jacobian is 1, and the comparison goes through with no correction at all.

Differencing is a transformation for which no such re-expression exists. A series $y$ of length $N$ corresponds one-to-one to the pair of its first $d$ levels and the $d$ -times differenced series of length $N-d$ . Modeling only the differenced series amounts to discarding the first $d$ points from that pair, and the model assigns no probability to the discarded levels. A dimension-reducing map has no Jacobian term to convert with; the only way to fill in the missing part is to add a distribution for the initial levels to the model.

But a candidate with $d \ge 1$ treats the level series as a process that becomes stationary only after differencing. Under such a process the variance of the levels grows without bound over time and no unconditional distribution of the levels exists, so the model itself provides no distribution for the initial levels to add. Candidates with different $d$ therefore fail the same-data condition, and their AIC differences do not estimate differences in divergence. BIC fails to compare across $d$ for the same reason.

The differencing order must be set by a criterion other than the likelihood computed on the data the model was fitted to. Whether the series has been differenced enough is the question of whether the differenced series is stationary, so MIDAS decides $d$ with a stationarity test.

The KPSS test

The KPSS test — the test by Kwiatkowski, Phillips, Schmidt, and Shin — takes level stationarity as its null hypothesis and a unit root as its alternative. Level stationarity means the series can be written as stationary fluctuation around a fixed level $\mu$ , as $y_t = \mu + \varepsilon_t$ . The fluctuation $\varepsilon_t$ may be autocorrelated, but departures from the level dissipate over time and the series keeps returning to the neighborhood of $\mu$ .

A series with a unit root has no such pull: the effect of each shock does not decay and stays in the level permanently, so the series wanders with no fixed level to return to. This persistent component is called a stochastic trend, and a random walk is the typical case.

The relationship between the two hypotheses becomes clear when the series is decomposed into the sum of a random-walk component and stationary fluctuation. Writing $y_t = \xi_t + \varepsilon_t$ with $\xi_t = \xi_{t-1} + u_t$ and $\operatorname{Var}(u_t) = \sigma_u^2$ , if $\sigma_u^2 = 0$ the component $\xi_t$ never moves from its initial value and the series is level-stationary, while if $\sigma_u^2 > 0$ the series carries a unit root. What KPSS tests is $\sigma_u^2 = 0$ .

Because the null hypothesis sits on the side of stationarity, differencing proceeds only on positive evidence against it, which makes the procedure conservative about differencing. The test statistic takes the cumulative sums of the mean-centered residuals and normalizes their magnitude by an estimate of the series' long-run variance. If a random-walk component is present, the cumulative sums grow without bound and the statistic grows with the sample size.

MIDAS uses the 5% critical value $0.463$ of the statistic's asymptotic null distribution as the decision threshold. It is not an absolute standard but a fixed constant for deciding whether to difference.

Why the sequential procedure works, and the role of the threshold

The procedure applies the test to the same series repeatedly, and each outcome decides whether to test again. This is sequential testing — exactly the shape that raises multiple-testing concerns. What justifies it is that no individual test result is interpreted as an inference. The whole procedure is treated as a single decision rule for $d$ , and what is evaluated is the probability that this rule returns a wrong $d$ under an assumed true model.

Suppose the true process carries $d_0$ unit roots. The rule's behavior splits into three cases.

Under-differencing (stopping before $d_0$ ): at each stage with $d < d_0$ , the differenced series still contains a stochastic trend and the KPSS statistic diverges with the sample size, so the probability of stopping early tends to 0 as the sample grows.
Stopping exactly at $d_0$ : at $d = d_0$ the series is stationary and the null holds, so the test falsely rejects with asymptotic probability $\alpha$ , and the probability of stopping exactly at $d_0$ tends to $1-\alpha$ .
Over-differencing (going past $d_0$ ): this happens with the remaining probability of about $\alpha$ . A series differenced past $d_0$ is also stationary and the null holds, so the over-differencing almost always stops after one extra order.

The $\alpha$ attached to the threshold thus directly controls the asymptotic probability of over-differencing. On the asymptotic evaluation alone, a smaller $\alpha$ always looks better: the probability of under-differencing tends to 0 with the sample size, so the only error left is over-differencing with probability $\alpha$ .

In finite samples, however, under-differencing does happen. At the stages with $d < d_0$ , the probability that the test detects the unit root — its power — is below 1, and for a series whose stochastic trend is weak relative to the stationary fluctuation, or for a short series, the test can fail to reject and stop early. Lowering $\alpha$ reduces over-differencing but also lowers the power and increases under-differencing. In finite samples, $\alpha$ sets the balance between the two errors.

Under-differencing leaves a stochastic trend in the model, making the fitted model non-stationary. Estimating AR coefficients presupposes a stationary process, so fitting, say, AR(1) to a random walk without differencing puts the true value at a coefficient of exactly 1, and the estimate piles up at the edge of the stationary region (between $-1$ and $1$ ) and becomes unstable. An AR coefficient sitting very close to 1 in the coefficient table is the symptom.

The cost of over-differencing is of a different kind. Differencing a stationary series once more leaves it stationary; instead, the process acquires an MA coefficient that offsets the unneeded difference. For example, differencing white noise one extra time produces an MA(1) with coefficient exactly $-1$ . Estimating an MA coefficient presupposes the inside of the region between $-1$ and $1$ (an MA inside this region is called invertible), so when the true value sits exactly on the edge of the region, the estimate piles up at the boundary and becomes unstable.

There is no theoretically optimal balance. Optimizing it would require an assumption about the strength of the stochastic trend and a specified loss function, and neither comes from the data. The 5% ( $\alpha = 0.05$ ) is a conventional value, tilted away from making $\alpha$ too small because the cost of under-differencing is weighed more heavily than the cost of over-differencing.

Three caveats. First, this evaluation assumes only stochastic trends. Because the KPSS test takes level (constant-only) stationarity as its null, the statistic diverges with the sample size even for a series with no unit root and only a deterministic linear trend — a trend-stationary series — so the series is all but certain to be differenced even though its true differencing order is 0. This is systematic over-differencing, separate from the over-differencing held to probability $\alpha$ . After the difference, the slope of the trend is estimated as the drift — the constant term of the differenced model — so the trend itself is not lost, but the costs of over-differencing arise all the same.

This systematic over-differencing can be avoided by choosing a model with a linear trend term. Enabling Include Linear Trend in the ARIMA tab (includeTrend in the Agent API) adds a deterministic linear trend term to the model, and the decision on the undifferenced series switches to the KPSS variant whose null hypothesis is trend stationarity (decision threshold $0.146$ ). For a trend-stationary series the null then holds, the statistic does not diverge, and the probability of differencing by mistake is held asymptotically to $\alpha$ , just as the level variant does for a level-stationary series. When the procedure stops at $d = 0$ , the slope of the trend is estimated by maximum likelihood together with the intercept and the AR/MA coefficients, and it enters the confidence intervals and the information criteria. Once the series has been differenced, a linear trend becomes a constant (the drift), so decisions on the differenced series revert to the level variant. The finite-sample limits on power remain, in the same form as for the level variant. Because the null side of the trend variant absorbs a deterministic trend, its power against a unit root at the same sample size is lower than the level variant's, so a series with a unit root can fail to be rejected and be fitted, undifferenced, as a model with a trend — under-differencing again. The sign is the same: an AR coefficient estimated very close to 1.

Second, KPSS has finite-sample size distortion and can over-reject under strong short-run autocorrelation, depending on the long-run variance estimate; the $\text{maxD}$ limit bounds that.

Third, the selected $d$ is a data-dependent random variable, so the confidence intervals for the coefficients are conditional on the chosen order — not specific to KPSS, but common to any automatic order selection.

Order selection in seasonal models

Specifying a seasonal period $s$ of 2 or more turns the model into ARIMA(p,d,q)(P,D,Q)[s] — also known as seasonal ARIMA, or SARIMA — adding the seasonal differencing order $D$ and the seasonal AR/MA orders $P, Q$ to the selection. The period $s$ is the number of observations per cycle — 12 for the yearly cycle of monthly data. The framework stays the same even so. Seasonal differencing — taking the difference from the value $s$ periods earlier — swaps out the data the model explains just as ordinary differencing does, so candidates with different $D$ cannot be compared by likelihood. The differencing orders $d, D$ are set by tests first, and AIC/BIC selection is limited to $p, q, P, Q$ under the same $(d, D)$ .

The procedure starts with the seasonal difference. Just as the level of a series with a unit root wanders, in a series with a seasonal unit root the effects of shocks stay in the seasonal pattern permanently and the pattern itself changes over time. MIDAS sets $D$ to 0 or 1 with the OCSB test — the seasonal unit-root test by Osborn, Chui, Smith, and Birchenhall — applied to the original series, and never selects a seasonal difference of order 2 or higher. As with KPSS, the test serves as a decision rule for whether to difference, and what MIDAS reports is only the selected $D$ and the operational fact of whether the statistic exceeded the decision threshold.

The ordinary differencing order $d$ is then set by the KPSS test on the seasonally differenced series. The difference from $s$ periods earlier is the sum of the last $s$ period-to-period changes, so a seasonal difference contains the work of one ordinary difference. Judging $d$ on the original series would count that difference a second time and over-difference, so KPSS is applied to the seasonally differenced series. The remaining orders $p, q, P, Q$ are chosen by fitting every combination with the selected $(d, D)$ held fixed and taking the candidate with the smallest AIC or BIC.

Combining ordinary and seasonal differencing can reach $d + D \geq 2$ — the airline model ARIMA(0,1,1)(0,1,1)[s] has $d = D = 1$ . At $d + D \geq 2$ MIDAS does not estimate a constant term, even when one is requested. A constant in a series differenced $d + D$ times integrates back to a deterministic polynomial trend of degree $d + D$ in the original series — a quadratic trend at $d + D = 2$ . At $d + D = 1$ the constant is kept and reported as the drift described above.

References

Hyndman, R. J., & Khandakar, Y. (2008). Automatic time series forecasting: The forecast package for R. Journal of Statistical Software, 27(3), 1-22. https://www.jstatsoft.org/article/view/v027i03

Also available as a Markdown file.