214
Handbook of Discrete-Valued Time Series
ACF Pearson residuals PIT histogram
0.20
0.20
0.15
0.15
0.10
0.05
0.10
–0.00
0.05
–0.05
0.00
–0.10
F
m
(u*)
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
1 3 5 7 9 11 13 15 1 2 3 4 6 7 8 10 0.1 0.3 0.5
0.7 0.9
Lags u*
FIGURE 9.11
Graphical results for the third Monte Carlo experiment.
TABLE 9.5
Numerical Results for the Third Monte Carlo Experiment
GP(2) Model GP(1) Model
Pearson residual
Mean −0.0001 0.0001
Variance 1.0023 0.9999
Scoring rules
logs 1.7585 1.8363
qs 0.7845 0.8074
rps 0.8515 0.9111
AIC 87924.38 91838.88
BIC 87946.43 91852.11
G 2.551 4.710
p-value (0.9795) (0.8588)
generated data and lead to (marginal) overdispersion. The parameters have been chosen
such that the data can be tted to two stationary specications, namely, the GP(2) and the
GP(1) models as used in the previous experiment, as well as to the true specication. From
a practical point of view, if the harmonics were “too strong,” standard optimization rou-
tines may encounter problems tting the misspecied models to the data and we seek to
215 Model Validation and Diagnostics
avoid this unnecessary complication. The idea behind tting the two incorrect models is,
of course, to see which diagnostic tool(s) detect model misspecication when the deter-
ministic seasonality is ignored. In this experiment, we concentrate on the correlogram of
the Pearson residuals and the Tsay parametric bootstrap procedure using the sample serial
correlations as functional and set the sample size to 10, 000.
Figure 9.12 provides a row of panels corresponding to three tted models: the top row
arises from tting the true model with included regression effects, the second to a misspec-
ied GP(2) model ignoring harmonics, and the nal one to a misspecied GP(1) model
omitting harmonics. The top row of panels shows that neither the autocorrelation plot of
the Pearson residuals nor the parametric bootstrap analysis nds any evidence of misspeci-
cation; this is unsurprising, since the true generating process was tted. The middle panel
of the gure based on a GP(2) estimated model shows that both the ACF of the Pearson
residuals and the parametric bootstrap diagnostics indicate misspecication of this model.
However, the former shows this markedly less than the latter and then only at high lags. As
far as the tted GP(1) specication goes, it is evident that both diagnostic devices indicate
indubitably that this model is misspecied.
We conclude from these experiments that the diagnostic methods we present can be fruit-
fully applied in model validation and diagnostic checking in integer autoregressive models
for count time series. Moreover, all four potential problems mentioned in the penultimate
paragraph of Section 9.1 may be detectable with one or more of the tools discussed in this
chapter.
ACF Pearson residuals Parametric bootstrap diagnostics
(a)
0.20
0.15
0.10
0.05
–0.00
–0.05
–0.10
–0.15
–0.20
12 3 4 5 6 7 8 9 11 13 15 0 1 2 3 4 5 6 7 8 9 10 12 14 16
1.0
0.8
0.6
0.4
0.2
–0.0
–0.2
–0.4
FIGURE 9.12
Graphical results for the fourth Monte Carlo experiment. (a) Estimated model based on the true DGP.
(Continued)
216
Handbook of Discrete-Valued Time Series
ACF Pearson residuals
Parametric bootstrap diagnostics
0.20
1.0
0.15
0.8
0.10
0.6
0.05
0.4
–0.00
0.2
–0.05
–0.0
–0.10
–0.2
–0.15
–0.20 –0.4
123456789 11 13 15 012345678910 12 14 16
(b)
ACF Pearson residuals Parametric bootstrap diagnostics
0.20
1.0
0.15
0.8
0.10
0.6
0.05
0.4
–0.00
0.2
–0.05
–0.0
–0.10
–0.15
–0.2
–0.20
–0.4
1234 56789 11 13 15 0
12345678910 12 14 16
(c)
FIGURE 9.12 (Continued)
Graphical results for the fourth Monte Carlo experiment. (b) GP(2) estimated model, and (c) GP(1) estimated
model.
217 Model Validation and Diagnostics
9.6 Conclusions
Model validation and the assessment of the adequacy of a tted specication in discrete-
valued time series models has not been the primary concern of many authors in the eld
so far. However, it should form an integral part of any modern iterative modeling exercise,
and so we exposit a number of tools and methods to help ll this gap. Considering these as
probabilistic forecasting tools allows us to draw upon a well-developed body of literature
and adapt it for our purposes when such adaptation is required.
We focus on a range of diagnostic tools that help to assess the adequacy of t of a cho-
sen model specication. Many of these facilitate comparison between two or more model
specications. Some methods are graphical in nature, some are scoring rules, and others are
based on statistical tests. We demonstrate the applicability of our methods to two example
data sets, one each from economics and nance. Further, in a range of carefully designed
simulation experiments, we document the ability of these tools and methods to serve our
purpose for the popular class of integer autoregressive models.
We try to show how the search for a data coherent model for a count data set can be
aided by using some or all of the methods discussed. Despite the caveat quoted from Tsay
(1992) in Section 9.1, our experience is that model misspecication in integer time series
models will usually be revealed by more than one diagnostic device. Moreover, we would
counsel that no model should be used for probabilistic forecasting until it passes satisfac-
torily all the diagnostic checks advocated, for the subject matter of this chapter remains
an area of statistical methodology ripe for further development. From a practical point of
view, what we suggest here should be thought of as fairly minimal requirements that ought
to be satised by a proposed model for data. Further, any chosen model should perform at
least comparably with respect to other specications entertained with regard to summary
statistics such as scores. Finally, we hope that this work will achieve two aims: rst, to stim-
ulate researchers to improve on what we have put forward and second, to provide applied
count time series analysts with model checking techniques that can be routinely applied in
their current work.
References
Berkowitz, J. (2001). Testing density forecasts, with applications to risk management. Journal of
Business and Economic Statistics, 19:465–474.
Boero, G., Smith, J., and Wallis, K. F. (2011). Scoring rules and survey density forecasts. International
Journal of Forecasting, 27:379–393.
Box, G. and Jenkins, G. (1970). Time Series Analysis: Forecasting and Control. Holden-Day,
San Francisco, CA.
Chesher, A. (1983). The information matrix test: Simplied calculation via a score test interpretation.
Economics Letters, 13:45–48.
Czado, C., Gneiting, T., and Held, L. (2009). Predictive model assessment for count data. Biometrics,
65:1254–1261.
Dawid, A. P. (1984). Statistical theory: The prequential approach. Journal of the Royal Statistical Society:
Series A, 147:278–292.
Denuit, M. and Lambert, P. (2005). Constraints on concordance measures in bivariate discrete data.
Journal of Multivariate Analysis, 93:40–57.
218 Handbook of Discrete-Valued Time Series
Diebold, F. X., Gunther, T. A., and Tay, A. S. (1998). Evaluating density forecasts with applications to
nancial risk management. International Economic Review, 39:863–883.
Freeland, R. K. and McCabe, B. P. M. (2004). Analysis of count data by means of the Poisson
autoregressive model. Journal of Time Series Analysis, 25:701–722.
Geweke, J. and Amisano, G. (2010). Comparing and evaluating Bayesian predictive distributions of
asset returns. International Journal of Forecasting, 26:216–230.
Geweke, J. and Amisano, G. (2011). Optimal prediction pools. Journal of Econometrics, 164:130–141.
Gneiting, T., Balabdaoui, F., and Raftery, A. E. (2007). Probabilistic forecasts, calibration and
sharpness. Journal of the Royal Statistical Society: Series B, 69:243–268.
Gneiting, T. and Raftery, A. E. (2007). Strictly proper scroing rules, prediction, and estimation. Journal
of the American Statistical Association, 102:359–378.
Grunwald, G., Hamza, K., and Hyndman, R. (1997). Some properties and generalizations of
non-negative Bayesian time series models. Journal of the Royal Statistical Society: Series B,
59:615–626.
Harvey, A. C. and Fernandes, C. (1989). Time series models for count or qualitative observations.
Journal of Business & Economic Statistics, 7:407–417.
Joe, H. (1996). Time series models with univariate margins in the convolution-closed innitely
divisible class. Journal of Applied Probability, 33:664–677.
Jose, V. R. R., Nau, R. F., and Winkler, R. L. (2008). Scoring rules, generalized entropy, and utility
maximization. Operations Research, 56:1146–1157.
Jung, R. C. and Tremayne, A. (2011a). Useful models for time series of counts or simply wrong ones?
Advances in Statistical Analysis, 95:59–91.
Jung, R. C. and Tremayne, A. R. (2011b). Convolution-closed models for count time series with
applications. Journal of Time Series Analysis, 32:268–280.
Li, W. K. (2004). Diagnostic Checks in Time Serie. Chapman & Hall, Boca Raton, FL.
McCabe, B. P. M. and Leybourne, S. J. (2000). A general method of testing for random parameter
variation in statistical models. In Heijmans, R., Pollock, D., and Satorra, A., eds., Innova-
tions in Multivariate Statistical Analysis: A Festschrift for Heinz Neudecker, pp. 75–85. Kluwer,
London, U.K.
McCabe, B. P. M., Martin, G. M., and Harris, D. (2011). Efcient probabilistic forecasts for counts.
Journal of the Royal Statistical Society: Series B, 73:253–272.
McKenzie, E. (2003). Discrete variate time series. In Shanbhag, D. and Rao, C., eds., Handbook of
Statistics, Vol. 21, pp. 573–606. Elsevier Science, Amsterdam, the Netherlands.
Pavlopoulos, H. and Karlis, D. (2008). INAR(1) modeling of overdispersed count series with an
environmental application. Environmetrics, 19:369–393.
Psaradakis, Z., Sola, M., Spagnolo, F., and Spagnolo, N. (2009). Selecting nonlinear time series models
using information criteria. Journal of Time Series Analysis, 30:369–394.
Rosenblatt, M. (1952). Remarks on a multivariate transformation. Annals of Mathematical Statistics,
23:470–472.
Tsay, R. S. (1992). Model checking via parametric bootstraps in time series analysis. Applied Statistics,
41:1–15.
Wecker, W. E. (1989). Comment: Assessing the accuracy of time series model forecasts of count
observations. Journal of Business & Economic Statistics, 7:418–422.
Weiß, C. H. (2009). Modelling time series of counts with overdispersion. Statistical Methods and
Applications, 18:507–519.
White, H. (1982). Maximum likelihood estimation of misspecied models. Econometrica, 50:1–25.
Zhu, F. (2011). A negative binomial integer-valued GARCH model. Journal of Time Series Analysis,
32(1):54–67.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.