Bias-Variance Illustration

[ckwolf92]

Hi both,

I've made some progress towards having a simple illustration of the bias-variance trade-off. For ease of tracking our discussions I've decided to open an "issue" here, rather than starting a new email chain. I hope that's alright.

The results are in lp_var_simul/Notes/20_1113, but I'll provide a quick summary here.

• The simplest possible DGP to illustrate the bias-variance trade-off and map into the Schorfheide (2005) results is an ARMA(1,1). In the note I set up that DGP, map it into the Schorfheide notation, and derive relatively simple closed-form solutions for the limit distributions of the AR and LP estimators. It would be great if you could check the mapping and see if I screwed up somewhere. In particular, I think that the V(lfe,p) expression in Theorem 1 of the Schorfheide paper needs to be fixed slightly.

• From the closed-form characterizations you can establish some neat results. First, for every horizon > 1, there is a bias-variance trade-off: The LP variance is always strictly bigger and the bias is always strictly smaller. Thus, there is always an interior "indifference weight" on bias. I'm pretty sure that it can be established that this weight is increasing in the horizon, consistent with our findings. Second, the "indifference curve" is uniformly shifted up or down with the distortion parameter \alpha.

• The attached figure illustrates this graphically. For large \alpha, and fixing the bias weight at 0.5, LP is preferred at short horizons, while for small \alpha the AR is preferred for all horizons. The right panel shows the corresponding indifference weights.

• For the paper, I'd prefer to have a bivariate illustration, simply because that will allow us to talk explicitly about the distinction between forecasts and structural IRFs. What do you think?

• Going back to some of the comments I got at UCL, I think we may phrase the motivating "impossibility result" as follows: Fix arbitrary second-moment properties of the data, as well as a model horizon $p$. Suppose that no VAR($p$) perfectly matches the stated second-moment properties, so instead write the DGP as a VARMA($p$,$\infty$). We now consider the Schorfheide (2005) experiment, asymptoting towards the VAR($p$) and so killing the MA component. Along this asymptote, for each horizon $h$, there exists a horizon-dependent interior cut-off on the bias weight $\omega^*(h)$ such LP is preferred for higher weights, and VAR is preferred for lower weights. Section 2 could state this result and then use a simple DGP to illustrate. The rest of the paper is all about the position of the indifference line in IRF horizon space -- in an empirically sensible DGP, is the line close to 0, or close to 1, or something in between?

Best,
Chris

[mikkelpm]

Now fully incorporated into paper.