/lp_var_simul

Simulation study of Local Projections, VARs, and related estimators

Primary LanguageMATLABMIT LicenseMIT

Simulation study of Local Projections, VARs, and related estimators

Matlab code for large-scale simulation studies of impulse response estimators, including Local Projections (LPs), Vector Autoregressions (VARs), and several variants of these

Reference: Li, Dake, Mikkel Plagborg-Møller, and Christian K. Wolf (2024), "Local Projections vs. VARs: Lessons From Thousands of DGPs", Journal of Econometrics (published version, working paper, supplement)

Tested in: Matlab R2023a on Windows 10 PC (64-bit)

Contents

Documents: Paper, supplement, and documentation

Estimation_Routines: General-purpose impulse response estimation functions

DFM: Simulation study based on encompassing Dynamic Factor Model (DFM)

Replication

Main results

  1. Estimate IRFs from simulated data: Run the following scripts to select 6000 DGPs (under observed-shock identification), repeat 5000 Monte Carlo simulations for each DGP, and apply multiple estimators for each simulation. This step produces the raw IRF estimates.

    • In Settings/shared.m, set settings.specifications.random_n_spec = 100, and settings.simul.n_MC = 5000.
    • In run_dfm.m, first set estimand_type = 'ObsShock', lag_type = 4, and mode_type = 1.
    • After the setup above, run run_dfm.m 60 times, by varying the following, to iterate through 6000 DGPs:
      • dgp_type from 'G' (for fiscal policy type) to 'MP' (for monetary policy type);
      • spec_id from 1 to 30 (for 30 distinct seeds, where each seed draws 100 random DGPs). (Note: The code has been set up this way to allow the simulations to be split into several independent jobs on a research computing cluster.)
    • Raw IRF estimates will be saved in the directory "DFM/Results/". (Warning: File sizes will be very large.)

  2. Summarize key statistics: Run the following scripts to obtain summary statistics of raw IRF estimates across 5000 simulations. This step reduces the dimensionality of the results.

    • In run_combine.m, first set spec_id_array = [1:30], dgp_type = 'G' (or 'MP'). Additionally, specify estimand_type, lag_type, and mode_type to be consistent with Step 1.
    • Finally, run run_combine.m once to summarize the 5000 simulations as 9 summary statistics for each DGP.
    • These summary statistics will also be saved in the directory "DFM/Results/".

  3. Properties of selected DGPs and true IRFs: Run the following scripts to summarize the properties of the selected DGPs and their true IRFs (Section 3.4).

    • In Reporting/run_plot_dgp.m, set mode_select = 1, lags_select = 2, and exper_select_group = {[2,5]}.
    • Then run Reporting/run_plot_dgp.m to summarize selected DGPs (Table 1) and plot examples of true IRFs (Figure 1). (Warning: In Table 1, the row "IV first stage F-statistic" will be computed in Step 5 below.)
    • Outputs are all saved in the directory "Reporting/fig/".

  4. Evaluate loss for observed-shock estimators: Run the following scripts to show bias and variance profiles of each estimator under observed shock identification, and compare their loss at different bias weights and target horizons (Sections 5.1-5.3).

    • Always set mode_select = 1, lags_select = 2, and exper_select_group = {[2,5]} below.
    • Run Reporting/run_plot_loss.m to get bias and variance profiles separately for each estimator (Figures 2-3).
    • Run Reporting/run_plot_tradeoff.m to get head-to-head loss comparison between two estimators (Figures 4-5 and 7-9).
    • Run the Jupyter Notebook Reporting/plot_best_method.ipynb (with folder set to the output directory in Step 3), to depict the optimal estimator given different bias weights and target horizons (Figure 6).
    • Outputs are all saved in the directory "Reporting/fig/".

  5. Evaluate loss for IV estimators: Run the following scripts to get bias and variance profiles for estimators under IV identification (Figures 10-11).

    • Redo Steps 1-4, but change estimand_type = 'IV' in Steps 1-2, and exper_select_group = {[1,4]} in Steps 3-4.

Additional results in the online appendix

  1. Examples of estimated IRFs: Run the following scripts to plot examples of IRF estimates, as in Appendix E.

  2. Further evaluations for IV estimators: Run the following scripts to further examine bias and variance profiles of IV estimators, as in Appendix F.1.

    • Figures F.1-F.2 should have been generated in the outputs of Step 5.
    • For Figures F.3-F.4, repeat Step 5, but specify DGP_select to 2 (low degree of invertibility) or 3 (high degree of invertibility) in Reporting/run_plot_loss.m.

  3. Robustness checks: Run the following scripts to revisit the bias and variance trade-off in various extensions (Appendices F.2-F.10).

    • Each of the sub-steps below requires repeating Steps 1-4, but with slight adjustments.
    • Stationary DGPs (Table F.1, Figures F.5-F.7): change mode_type = 6 in Steps 1-2, and mode_select = 6 in Steps 3-4.
    • Recursive identification (Table F.2, Figures F.8-F.10): change estimand_type = 'Recursive' in Steps 1-2, and exper_select_group = {[3,6]} in Steps 3-4.
    • Salient observables (Figures F.11-F.13): change mode_type = 4 in Steps 1-2, and mode_select = 4 in Steps 3-4. (Warning: In Step 1, the code will not execute for sufficiently large values of spec_id, since the total number of available DGPs is exhausted. Simply skip to the next step when this happens.)
    • 90th percentile loss (Figures F.14-F.16): change loss_quant = 0.9 in Step 4.
    • Fiscal and monetary shocks (Figures F.17-F.20): change exper_select_group = {[2]} (to display fiscal shocks only) or exper_select_group = {[5]} (to display monetary shocks only) in Steps 3-4.
    • Longer estimation lag length (Figures F.21-F.23): change lag_type = 8 in Steps 1-2, and lags_select = 3 in Steps 3-4.
    • Smaller sample size (Figure F.24-F.26): change mode_type = 2 in Steps 1-2, and mode_select = 2 in Steps 3-4.
    • Larger sample size and estimation lag length (Figures F.27-F.29): change mode_type = 3 and lag_type = 12 in Steps 1-2, coupled with mode_select = 3 and lags_select = 4 in Steps 3-4.
    • More observables (Figure F.30-F.32): change mode_type = 5 in Steps 1-2, and mode_select = 5 in Steps 3-4. Also redo Step 5 for IV estimators.

  4. Splitting by variable categories: Table F.3 should have been generated in the outputs after finishing Steps 1-4 (Appendix F.11).

Acknowledgements

We rely on BVAR code by Domenico Giannone, Michele Lenza & Giorgio Primiceri, penalized LP code by Regis Barnichon & Christian Brownlees, as well as VAR model averaging code by Bruce Hansen. We have slightly modified these sets of code to improve their run-time without affecting their numerical output. We also use Dynamic Factor Model code and data by Eben Lazarus, Daniel Lewis, Jim Stock & Mark Watson.

Plagborg-Møller acknowledges that this material is based upon work supported by the NSF under Grant #2238049, and Wolf does the same for Grant #2314736.