Search Results

You are looking at 1 - 10 of 24 items for :

  • "bias-variance tradeoff" x
Clear All
Marijn A. Bolhuis and Brett Rayner

Front Matter Page European Department Contents Abstract I. Introducing Optimal Pooling with Machine Learning II. A Two-Step Method for Optimal Pooling III. Applying the Method II. IV. Conclusions V. References BOX 1. The Bias-Variance Tradeoff FIGURES 1. Step 1—Proximity 2. Step 2—Optimal Pool 3. Turkey-Most Proximate Countries 4. Turkey-Least Proximate Countries 5. Turkey—Relative Forecast Errors for Different Pools 6. Other Countries—Relative Forecast Error of Different Pools ANNEXES I. Machine Learning and Cross

Marijn A. Bolhuis and Brett Rayner
We leverage insights from machine learning to optimize the tradeoff between bias and variance when estimating economic models using pooled datasets. Specifically, we develop a simple algorithm that estimates the similarity of economic structures across countries and selects the optimal pool of countries to maximize out-of-sample prediction accuracy of a model. We apply the new alogrithm by nowcasting output growth with a panel of 102 countries and are able to significantly improve forecast accuracy relative to alternative pools. The algortihm improves nowcast performance for advanced economies, as well as emerging market and developing economies, suggesting that machine learning techniques using pooled data could be an important macro tool for many countries.
Marijn A. Bolhuis and Brett Rayner

. There is a tradeoff between bias and variance in expanding a dataset to include data from additional countries. Adding countries with a similar economic structure (i.e., data-generating process) may reduce variance and improve forecasts, but adding countries with a dissimilar economic structure may introduce bias to the forecasts. Optimal pooling amounts to solving a version of the bias-variance tradeoff for which machine learning methods have specifically been developed. The aim of this paper is to use insights from machine learning to provide a method for selecting

Marijn A. Bolhuis and Brett Rayner

Front Matter Page European Department Contents ABSTRACT I. INTRODUCING MACHINE LEARNING II. THE BASICS OF FORECASTING—A BIAS-VARIANCE TRADEOFF A. Shortcomings of OLS-Based Forecasting Methods B. The Advantages of Machine Learning Methods III. A FRAMEWORK FOR MACRO FORECASTING WITH MACHINE LEARNING A. Limiting Preselection B. Identifying Complementary Algorithms C. Evaluating Performance and Interpreting Results IV. RESULTS—MORE ACCURATE FORECASTS V. CONCLUSIONS VI. REFERENCES ANNEXES I. The Bias-Variance Tradeoff II

Marijn A. Bolhuis and Brett Rayner
We develop a framework to nowcast (and forecast) economic variables with machine learning techniques. We explain how machine learning methods can address common shortcomings of traditional OLS-based models and use several machine learning models to predict real output growth with lower forecast errors than traditional models. By combining multiple machine learning models into ensembles, we lower forecast errors even further. We also identify measures of variable importance to help improve the transparency of machine learning-based forecasts. Applying the framework to Turkey reduces forecast errors by at least 30 percent relative to traditional models. The framework also better predicts economic volatility, suggesting that machine learning techniques could be an important part of the macro forecasting toolkit of many countries.
Marijn A. Bolhuis and Brett Rayner

Forecasting—a Bias-Variance Tradeoff All forecasting methods aim to minimize expected forecast errors. Forecasting consists of selecting a function that maps indicator data to a forecast while minimizing a particular loss function. Suppose a researcher wants to forecast a variable y t (e.g., real GDP growth) using K predictor variables summarized in the K x 1 vector X t , with the h-step ahead forecast of y t denoted as y t+h : y t + h = f ( X t ) + ϵ t + h where ε t+h is an

Klaus-Peter Hellwig
I regress real GDP growth rates on the IMF’s growth forecasts and find that IMF forecasts behave similarly to those generated by overfitted models, placing too much weight on observable predictors and underestimating the forces of mean reversion. I identify several such variables that explain forecasts well but are not predictors of actual growth. I show that, at long horizons, IMF forecasts are little better than a forecasting rule that uses no information other than the historical global sample average growth rate (i.e., a constant). Given the large noise component in forecasts, particularly at longer horizons, the paper calls into question the usefulness of judgment-based medium and long-run forecasts for policy analysis, including for debt sustainability assessments, and points to statistical methods to improve forecast accuracy by taking into account the risk of overfitting.
Klaus-Peter Hellwig

after controlling for a large number of candidate predictors, WEO forecasts contain valuable information that would otherwise not be captured by a linear prediction model – though less so for developing countries. But the gains in accuracy from reducing the weight of judgement in the IMF’s forecasts appear to be significant. The rest of the paper is organized as follows: First, I revisit the bias-variance trade-off. Then I apply Copas’ approach to detect overfitting in IMF forecasts. Section 4 benchmarks the accuracy of forecasts against a naive prediction rule and

El Bachir Boukherouaa, Khaled AlAjmi, Jose Deodoro, Aquiles Farias, and Rangachary Ravikumar

. Overfitting is one of the main challenges in ML and it occurs when the model performs very well on the training data set, but poorly on new, unseen data (test data). The better a model performs on the training data, the less biased it is. However, its performance can change dramatically on a set of unseen data. In this case, in order to achieve robustness in performance, the researcher may opt to choose a model that does not perform as well on the training set but maintains its performance on the test set. This situation is known as the bias-variance trade-off

International Monetary Fund

estimates. On the downside, by discarding past information, the variance of parameter estimates increases, which is reflected in larger forecast errors. In sum, when estimating model parameters under structural change, there is a balance between how much information one should use and how much of it should be discarded. In other words, there is a bias-variance tradeoff that can be exploited. 18. If the sample size is large enough, as it is typically in financial applications or in macroeconomic data available for advanced countries, one way to exploit the bias-variance