Annual assessment of the forecasts included in GEO
Every month, Global Economic Outlook (GEO) provides an overview of the latest economic forecasts issued by international institutions, selected central banks and Consensus Economics. The outlooks for last year were strongly affected by the onset of the war in Ukraine and the resulting energy crisis in Europe. The monitored institutions were thus on average optimistic about the outlooks for GDP growth in 2022, expecting higher growth, which, however, failed to materialise. By contrast, higher-than-expected inflation came as a surprise. A long-running period of low interest rates combined with analysts’ conservatism were reflected in a substantial underestimation of outlooks for short-term interest rates at the one-year horizon for the euro area and the USA. The exchange rates of the monitored currencies against the dollar saw an increase in volatility last year due not only to uncertainties, but also differing inflation. The oil price was underestimated in the forecasts over the entire assessment period, as nobody expected another energy shock.
Published in Global Economic Outlook – July 2023 (pdf, 2.1 MB)
Introduction
Every year, we assess the accuracy of the forecasts of the economic variables regularly monitored in GEO. The results of this assessment provide valuable information about which of the monitored institutions produced estimates that were the closest to the subsequently recorded outcomes and were thus the most successful in their forecasts. In addition to Consensus Forecasts (CF), we assess the outlooks derived from market contracts when assessing the forecasts for interest rates, the dollar exchange rate and oil prices. The assessment always applies to the past year. In the case of the forecasts for GDP growth and CPI inflation for a given calendar year (fixed-event forecasts), we are now assessing the forecasts for 2022. In the case of the forecasts published for a fixed horizon that shifts further into the future each time a new forecast is published (rolling-event forecasts), the assessment covers the predictions since 2021. From the outlooks regularly published in GEO, this category of rolling forecasts contains, for example, the three-month and one-year outlooks for foreign interest rates, oil prices and the outlooks for exchange rates of the monitored currencies against the dollar. The general characteristics of the outlooks are quite obvious – the shorter the horizon, the more precise the forecasts become (see Chart 1).
Chart 1 – Gradual improvement of forecasts for 2022
(RMSE average)
Source: authors‘ calculation
Note: All the monitored institutions and all six regions (the USA, the euro area, China, the United Kingdom, Japan and Russia).
Owing to the short length of the time series under assessment, the analysis mainly uses the simple mean forecast error (MFE). The forecast error et is calculated as the difference between the ex post known actual value at and the corresponding forecast ƒt: et=at – ƒt. A positive forecast error therefore means that the forecasted value undershot the subsequent outcome, while a negative error means that it overshot it.[1]
Moreover, we use the RMSE indicator to assess the accuracy of the GDP and inflation growth forecasts across institutions. We also use the mean absolute percentage error (MAPE) to assess forecast accuracy across the currencies’ exchange rates against the dollar. This variable, expressed in percentages, is suitable for cross-checking variables of various dimensions. Moreover, the individual errors are given in absolute terms, so (similarly to the RMSE) positive and negative forecast errors do not cancel each other out, as is the case with the MFE. The formal notation is as follows:
Assessment of the accuracy of the GDP growth and CPI inflation forecasts for 2022
GEO regularly monitors actual and predicted GDP growth and CPI inflation in the euro area, the USA, the UK, Japan, China and Russia. The forecasts for GDP growth and inflation for these countries are taken primarily from the CF survey, the International Monetary Fund (IMF) and the Organisation for Economic Cooperation and Development (OECD). These three institutions cover all the countries monitored. In the case of advanced economies, we also monitor the forecasts of their central banks, i.e. the European Central Bank (ECB), the Federal Reserve, the Bank of Japan (BoJ) and the Bank of England (BoE). Oxford Economics forecasts are used for China and Russia instead. These institutions differ in the frequency and date of publication of their forecasts. The forecast updates are either monthly (CF and the OE) or quarterly (the IMF, OECD, ECB, Fed and BoJ). Assessed are quarterly forecasts, specifically the February, May, August and November forecasts.
The average expectations of economic growth in the monitored economies were higher than the actual outcomes. The opposite was true for inflation, except for China. Features of stagflation have thus appeared in the individual countries in connection with the war (see Chart 2). The charts in Appendix 1 show the deviations of the GDP growth forecasts from the actual outcomes in the monitored countries. The largest variability[2]> of outlooks for both GDP growth and inflation was naturally recorded for Russia, which was impacted not only by the war but also international sanctions. The euro area GDP growth outlooks were the most accurate (as measured by the RMSE). In addition, the euro area was characterised by low variability of forecasted growth. The GDP growth of 4.1% expected by CF at the start of 2021 ultimately came in at 0.6 pp lower. Much larger differences were seen for the other countries, with the biggest difference recorded by Russia (4.8 pp). The outlooks for most countries were most underestimated during 2021, i.e. even before the start of the war in Ukraine. The outlooks for economic growth were revised downwards after the outbreak of the war (by 14.7 pp in the case of Russia). The accuracy of the economic growth forecasts of all the institutions under comparison was very similar across all countries (see Chart 2).
Chart 2 – Comparison of the accuracy of institutions forecasting GDP and inflation for all countries
(RMSE)
Source: authors‘ calculation
Note: CF – Consensus Forecasts, IMF – International Monetary Fund, OECD – Organisation for Economic Cooperation and Development, CB/OE = central bank or Oxford Economics.
Owing to the energy crisis caused by the war and the fading effect of the post-pandemic recovery, actual inflation (with the exception of China) was well above its initial expected levels in early 2021 (see Appendix 2). There is therefore a clear inverse relationship between the forecast errors for economic growth and inflation according to CF for all countries under review (see Chart 3). Central banks’ inflation outlooks were characterised by similar inaccuracies as the outlooks of other institutions. The greatest inflation forecast variability was again specific to Russia and, to a lesser extent, the United Kingdom. The average expected inflation in the euro area for 2022 reached 1.1% in early 2021, but the actual outcome was a sizeable 7.3 pp higher. CF analysts had the most accurate inflation outlooks on average for all countries (see Chart 2). However, no more general conclusions can be drawn from the results for only one year, as the accuracy of forecasts usually changes over time and across institutions.[3] However, the charts capturing deviations in GDP and inflation growth show a general trend where the shorter the forecast horizon, the more accurate the forecasts gradually become (Chart 1).
Chart 3 – Relationship between the deviations in GDP and CPI
(%)
Source: authors‘ calculation
Note: CF forecasts for the countries under review.
Assessment of the accuracy of the forecasts for foreign interest rates
In GEO, interest rate outlooks are monitored for the euro area and the USA. In addition to the CF outlook, the monitored outlooks for three-month interest rates are also accompanied by an outlook derived from futures. By contrast, outlooks for long-term (ten-year) government bond yields are taken from CF only. As inflation and inflation expectations started to increase last year, central banks responded by increasing their rates. Now, in mid-2023, we are not done yet.
Market expectations regarding short-term interest rates were quite reliable from the second half of last year but still too conservative for one-year rates (see Charts 4 and 5). The ECB’s communication regarding the temporary nature of inflation gradually subsided in early 2022, and markets and analysts were unable to estimate when rates would start to increase. Rates increased for the first time in the middle of the year and the market started to estimate the future situation fairly well after that. But the analysts were more cautious about the tightness of the ECB’s monetary policy (see Chart 4a). Chart 4b, capturing the outlooks at the one-year horizon, shows a much larger degree of uncertainty in expectations and much more cautious analysts (CF outlooks) compared to market outlooks. Expectations regarding rate increases were very cautious and accelerated only in the second half of 2022. At the same time, one-year outlooks are now subject to a high degree of uncertainty regarding a subsequent decrease in rates, which market outlooks expect in about a year.
Chart 4a – 3M outlook for 3M interest rates in the euro area and a comparison with the actual outcome
(%)
Source: CF, Bloomberg
Note: The blue area indicates the range between the minimum and maximum prices in the given month. The vertical line indicates the end of the observed data.
Chart 4b – 1Y outlook for 1Y interest rates in the euro area and a comparison with the actual outcome
(%)
Source: CF, Bloomberg
Note: The blue area indicates the range between the minimum and maximum prices in the given month. The vertical line indicates the end of the observed data.
The outlooks for the United States are very similar; unlike the market outlooks, the analysts did not believe in such radical measures. The US Fed responded to inflation pressures faster, setting a very hawkish tone already at the end of 2021, which was reflected in growth in outlooks for both short-term and long-term rates (see Chart 5). It also holds true that market outlooks responded much more flexibly and faster than CF outlooks, which are thus significantly delayed and rather more conservative. The assessment also shows that there was a large difference between the market outlook and the analysts’ outlook at the one-year horizon, which still persists, and CF analysts do not believe in higher rates and expect them to decrease from the current high levels, whereas the market expects them to stay at just below 5% even in mid-2024 (see Chart 5b).
Chart 5a – 3M outlook for 3M interest rates for the United States and a comparison with the actual outcome
(%)
Source: CF, Bloomberg
Note: The blue area indicates the range between the minimum and maximum prices in the given month. The vertical line indicates the end of the observed data.
Chart 5b – 1Y outlook for 1Y interest rates for the United States and a comparison with the actual outcome
(%)
Source: CF, Bloomberg
Note: The blue area indicates the range between the minimum and maximum prices in the given month. The vertical line indicates the end of the observed data.
Assessment of the accuracy of the forecasts for the dollar exchange rate
In 2022, the monitored exchange rates were greatly affected by Russia’s aggression in Ukraine, which mainly affected the exchange rate of the rouble (see Chart 6). GEO provides information about the outlooks for the exchange rates of selected currencies against the US dollar based on CF forecasts. In addition, forward rates are provided for the euro, the Japanese yen and, since 2017, sterling. They are based on covered interest parity and represent the current ability to hedge the future exchange rate rather than the outlook. The accuracy of CF outlooks and market contracts does not differ, as illustrated in Chart 7, which shows their monthly evolution over more than the last two years.
Chart 6 – Forecast errors of the exchange rate of selected currencies against the dollar (3M outlooks)
(MAPE; right-hand scale: volatility)
Source: CF, Bloomberg
Chart 7 – 1Y outlook for 1Y interest rates for the United States and a comparison with the actual outcome
(USD/EUR)
Source: CF, Bloomberg
Note: The blue area indicates the range between the minimum and maximum prices in the given month. The vertical line indicates the end of the observed data.
Overall, the three-month outlooks for the euro-dollar exchange rate were relatively closely linked to the observed outcome in the period under review. Chart 6 illustrates first a weakening of the dollar against the euro since the start of 2020, followed by its appreciation from spring 2021 to the end of 2022 and then the appreciation of the euro. Both the market outlooks and CF outlooks have always been equally skewed and have been unable to capture a change in the trend at the three-month horizon.
The exchange rate of the Japanese yen was the most accurate of all the currency pairs at both the three-month and one-year horizons. The forecasts for the Japanese yen performed the best for most of the previous years. As usual, the forecasts for the Russian rouble were the worst performers at the three-month horizon (see Chart 6). The accuracy of the forecasts was also determined by the observed volatility of the monitored currencies, which, however, increased significantly last year compared to 2021 due mainly to uncertainty relating to Russia’s aggression in Ukraine.
Assessment of the accuracy of the Brent crude oil price forecasts
The Brent crude oil price is one of the most important commodity price outlooks presented in GEO. The accuracy of the forecasts for the Brent crude oil price according to both futures contracts and CF was the same on average. GEO regularly describes both sources of outlooks, and Charts 8 and 9 show that the values and trends of the outlooks differ only marginally. The oil price increased significantly at the start of last year due to Russia’s aggression in Ukraine, as Russia exported most of its production to Europe and European countries gradually started to rid themselves of their dependence on Russian oil. Of course, this situation could not be captured at both the three-month and one-year horizon. The prediction error was naturally higher at the one-year horizon, which reflected 2021 inflation. Market outlooks were able to better capture the subsequent price decrease than CF outlooks, which predicted a slower price decrease. The outlook is currently almost constant at around USD 75/bbl at the short-term horizon. Market outlooks expect the price to be around USD 70/bbl at the one-year horizon, while CF analysts expect it at slightly above USD 80/bbl. Owing to the ongoing conflict in Ukraine, a more pronounced price decrease cannot be expected.
Chart 8 – Three-month outlook for Brent oil price
(USD/barrel)
Source: CF, Bloomberg
Note: The blue area indicates the range between the minimum and maximum prices in the given month. The vertical line indicates the end of the observed data.
Chart 9 – One-year outlook for Brent oil price
(USD/barrel)
Source: CF, Bloomberg
Note: The blue area indicates the range between the minimum and maximum prices in the given month. The vertical line indicates the end of the observed data.
Conclusion
Last year saw a major unexpected external shock in the form of the war in Ukraine. The war significantly affected the original estimates of GDP growth and inflation in the countries under review, as well as commodity prices. The energy crisis in Europe caused an even more pronounced increase in inflation pressures and led, to a lesser extent, to a downward revision of GDP growth. Financial variables in particular were significantly affected by the long-running easy monetary policy of central banks and a lack of confidence in the rapid monetary policy tightening especially among the analysts.
This article uses simple methods to assess the accuracy of the forecasts monitored in GEO over the past year. The accuracy of the forecasts of the institutions covered by GEO changes from year to year. This is one of the reasons why several institutions’ forecasts are monitored in GEO. The accuracy of CF forecasts has long been comparable to the available alternative forecasts, which was also the case in 2022. In addition, CF has the advantage of being published monthly and covering a relatively wide range of economic variables. The accuracy of CF stems from its defining characteristic, namely that it is the simple average of the forecasts from the contributing private institutions.[4 The disadvantage is a slow response of predicted variables, especially in turbulent times, which then “lag behind” market outlooks.
Contrary to expectations, 2022 was another turbulent year, and the resilience of economies came as a surprise. Pessimistic expectations and high volatility were shaped by the initial shock caused by Russia’s aggression in Ukraine at the start of 2022. Subsequently though, economies adjusted relatively quickly to the sanction measures introduced and tensions on the energy market gradually moderated due to favourable weather. High demand for goods and a robust labour market coupled with energy prices contributed to the rapid growth in inflation, which was the main economic topic in 2022.
Authors: Filip Novotný and Petr Polák. The views expressed in this article are those of the authors and do not necessarily reflect the official position of the Czech National Bank.
Annexes
Annex 1 – Forecast errors for GDP growth for 2022
(percentage points)
Source: authors‘ calculation
Note: CF – Consensus Forecasts, IMF – International Monetary Fund, OECD – Organisation for Economic Cooperation and Development, ECB – European Central Bank, Fed – Federal Reserve System of the USA, Boe – Bank of England, BoJ – Bank of Japan, EIU - Economist Intelligence Unit. MFE is the mean forecast error for the given year.
Annex 2 – Forecast errors for consumer price inflation for 2022
(percentage points)
Source: authors‘ calculation
Note: CF – Consensus Forecasts, IMF – International Monetary Fund, OECD – Organisation for Economic Cooperation and Development, ECB – European Central Bank, Fed – Federal Reserve System of the USA, Boe – Bank of England, BoJ – Bank of Japan, EIU - Economist Intelligence Unit. MFE is the mean forecast error for the given year.
Keywords
Forecast error, economic outlook, Consensus Forecasts
JEL Classification
E66, E27, C18
[1] The source of actual levels of GDP and consumer price inflation growth for 2022 is the publication Consensus Forecasts. The source of the actual levels of the other variables is Refinitiv. Futures contracts for interest rates, exchange rates and the Brent crude oil price are obtained from the Bloomberg database.
[2] Variability is measured by the standard deviation in the analysis.
[3] A longer forecast period was assessed,for example, in Novotný and Raková (2010): Assessment of Consensus Forecasts Accuracy: The Czech National Bank Perspective. CNB WP.
[4] The characteristics of CF are described in more detail in an earlier article “How consensus has evolved in Consensus Forecasts” by Adam and Hošek in GEO 04/2015.