Central banks’ macroeconomic forecasts: When two do the same thing, it is not the same thing
What do central bankers base their interest rate decisions on? In this article, we compare the forecasting and analytical tools and processes applied by inflation-targeting central banks when making monetary policy. We draw on a questionnaire prepared by the Czech National Bank and distributed to the relevant units of 22 central banks in developed and developing economies in the second half of 2023. The responses reveal that a wide diversity of detail underlies the apparent similarity in central banks’ internal macroeconomic forecasting processes. That diversity manifests in the range of analytical and forecasting tools used and the degree of transparency in the monetary policy process. We also show how the ownership of the forecast varies across banks and how intensively the bank’s management is involved in preparing the macroeconomic forecast. Both have implications not only for the structure of the internal discussions inside the central bank, but also for the bank’s external communications. We conclude by considering how the Covid and energy crises occurring in rapid succession gave rise to a new challenge for forecasting and for the role of forecasts in the monetary policy process in a context of growing tensions in the global economy. These events, among others, are motivating many central banks to review their monetary policy frameworks and often their modelling frameworks as well.
Cast in the same mould
Our survey shows that central banks’ internal processes are very similar to each other in modern (inflation targeting-based) central banking. All the central banks (CBs) in our sample organise their monetary policy decision-making around a macroeconomic forecast prepared by experts at regular intervals using a combination of (semi-)structural macroeconomic models, statistical methods and expert judgement. All the CBs we surveyed complement the baseline scenario of their forecasts with alternative/sensitivity scenarios to capture key risks.
This similarity will be no surprise to those with knowledge of the field. Unlike other monetary policy regimes, inflation targeting is relatively new[1] and its intellectual tradition can be traced in specific academic publications. The operational procedures of inflation targeting have gradually been refined into “best practices” promoted by international financial institutions, most notably the IMF. The FPAS (forecasting and policy analysis) “gospel”, which summarises the main principles, has been and continues to be spread via technical assistance to more and more countries (see, for example, Adrian, Laxton and Obstfeld, 2018)[2]. In other words, the similarity is no accident; it is a result of casting in the same mould.
There are, however, striking differences between central banks in the transparency and management of processes and in analytical and forecasting tools and their use in the monetary policy process. Some CBs rely on a single core model, while others prefer to use a set of models. Closely linked with this are differences in the perceived hierarchy between structural macroeconomic models, short-term forecasts and expert judgement. In this article, we describe these differences in CBs’ practices in detail, looking at the trends in recent years and the outlook for the future.
Glass walls don’t suit everyone
Although inflation-targeting central banks have similar forecasting infrastructures and internal processes, the CBs we surveyed differ widely in their willingness to let the public look under the hood. In other words, there are differences in how transparent the CBs are.[3] Most of them publish their core model and its structure on their websites, but almost a third do not.[4] There turns out to be a clear correlation between disclosure of the core model and the transparency index constructed by Dincer, Eichengreen and Geraats (2022). This is not surprising given that published model documentation is one of the 15 transparency index criteria. However, other questions from our questionnaire that are not explicitly included in the index also correlate strongly with it. For instance, all the CBs in our sample prepare alternative/sensitivity scenarios in addition to the baseline forecast, but fewer than 40% publish them usually or always. The overwhelming majority of these are CBs that have the highest transparency index values. More transparent CBs have also more often published reports on monetary policy reviews conducted in the last five years, although the statistical relationship is less clear-cut in this case.
Yet even the most transparent CBs keep some forecasting results to themselves. The survey reveals that CBs that use a competing forecasting model alongside their core model do not publish alternative outlooks.[5]
Who owns the forecast?
There are also differences between central banks in the division of responsibility for the forecast between staff (experts) and the Board[6] (the decision-making body). Staff members are formally responsibility for the macroeconomic forecast in half (11) of the CBs surveyed, while the Board “owns” the forecast in five. In the rest, the ownership is shared – the forecast is assigned either by consensus to the institution as a whole or to the Board and staff jointly. In two CBs, the published forecast is owned by the Board, while the staff prepare an independent forecast internally as well (see Figure 1).
Figure 1 – Ownership of the forecast and the Board’s degree of involvement in preparing it
Source: Authors’ calculation
Most boards participate actively in forecasting even when the official forecast is assigned to staff. This involvement most frequently takes the form of consultation (13 CBs). Only in six cases does the Board completely refrain from involvement (see Figure 1). But even in these CBs, staff will often incorporate board members’ opinions in the form of alternative scenarios. In CBs where the Board “owns” the forecast, it is more closely involved and usually also determines the final wording of the forecast. The above differences between countries in the management of monetary policy processes are most probably linked with national specifics and historical tradition – we found no clear correlation with the level of development of the economy, the monetary policy regime, or the CB’s transparency or legal independence in the data. It seems that “old habits die hard”.
Where the legal status of the CB can play a role, however, is in the fiscal part of the macroeconomic forecast. Fiscal policy has a significant influence on the macroeconomic environment and hence on future inflation. This is due to measures taken by the government (such as tax changes), the existence of administered prices in the economy, and also the size of central and local government budgets in the economy and the fiscal stimuli ensuing from them. Inflation-targeting CBs must therefore predict fiscal policy in their forecasts and keep track of the legislative process. Most of the CBs we surveyed prepare their own independent forecasts of fiscal variables, but a significant minority (five) – mostly CBs that have less legal independence than the others surveyed – only use the forecasts of their national fiscal authorities.
The interplay between models and expert judgement
All the CBs we surveyed use structural models, data-driven methods and expert judgement to construct their macroeconomic forecasts, but the roles and hierarchy of these three inputs differ visibly between banks. An integrated model-based framework is key for almost all the CBs. This strategy often involves adjusting outcomes using auxiliary models and incorporating expert judgement. By contrast, only two CBs emphasise a judgement-based approach over a model-based one. In these cases, the macroeconomic model serves more as a consultation point for sectoral experts.
More than a third of the CBs rely on one key forecasting model, while the majority apply a concurrent approach, engaging with multiple models. Even within this diverse modelling setup, a clear hierarchy emerges, with certain models exerting more influence than others. Ten of the twelve CBs that reported using multiple models regard one of them as the core one. The remaining two CBs treat the outcomes of their models as equal and average them in their forecasts. Even among the CBs that regard one model as the core one, there are differences in their perception of the function of complementary models. Some described their role as auxiliary to the core model, for example for calibrating adjustments to the core model, assessing its properties or capturing channels that the core model lacks. Other CBs emphasise interpreting the set of models as a diverse range of views on the economy, offering a different perspective or providing a control/alternative to the core model. In some cases, a complementary model is employed primarily for alternative scenarios, owing to its richer structure of economic relations, while a less structural and more data-driven core model is used for the baseline scenario.
Compromises between theory and data (and other choices)
The modelling framework used to prepare the forecast has a strong bearing on CBs’ internal discussions and external communications. However, there are several forecasting models/model “families” available, and each CB faces compromises between intuitiveness, flexibility, theoretical foundations, data fitting and complexity of economic linkages when choosing its core model. For example, many CBs prefer semi-structural models, which try to strike a balance between data-driven approaches and structurual approaches, which rely more on theoretical relationships often derived also from microeconomic foundations.[7] The outputs of semi-structural models tend to be simpler and more intuitive than those of more complex Dynamic Stochastic General Equilibrium (DSGE) models, which contain a wider range of theoretical linkages and transmission channels. By contrast, other CBs favour DSGE models as their core tool because of the theoretical micro-foundations of the behaviour of households and firms and the internal consistency of all the variables in such models. Other approaches employed by CBs include time series models (often based on Bayesian econometrics), which call for minimal theory and let the data speak for itself as far as possible.
Another important decision for the modelling framework is whether the forecast should be conditional on a given external interest rate path (such as a constant rate level or market expectations), or whether it should be unconditional, with rates determined consistently with the other variables inside the model.[8] The majority of CBs use unconditional forecasts. However, this does not mean that they do not prepare sensitivity scenarios simulating the impacts of constant rates or a required rate path for certain monetary policy issues.
We asked several questions about the properties of the models used by the CBs in our survey. In our subsequent interpretation, we also took on board the detailed comments attached to the responses and the published model documentation. To ascertain similar behaviour, we applied a machine-learning classification method adapted to work with categorical data. It uses a relatively simple principle to split selected countries into clusters, or groups, according to the similarity of the responses.
Figure 2 – Modelling frameworks across groups
Source: Authors’ calculation
Note: Countries are allocated to groups on the basis of clustering of mixed-type data (including categorical data) using partitioning around the medoids based on the Gower distance.
Our classification algorithm found five groups in which the modelling framework and forecast preparation setups were relatively similar (see Figure 2). The relatively small group 1 contains countries where both structural and DSGE models are used and the forecasts are conditional but expert judgement plays the key role. In groups 2 and 3 (the largest ones in terms of numbers), CBs make more use of the standard modelling framework based on multiple models concurrently, and the forecasts tend to be unconditional. However, the two groups differ in that DSGE models tend to play the main role in group 3, whereas countries in group 2 more often rely primarily on semi-structural models (see Figure 3). Groups 4 and 5 are also similar to each other, relying on just one key model – DSGE models in group 4 and semi-structural models in group 5.[9] This means that relatively few CBs have setups similar to that of the Czech National Bank, i.e. a modelling system in which a DSGE model has the dominant role (group 4). Although DSGE models are the most widespread forecasting tool among the CBs we surveyed, the majority of CBs take a richer set of complementary models into account.
Figure 3 – Model hierarchy in central banks applying multi-model approaches
Source: Authors’ calculation
Note: In this figure, the sample of CBs is narrowed to those which rely on multiple forecasting models concurrently and described the hierarchy of those models in the questionnaire or model documentation. Countries are allocated to groups on the basis of clustering of mixed-type data (including categorical data) using partitioning around the medoids based on the Gower distance.
The choice of modelling framework depends on many factors. CBs of wealthy and advanced economies can be found in all our groups. However, where a CB reports a rather looser framework in which expert judgement plays a greater role than modelling, it ranks among those from the wealthiest countries (countries with the highest GDP per capita), as Figure 4 shows. The CBs of smaller countries that are catching up with wealthy neighbours tend to use a tried-and-tested recipe, but a whole range of other factors also play a role here. The IMF is undoubtedly an important authority in this regard, not only providing technical assistance in many CBs and introducing best CB governance practices, but also coordinating international technical and research collaboration and facilitating personal contacts between central bank experts.
Figure 4 – Characteristics by group
Source: Authors’ calculation
Note: x-axis – group numbers, y-axis – mean value of given indicator in group. Central bank independence à la Garriga (2016) and transparency à la Dincer et al. (2022).
Time for a change?
Central banks all around the world are facing criticism for having failed to predict the inflation surge in 2021 and repeatedly getting their subsequent inflation forecasts wrong. Unprecedented shocks to the global economy – the pandemic and the related lockdowns and fiscal stimuli, the subsequent global supply chain disruptions, the energy crisis and Russia’s invasion of Ukraine – have made their job harder. In our questionnaire, we asked the CBs how these shocks had affected their forecasting processes. We were also interested whether they were planning to make any major changes to their modelling frameworks or had conducted monetary policy reviews covering aspects of forecasting, or were planning to do so.
The CBs in the survey said they had encountered problems with modelling and preparing forecasts during the turbulent period. They mentioned, for example, high error rates of forecasts relying on past economic linkages, the breakdown of traditional seasonal equations, long data lags and large data revisions. The problems sometimes meant they had had to temporarily suspend their main modelling framework and put more weight on alternative data sources – mostly leading indicators – and expert judgement. Some CBs had added new sources of shocks to their core models. Another adaptation strategy had been to prepare more alternative scenarios reflecting the greater uncertainties and risks. CBs that generally rely more on expert judgement in the forecasting process[10] had met with fewer problems. Some institutions reported that the uncertainties had since diminished again and they had largely been able to return to previous practices in their forecasting processes; others, by contrast, said that the degree of expert judgement needed remained elevated.
Monetary policy reviews – especially those which focus on the macroeconomic forecast preparation process – also provide CBs with some feedback. A small set of CBs conduct reviews on a regular basis. This reflects their internal needs or legislative requirements. The majority of reviews are ad hoc, reflecting changes in communication or monetary policy, responding to difficult economic situations, or preceding the introduction of new instruments. The reviewers are usually CB staff, often supported by outside experts. CBs issue press releases on MP reviews. Most reviews are made publicly available, albeit occasionally in a partial format. CBs deem the reviews valuable and implement the recommendations arising from them. The quality of the forecasting framework tends not to be the core focus of MP reviews but is occasionally included. According to the CBs’ comments, MP reviews have not fundamentally questioned the quality of forecasts or modelling frameworks. However, they have frequently prompted CBs to make refinements or give priority to particular areas.
The overwhelming majority of the CBs we surveyed are not planning to make major changes to their modelling frameworks. Some of the respondents identify their adaptations during the turbulent period and the development of new tools as evidence of the flexibility of the current processes. Numerous others mention plans to further develop their current core models or expand the set of models they use. Only one plans to alter its main modelling framework. Several others emphasise the need to adopt a multi-model approach, particularly in the aftermath of the recent shocks. According to some CBs, the multi-model approach will improve their ability to navigate evolving economic conditions. These findings may be linked to the fact that no new modelling approach having the potential to replace structural, semi-structural or DSGE models has yet been implemented, nor has academia offered a clear candidate.
The CBs have a similar view of the evolution of their forecasting tools over the past 20 years. The majority (14 CBs) replied that they had not stopped using any specific macroeconomic model or modelling framework in the past 20 years. Changes are interpreted more frequently as gradual improvements to models or expansion of the set of models used. The minority of cases (8 CBs) in which a framework was abandoned in the past most often involved a move towards more structural modelling (e.g. extension of theoretical relationships or implementation of a DSGE model). In one case, by contrast, a DSGE model was gradually replaced by a semi-structural model. In another, a DSGE model was used to shadow the core model but did not replace it, as it had insufficient predictive power.
Conclusion
Several main findings arise from the Czech National Bank’s autumn 2023 survey. The first is the striking similarity of the CBs’ key internal processes. On closer inspection of the responses, however, we see minor differences in the way macroeconomic forecasts are prepared and in the role of models across CBs and indirectly also over time. This is in line with our expectations, because (to borrow from biology) each CB went through its own evolutionary process in this area and its modelling framework is thus endemic to some degree. The second finding is that CBs have differing levels of transparency, especially with regard to the preparation of their macroeconomic forecasts. We find, for example, that even the most transparent CBs conduct part of their forecasting exercises for internal purposes only. The third finding concerns the ownership of the forecast, which also varies across CBs in terms of the role played by management in the preparation of the forecast. In some CBs, the forecast is produced almost solely by experts; in others, it is created jointly by experts and managers; and in some, management plays the pivotal role.
A total of 22 inflation-targeting CBs, including four key reserve banks, took part in the survey. We would like to take this opportunity to thank all the participating CBs again for their willingness and openness to share monetary policy-making information. The fraught start to this decade – the Covid, energy and security crises and the related tensions in the global economy – naturally laid bare the unspoken truth that forecasting is far more difficult in crisis periods than in times of normal business cycle uncertainty. Economists know that the modelling framework is an auxiliary tool that gives structure and multi-dimensionality to the monetary policy debate, not a machine for making final monetary policy decisions. The threshold effect of the post-2020 events clearly increased the incentive for CBs to review their modelling approaches and macroeconomic forecasting processes (using internal or external experts or a combination of the two) in order to identify what they could do better in the event of similar future shocks and to make sure their processes are correctly set up and robust. We hope that the shared aggregated and anonymised results of our questionnaire will help all central banks through this difficult process.[11]
As the processors of the questionnaire, we believe that the sequence of events in recent years may prompt central banks to reconsider and, where necessary, revise their “best practices”. We feel that there is a need for a better understanding of some practices in those central banks which, for example, use several (competing and complementary) modelling outputs concurrently in their monetary policy decision-making – not only in the context of taking decisions on interest rates, but also in terms of making their decisions transparent and understandable for the public. We intend to address these and other issues in a follow-up questionnaire currently under preparation.
Authors: Soňa Benecká, Martin Kábrt and Luboš Komárek. The views expressed in this article are those of the authors and do not necessarily reflect the official position of the Czech National Bank. The authors would like to thank the questionnaire respondents for their time and responses, as well as Petr Polák, Michaela Ryšavá and Anna Drahozalová for their assistance in collecting, processing and interpreting the data.
References
Adrian, T., Laxton, D., and Obstfeld, M. (eds.) (2018): Advancing the Frontiers of Monetary Policy. Washington, D.C.: International Monetary Fund.
CNB (2003): The Czech National Bank´s Forecasting and Policy Analysis System. Czech National Bank, p. 97.
Dincer, N., Eichengreen, B., and Geraats, P. (2022): Trends in Monetary Policy Transparency: Further Updates. International Journal of Central Banking. March 2022.
Garriga, A. C. (2016): Central Bank Independence in the World: A New Dataset. International Interactions 42 (5):849-868.
Keywords: central banking, macroeconomic modelling, questionnaire
JEL Classification: C53, E58, C83
[1] New Zealand became the first inflation-targeting country in 1990.
[2] Implementation of FPAS in the Czech Republic is described by ČNB (2003).
[3] From the theoretical perspective, there is probably an optimal degree of transparency a CB can work towards. Lower-than-optimal and higher-than-optimal transparency both make a CB less understandable. However, it is not easy to determine the notional optimum level of transparency.
[4] Although some have published partial model documentation in research publications without disclosing the specific values of key parameters.
[5] However, complementary models are not always viewed as competing. We examine the different roles of these models later in the article.
[6] In this article, we use the term “Board” to mean the generally competent authority that decides collectively on monetary policy in the country (the Bank Board, the Monetary Policy Committee etc.).
[7] A specific type of semi-structural model is a popular choice; prominent examples include the Fed’s FRB/US model, the ECB’s BASE model, the Bank of Canada’s LENS model and the Bank of Japan’s Q-JEM model. Another popular framework is the gap Quarterly Projection Model, which is used in a whole range of developing economies as well as several developed countries.
[8] Here again, there is a middle-ground option where rates are determined by a simple monetary policy rule. Some CBs use this option, while one told us that it regularly prepares both conditional and unconditional forecasts.
[9] Near-term forecasting is significant in all CBs, with no systematic differences between banks. It plays a specific role in group 1, where the CBs reported a generally expert judgement-based approach to forecasting.
[10] In the Czech National Bank’s forecast preparation process, for example, near-term forecasting (NTF) played a larger role in the areas of GDP and wages. It was inserted into the model-based forecast instead of the standard coverage of the first quarter to one year ahead, replacing the purely model-based forecast from the core forecasting model (DSGE).
[11] The Czech National Bank is currently preparing a modelling exercise designed to determine what the outcomes of the core (DSGE) forecasting model probably would have looked like had we known the paths of the key economic variables that we normally base our simulations on (e.g. energy prices). This will form part of a broader monetary policy review to take place this year.