Macroblog

May 19, 2022

An Evaluation of GDP Nowcasts during the Pandemic

On April 28, the US Bureau of Economic Analysis reported that real gross domestic product (GDP) contracted an annualized rate of 1.4 percent last quarter. This decline "surprised" GDPNow, the Atlanta Fed's GDP tracking model, which had projected a 0.4 percentage point growth rate the day before the official release. Professional forecasters, who generally expected a rate around 1 percent based on economist surveys from Reuters and the Wall Street Journal (WSJ), also turned out to be overly optimistic.

The lines in chart 1 represent errors for final forecasts of first-release estimates of real GDP growth from the WSJ Economic Forecasting Survey and GDPNow. In the five-and-a-half years before the pandemic, the WSJ survey and GDPNow both had average absolute forecast errors—that is, without regard to sign, or mean absolute errors (MAEs)—of about 0.5 percentage points. Since then, the MAEs have been 1.7 percent and 1.9 percent, respectively. The stacked bars in the chart represent a decomposition of the GDPNow line into forecast errors of subcomponent contributions to GDP growth. The chart makes it evident that the size of the bars has fallen since 2020, but continues to be larger than before the pandemic. Even GDPNow's relatively accurate forecasts for the second and fourth quarters of 2021 were largely the result of the positive and negative subcomponent errors fortuitously offsetting each other. GDPNow's error last quarter was largely concentrated in net exports. The model's final forecast of the growth rate of real final sales to private domestic purchasers—which excludes inventories and government spending in addition to net exports and has been shown to be a better leading indicator of one-quarter-ahead GDP growth than GDP growth itself is—was only 0.3 percentage points below the initial estimate of 3.7 percent.

The deterioration in GDP forecasting accuracy during the pandemic has not been isolated to the shortest horizon projections. Chart 2 shows MAEs for roughly 75-day-ahead forecasts of growth rates of real GDP and its subcomponents (as well as net exports and inventory investment contributions to growth) from both GDPNow and the Philadelphia Fed's Survey of Professional Forecasters (SPF). The decline in forecast accuracy for the SPF has been similar to the decline for GDPNow across subcomponents, with the exception of the state and local (S&L) government spending subcomponent, where the deterioration for GDPNow has been much starker. A Macroblog post from a year ago discussed the reasons for this.

The left-hand panel of chart 3 shows the MAEs for subcomponent contributions to GDP growth prior to the pandemic, and the right-hand panel shows contribution MAEs during the pandemic. Because the errors have been so much larger during the pandemic, the vertical axis is scaled to be six times larger in the right-hand panel than in the left-hand panel.

To show how forecast accuracy has evolved over a typical quarter, the figures in both panels of the charts begin with the MAE after the initial (roughly 90-day ahead) GDPNow forecast, end with the final GDPNow forecast, and use up to five other forecasts following particular data releases in between. Three of these releases are from the Institute of Supply Management's Manufacturing ISM Report On Business (its manufacturing report, specifically) for each of the three months of the quarter being forecasted, and the other two correspond to the personal income and outlays releases from the US Bureau of Economic Analysis for the first two months of the quarter.

Notice that although both panels show the subcomponent projections generally becoming more accurate over time, the rank ordering of the subcomponent accuracy has changed in some nontrivial ways during the pandemic. In particular, even though personal consumption expenditures (PCE) on services account for 45 percent of nominal GDP, it was consistently one of the smaller sources of error prior to the pandemic. But during the pandemic, it has been one of the largest. Government spending was also one of the smaller sources of error prior to the pandemic and has remained that way during the early part of pandemic quarters. But this decade, the government spending forecasts have not tended to become more accurate as quarters have progressed, so that recently, just prior to the GDP release, government spending has been nearly as large a source of error as services PCE.

In the GDPNow model, the forecasts of government spending, particularly S&L government sales to other sectors, were distorted by the large swings in the second and third quarters of 2020. This documentation on recent changes made to the GDPNow model  describes the changes made to keep these types of distortions to a minimum in the future, which is relevant to the data releases used in chart 3. Generally, a few days after the release of personal income and outlays data, the ISM releases its manufacturing data for the subsequent month, providing one of the first data snapshots for that month. The ISM data for this month are used to estimate the model's factor, that then feeds through to the forecasting equations for much of the yet-to-be-released monthly GDP source data.

Chart 4 shows the MAE from the five quarters beginning in the first quarter of 2021 for real GDP growth and the S&L government contribution to GDP growth. The dashed lines are for the version of GDPNow that had been in use at the time, and the solid lines are for the "COVID-adjusted" model in use since the first quarter of 2022. (The same data releases used in chart 3 are used here.) The modified model shows a clear improvement in the forecasts of S&L government spending. Moreover, forecast accuracy no longer deteriorates following the release of the first-month and second-month ISM manufacturing releases. After the second personal income and outlays report of the quarter—and the final one before the initial GDP report—the accuracy and forecast values of the two models became more similar.

Although this is good news for improving the forecast accuracy of GDPNow, it is not clear whether near-term forecasts of GDP growth will continue to be less accurate than they were in the decade leading up to the pandemic. Ongoing geopolitical and economic uncertainties remain elevated, which could continue to have a negative impact on forecast accuracy for some time to come.

By Pat Higgins, a policy adviser and economist, and

Austin McTier, an economist research analyst, both in the Atlanta Fed's Research Department

• May 19, 2022 in
  • Forecasts
  • GDP
• Permalink
• Comments

May 6, 2021

How Has GDPNow Performed during the Pandemic?

According to the U.S. Bureau of Economic Analysis (BEA), real gross domestic product (GDP) grew at an annualized rate of 6.4 percent in the first quarter of 2021, 1.5 percentage points below our final GDPNow model forecast. The size of this forecast error is large relative to GDPNow's prepandemic history but much smaller than the extremely large errors recorded last year. In short, the GDPNow model has struggled to accurately capture the unusual sources of variation in GDP caused by the COVID-19 pandemic and the various policy responses. The colored dashed lines in chart 1 show the evolution of GDPNow's forecast errors for each of the last five quarters. Unlike any of the four quarters of 2020, GDPNow's forecasts of real GDP growth last quarter was always within 5 percentage points of the BEA's first published estimate. Nevertheless, we can see that the GDPNow forecasts for GDP growth last quarter were still less accurate than before the pandemic.

Despite the recent deterioration in GDPNow's accuracy, its forecasts over the past 15 months share some broad similarities with professional forecasts, as chart 2 shows. From mid-March to mid-May 2020, both GDPNow and the consensus Wall Street Journal Economic Forecasting Survey (WSJ) forecasts of real GDP growth declined sharply. As described here, GDPNow did not anticipate the extent of the plunge in services consumption last March and the resulting impact on GDP growth for the first quarter of last year. The nosedive and rebound in economic activity last year were evident in GDPNow's forecasts for GDP growth in the second quarter of 2020. Forecasts for third-quarter GDP growth strengthened during much of the summer and early fall, and GDPNow performed as well or better than the consensus forecast over much of that period. However, for the last two quarters, GDPNow forecasts were more erratic, and generally less accurate, than the consensus.

It's not surprising that GDPNow's accuracy has deteriorated relative to the consensus of professional forecasts. Unlike GDPNow, professional forecasters have been able to use data on personal mobility and government mitigation measures (such as here, here, here, and here), and a number of studies have shown that these measures relate to cross-country differences in GDP growth observed in 2020 (for example, here and here). Transportation services and leisure and hospitality accounted for half the decline in consumer spending in the first two quarters of 2020, and a number of high-frequency measures of activity in these sectors (such as here, here, here, and here) are available before official spending data are. Finally, spikes in consumer spending following the distribution of stimulus payments included in the December and March fiscal packages were evident in some high-frequency measures of consumer spending (available here and here) before they were evident in the retail sales and personal income and outlays releases. In contrast, GDPNow uses a fixed methodology and source data for estimating GDP, making it much less adaptable to unusual circumstances.

Nevertheless, GDP forecasting during the pandemic has been challenging even for professional forecasters. Chart 3 shows the distributions of GDPNow and WSJ survey panelist forecast errors of real GDP growth for projections made about 20 days before the BEA's release of its initial GDP estimate. The gray cross marks represent individual WSJ panelist errors, and the blue rectangles represent interquartile ranges (IQRs), or middle halves, of the distribution of their forecast errors. The IQRs widened during the pandemic, and a number of panelists had errors in the middle two quarters of last year that were dramatically larger than usual.

Still, it is plausible that the extreme fluctuations in the data used by the GDPNow model have embedded some inaccuracies into the GDPNow forecasts. To examine this question, let's consider GDPNow's final projections of the main subcomponent contributions to overall real GDP growth since the model was released to the public in 2014. Chart 4 shows the stacked forecast errors of these projections. By construction, the sum of the bars in any quarter is approximately equal to the black dot showing the difference between the first official estimate of real GDP growth and that quarter's final GDPNow forecast. The bars have become much larger since the pandemic, but especially for services consumption and state and local government expenditures (S&L). In this post, I've noted issues related to services consumption, but most of the huge third-quarter S&L error fell into a category referred to as "sales to other sectors ." GDPNow expected these sales—which are largely health, hospital, tuition, and other educational charges and fees—to continue plummeting in the third quarter instead of rebounding strongly, as they did. (The S&L contribution error is negative because the BEA subtracts these sales from the S&L subcomponent, and from GDP, to avoid double-counting). The S&L contribution errors in the last two quarters were still quite large, and mostly the result of the GDPNow model incorrectly continuing to forecast declines of these "sales to other sectors."

In the coming months, we plan to tweak the architecture behind GDPNow in hope of mitigating these sorts of errors. For instance, the day before the most recent GDP release, a modified version of the GDPNow model (one that reduces the influence of the pandemic on some of its forecasting equations) predicted first-quarter real GDP growth of 5.5 percent. More than half of the difference between this forecast and the official final GDPNow forecast of 7.9 percent growth was the result of a lower forecast for growth in the S&L subcomponent (1.3 percent), a number much closer to the BEA's S&L growth estimate (1.7 percent) than the final GDPNow forecast (14.6 percent). Such enhancements might help improve GDPNow's forecasting accuracy down the line. But GDPNow will remain susceptible to forecast inaccuracies whenever unusual events (such as the COVID-19 pandemic) hit the economy and dramatically shift spending patterns.

By Pat Higgins, a policy adviser and economist in the Atlanta Fed's Research Department

• May 6, 2021 in
  • Forecasts
  • GDP
• Permalink
• Comments

April 7, 2021

CFOs Growing More Optimistic, See Only Modest Boost from Stimulus Plan

During the past few months, alongside an increase in COVID-19 vaccinations and amid a fresh round of fiscal support, optimism about the economic recovery from the COVID-19 pandemic has grown. Although reasons for concern over the potential unevenness of the recovery still exist, many economists, policymakers , and market participants have ratcheted up their growth expectations for 2021.

This growing optimism extends to decision makers who participate in The CFO Survey—a collaborative effort among the Atlanta Fed, Duke University's Fuqua School, and the Richmond Fed. CFOs and other financial decision makers in our survey grew more optimistic about the U.S. economy and their own firms' financial prospects, according to the first quarter's data released on April 7. Moreover, these firms see stronger prospects for sales revenue and employment growth in 2021 (similar to results from other business surveys, including the Atlanta Fed's Survey of Business Uncertainty).

Many people think the recently passed $1.9 trillion American Rescue Plan Act (ARPA) is behind these brighter expectations. However, the results of our CFO Survey suggest that many firms anticipate that the fiscal stimulus will have only a modest impact on their own future business activity.

In the first-quarter CFO Survey (fielded March 15–26, 2021), we posed a question asking respondents about the impact that ARPA might have their own firm's revenue growth, number of employees, representative price (the price of the product, product line, or service that accounts for the majority of their revenue), and total wage and salary costs (see chart 1). Firms had five response options, ranging from "decrease significantly" to "increase significantly." A majority of firms expect the recent fiscal measure to have "little to no impact" across all areas of their business activity. The results are perhaps most striking for employment, as nearly 80 percent of firms anticipate ARPA to bring little to no change in that area.

Considering the tepid impact of the stimulus on employment expectations, the survey results for total wage and salary costs are also interesting. Here, nearly 30 percent of the panel anticipates modest to moderate upward pressure on wage and salary costs, with another 5 percent or so expecting "significant" impact on their wage bill. The reasons for the expected effect on firms' total wage and salary costs are unclear, but we should note that labor quality and availability remain very high on CFOs' list of most pressing concerns.

Expectations around ARPA's impact on revenue growth appear a bit more diffuse. Though the survey's typical (or median) firm still anticipates that the bill will bring little to no change in sales revenue growth, nearly 40 percent of respondents expect the legislation to have a positive impact on sales, and a very small share of firms anticipate a negative impact on revenue.

Given the nature of these responses, we were curious whether CFOs who anticipated a positive impact from ARPA also held higher quantitative expectations for firm-level growth than firms who saw little-to-no impact. t. The CFO Survey elicits firms' quantitative expectations for sales revenue, employment, price, and wage growth early in the questionnaire, providing a useful way to check for consistency. Table 1 reports these results.

Apart from firms' anticipated growth in wage and salary costs, it does appear that firms that foresee a boost from the fiscal stimulus also hold higher growth expectations. The increase in expectations is particularly stark for employment growth and prices.

If we dig a little deeper into the small share of firms anticipating increased employment due to the stimulus—45 total—we find that 40 of them are in service-providing industries and employ fewer than 500 workers. We know from academic research, government statistics, and anecdotal reports that the COVID-19 pandemic has hit smaller, service-providing firms particularly hard, so it's perhaps not surprising to see these types of firms expecting the stimulus to aid in a rebound. These firms are also anticipating a stimulus-induced boost to the prices they can charge. The price growth for services has slowed markedly since the onset of the pandemic. As the economy begins to open up more fully, these firms might believe that measures to bolster household income (among other aspects of ARPA) will lead to a bit more pricing power.

Overall, however, our results suggest that the majority of firms anticipate ARPA to have little to no impact on their sales revenue, employment, prices, and wages. The smaller share of firms that do anticipate increased activity resulting from the stimulus largely expect the increase to be modest to moderate.

Importantly, these results do not rule out a surge in growth as the pandemic recedes and the vaccination rollout continues. As we've noted, most CFOs expect growth to occur regardless of ARPA's role in that growth. But the survey shows that firms, in general, do not pin any surge in demand on the legislation.

By John Graham, the D. Richard Mead Jr. Family Professor of Finance at Duke University's Fuqua School of Business,

Roisin McCord, associate regional economist at the Federal Reserve Bank of Richmond,

Brent Meyer, policy adviser and economist in the Atlanta Fed's Research Department,

Emil Mihaylov, research analyst in the Atlanta Fed's Research Department,

Nick Parker, the Atlanta Fed's director of surveys, and

Sonya Ravindranath Waddell, a vice president and economist at the Federal Reserve Bank of Richmond

• April 7, 2021 in
  • Economic conditions
  • Economic Growth and Development
  • Employment
  • Forecasts
  • Inflation Expectations
  • Labor Markets
  • Pricing
• Permalink
• Comments

May 12, 2020

Challenges in Nowcasting GDP Growth

Real gross domestic product (GDP) declined at an annualized rate of 4.8 percent in the first quarter, according to the first estimate from the U.S. Bureau of Economic Analysis (BEA), 3.8 percentage points more than the decline anticipated by the Atlanta Fed's final GDPNow model projection. Why was the error, which was easily the model's largest on record for final GDPNow forecasts, so big? Chart 1 looks at GDPNow's forecast errors since the model went live in mid-2014 and breaks them down into forecast errors for the various subcomponents' contributions to GDP growth. 

The clear culprit is the fact that the GDPNow model did not anticipate the record 9.5 percent monthly decline (not annualized) in real services consumption in March. At that time, GDPNow had March services data available only for electricity and natural gas use and purchased meals and beverages, as well as revised February data for net foreign travel. If the model would have correctly forecasted the March growth rate in services consumption for the subcomponents besides these, it would have actually slightly overstated the first-quarter decline in real consumer spending. (I should note that because of the timing and impact of last quarter's social distancing efforts stemming from COVID-19, the BEA used data outside of the scope of its routine procedures to estimate part of services spending in March—in particular, data about private credit card transactions for health care and recreation services.)

By design, GDPNow is a purely model-based prediction method as opposed to the models of some private forecasters, who were able to incorporate developments related to COVID-19 into their April forecasts for first-quarter GDP growth in a way that GDPNow did not and could not. As a result, their GDP predictions turned out to be relatively more accurate. For example, the consensus forecast of first-quarter GDP from the Wall Street Journal Economic Forecasting Survey in the first full week of April was a decline of 3.3 percent, and the CNBC Rapid Update survey late in the week prior to the GDP release anticipated a decline in GDP of 5.3 percent. Private forecasts will continue to be able to use news developments and high-frequency or nonstandard data sources (such as initial unemployment claims and OpenTable restaurant dining data) in a way that GDPNow and similar nowcasting models do not. The New York Fed's recently introduced Weekly Economic Index combines a set of weekly indicators into a single index with units comparable to four-quarter GDP growth, but it does not actually nowcast quarterly GDP growth.

Around the time of recessions, macroeconomic projections from professional forecasters tend to be less accurate and show more dispersion than during nonrecessionary periods. And though the National Bureau of Economic Research has not identified a 2020 business cycle peak, recent GDP forecasts show much more dispersion than they did during, or close to, past recessions. Chart 2 shows the difference between the top 10 and bottom 10 average forecasts of real GDP growth (for both the current quarter and one quarter ahead) in the Blue Chip Economics Indicators survey administered in the middle month of each quarter since 1991.

The top 10/bottom 10 difference for current-quarter GDP forecasts in the May 2020 survey is clearly much larger than around past recessions. In fact, it's larger than the difference between the highest and lowest quarterly growth rate of GDP after 1983, around the time economists date the onset of the Great Moderation—in reference to the decline in macroeconomic volatility—in the mid-1980s. Prior to the May 8 employment release, GDPNow was more optimistic about second-quarter GDP growth than most private forecasters were, but after the model forecast was revised down from a decline of 17.6 percent to a decline of 34.9 percent on the heels of that report, it fell more in line with the others.

The dispersion in the forecasts for GDP in the third quarter of 2020 is even starker. The optimistic forecasters project 2020:Q3 growth to be well above the highest rate on record (15.7 percent in 1950:Q1), and the pessimistic forecasters project contracting GDP. Of course, we will not be able to determine how accurate forecasts of second- and third-quarter GDP growth are until later in the year. Nevertheless, the wide range of forecasts implies that at least some forecasters' GDP projections will be wildly off by historical standards. As St. Louis Fed economist Michael McCracken recently noted, what the late Yankees catcher Yogi Berra said is more true than ever: "It's tough to make predictions, especially about the future." Berra's wisdom also will also apply to producing accurate and reliable economic forecasts for some time to come.

By Pat Higgins, a policy adviser and economist in the Atlanta Fed's research department

• May 12, 2020 in
  • Forecasts
  • GDP
  • Monetary Policy
• Permalink
• Comments