Forecasting and nowcasting: progress report
Jeremy Kwok
March 18th 2025
Understanding LSOAs and nowcasting the benefit claimant rate
Lower-layer Super Output Areas (LSOAs) are small geographical units in England, designed by the Office for National Statistics for reporting small-area statistics. Research indicates there are approximately 33,755 LSOAs, each with a typical population of about 1,500 residents or 650 households, ensuring consistent size for statistical comparison. The nowcasting model being developed forecasts the monthly benefit claimant count rate percentage for each LSOA, capturing localised socio-economic trends that broader regional analyses might miss.
This forecasting is vital for councils and public authorities, enabling them to allocate resources efficiently, develop targeted policies, and intervene early in areas with high benefit claim rates. For instance, identifying areas with rising claimant rates can help prioritise job training programs or social services, ensuring timely support for communities in need.
Nowcasting model
The first step in this nowcasting model development is to employ an AR(1) model—a straightforward yet effective tool—with performance visualised through various plots. This article delves into the methodology and key findings of this model.
What is an AR(1) model?
An AR(1) model (Auto Regressive, order 1) predicts future values based on the immediate past, offering a straightforward yet effective forecasting approach. This model serves as an initial benchmark in the development of a comprehensive nowcasting model suite. While the AR model provides a solid starting point, future iterations will incorporate more advanced methodologies, including machine learning techniques and multivariate factors, to enhance forecast accuracy.
Conceptually, using the AR(1) model is like predicting tomorrow’s weather based solely on today’s temperature. When applied to claimant rates, the model assumes that next month’s rate is a linear function of the current month’s rate, plus some random variation. Mathematically, this relationship is represented as:
In this equation, α is a coefficient derived from past data using ordinary least squares (OLS). For the 33,755 individual LSOAs, 33,755 AR(1) models were built to predict claimant rates.
In this equation, α is a coefficient derived from past data using ordinary least squares (OLS). For the 33,755 individual LSOAs, 33,755 AR(1) models were built to predict claimant rates.
Model development approach
1. Data preparation: The team began by reshaping claimant rate data into a long format with dates and rates for each LSOA. The dataset was divided into:
• Training data: Pre-2021 data to build the model.
• Actual data: Data from 2021–2022 to validate predictions.
2. Model fitting: An AR(1) model was applied to each LSOA using R’s ar function. To handle the vast dataset, LSOAs were processed in batches, covering over 33,755 areas.
3. Forecasting: Using the last value from the training data, the model generated 24 monthly forecasts (2021–2022) through iterative application of the AR(1) coefficient.
4. Error analysis: The model’s accuracy was assessed by calculating the absolute difference between predicted and actual rates, excluding extreme outliers with errors above 1,000.
The plot above shows predicted (red) versus actual (blue) claimant rates for a specific LSOA. The AR(1) model closely follows the actual data, especially during periods of stability and significant changes, demonstrating strong predictive accuracy. This is an ex-post forecast, meaning the model incorporates the latest available data, which enhances its predictive power and explains the high alignment between predicted and actual values.
The histogram above displays the distribution of forecast errors for the AR(1) model. Most errors cluster tightly around zero, indicating high accuracy in the model’s predictions across the majority of LSOAs. This result is expected in an ex-post forecast, where the model benefits from using the most recent data, minimising deviations between predicted and actual values.
Integration with our Annual Deprivation Index (ADI)
The nowcasting model complements our Annual Deprivation Index (ADI) which measures deprivation across regions using indicators like employment rates and levels of health deprivation.
Integrating the nowcasting model with the ADI enhances its predictive power by linking real-time benefit claimant forecasts with broader deprivation metrics. This synergy allows for a more comprehensive understanding of local economic conditions, supporting decision-makers in evaluating policies and allocating government resources effectively.
Next steps
In the next phase of this work, I’ll be developing a panel version of the AR(1) model. This will enable more effective model comparisons across different LSOAs and strengthen the overall forecasting approach. I’m also planning to explore machine learning techniques to build a more advanced nowcasting suite, allowing for more accurate and dynamic predictions of various economic and social variables.
Jeremy works within the Autonomy Data Unit (ADU) as a doctoral researcher (co-funded by Autonomy) focusing on nowcasting the UK economy.