Stock market volatility predictability: new evidence from energy consumption

Abstract

This research develops a group of novel indicators from the energy consumption perspective and assesses their ability to forecast stock market volatility using various techniques. Empirical evidence reveals that novel indicators, notably industrial non-renewable energy consumption, significantly enhance the forecasting of stock market volatility. The MIDAS-LASSO model, which integrates a mixed-data sampling method, effectively captures key information and outperforms other models in predictive accuracy. Further analysis reveals that the novel indicators contain useful forecasting information over the business cycle and crisis periods. Additionally, we indicate the forecasting ability of the novel indicators from the standpoint of investor sentiment variation. Our findings yield useful insights for the forecasting of stock market volatility, emphasizing the significant role of energy consumption.

Introduction

Stock market volatility, serving as a pivotal risk indicator, is essential for risk management and investment decision-making and has consistently been a focal point of financial market research (Bollerslev et al., 2018). Accurate volatility forecasting not only underpins asset pricing and portfolio allocation but also provides guidance for policymakers in formulating appropriate monetary policies (Engle and Ng, 1993). Accordingly, scholars strive to improve volatility forecast precision, employing new models and variables, as evidenced by contributions from Andersen et al. (2003), Bekaert and Hoerova (2014), Wen et al. (2016), and Wang et al. (2023). In the big data era, researchers refine stock market volatility predictions using diverse predictive factors, including macroeconomic variables (Paye, 2012; Engle et al., 2013), technical indicators (Neely et al., 2014), and investor sentiment (Behrendt and Schmidt, 2018). Confronted with crises, including the 2008 global financial crisis and the 2020 pandemic, traditional volatility predictors and models are often challenged, calling for new analytical perspectives.

Interestingly, previous research indicates that economic activity influences stock market performance (Cheung and Ng, 1998), which helps predict stock market volatility and the importance of the interplay between energy consumption and economic growth is well known (Zhang, 2011). Energy serves as a pivotal element in both production processes and the expansion of the economy and is intimately linked to economic activities, effectively functioning as a surrogate measure for them. As a strategic commodity, energy interacts with financial markets, potentially leading to cross-market volatility and risk spillover effects (Zhao et al., 2022). While the existing body of research predominantly focuses on fluctuations in energy price volatility, as noted by Bakas and Triantafyllou (2019), the ability of energy consumption to predict stock market fluctuations has been largely unexplored.

In this paper, we develop a set of energy consumption indicators from a new insight, and assess its ability to forecast stock market volatility, offering a novel perspective on stock market volatility using energy consumption. Notably, we conduct a systematic analysis of energy consumption by investigating the predictive power of this new set of indicators based on diverse energy-consuming sectors and subdivided non-renewable energy. Given that the stock market is affected by the economic environment and market conditions, we assess whether the newly developed energy consumption indicators retain constant predictive power across the business cycle and during various crises, in order to track stock market performance. Furthermore, we investigate whether fluctuations in investor sentiment influence the predictive efficacy of energy consumption indicators, considering the established idea that investor sentiment significantly impacts stock market dynamics, as documented by Baker and Wurgler (2006).

Traditional volatility forecasting literature employs autoregressive (hereafter, AR) models (Paye, 2012), which, however, suffer from a limitation in capturing the effects of higher-order lagged information. Li et al. (2022) find that MIDAS models efficiently harness the higher-order lags of fewer predictors, reducing overfitting and improving forecast accuracy. Based on this, this paper constructs nine single-factor MIDAS models (MIDAS-X) to investigate and evaluate the predictive capabilities of various energy consumption indicators on stock market volatility. However, maintaining stable predictive power under long-term conditions is extremely challenging for individual forecasters due to the volatility of financial markets (Ma et al., 2022). To achieve this, we employ four prevalent ensemble forecasting techniques—mean, median, trimmed mean, and discounted mean squared prediction error—coupled with three recognized dimensionality reduction approaches: principal component analysis (hereafter, PCA), partial least squares (hereafter, PLS), and scaled principal component analysis (hereafter, SPCA). These methods are used to bolster the stability of our model’s performance. However, the volatility of financial markets makes long-term stable forecasting a significant challenge. To fortify the model’s stability, our approach encompasses not only the application of ensemble forecasting and dimensionality reduction methods but also the integration of machine learning strategies. Specifically, we incorporate the Least absolute shrinkage and selection operator (LASSO) to develop MIDAS-LASSO models. We identify predictive signals within energy consumption data using these models, thereby facilitating the dependable anticipation of stock market volatility. LASSO is renowned for its capabilities in variable selection and regularization, effectively handling datasets with high multicollinearity. Compared to ensemble methods and dimensionality reduction models, LASSO more effectively captures the information of predictors, even if they are unexpected or transient (Chinco et al., 2019). Moreover, the sparsity feature of LASSO makes the model more interpretable, which is an important advantage for financial decision-makers.

Our contributions are as follows. Firstly, unlike most previous studies which concentrate on volatility forecasting using macroeconomic variables as discussed by Paye (2012), technical indicators as discussed by Neely et al. (2014), and investor sentiments, a focus of Behrendt and Schmidt (2018), we originally introduce a new set of indicators constructed based on an energy consumption perspective and assess, for the first time, their ability to forecast stock market volatility, aiming to provide a new approach and perspective in the related literature. Specifically, our analysis takes into account both primary and secondary energy consumption across industrial, commercial, and residential sectors, thereby providing a comprehensive reflection of actual economic activities. In this research, we concentrate our efforts on examining the interplay between U.S. energy usage and stock market volatility. Our dataset is derived from the U.S. Energy Information Administration (EIA), including a comprehensive and representative dataset, consisting of 498 monthly observations of energy consumption covering the period from April 1980 through to December 2021. Recognizing the strong link between energy consumption and economic dynamics, we hypothesize that shifts in U.S. energy usage could significantly impact stock market volatility. Consequently, this study is devoted to an exhaustive evaluation of the interplay between energy consumption and market volatility, with the objective of enhancing the theoretical underpinnings for volatility prediction models.

Secondly, we introduce the MIDAS-LASSO model to improve stock market volatility predictions. Our innovation is characterized by the combination of LASSO technology with the MIDAS method, resulting in the MIDAS-LASSO model. This model adeptly extracts predictive insights from energy consumption data, facilitating the reliable forecasting of stock market volatility. Leveraging the regularization and variable selection capabilities of LASSO, the model’s predictive accuracy and robustness are enhanced. The MIDAS-LASSO model enables us to distill essential insights from complex energy data, offering practical value to financial market participants. Our MIDAS-LASSO model demonstrates a significant improvement in the out-of-sample $R^2$ (hereafter, $R_{\mathrm{OOS}}^2$) value, increasing by 9.074%, a margin of enhancement that substantially surpasses that of the traditional third-order AR model and other competitive models, such as MEAN, MEDIAN, TMC, DMSPE-A, DMSPE-B, MIDAS-PCA, MIDAS-PLS, and MIDAS-SPCA. This significant improvement affirms the efficacy of our methodology and underscores the MIDAS-LASSO model’s distinct capabilities in delineating the interplay between energy consumption and stock market volatility. Furthermore, the MIDAS-LASSO’s performance at minimizing predictive errors is highlighted by comparison to the mean squared prediction error (hereafter, MSPE) and expected economic utility, showcasing its superiority over other models.

Thirdly, we apply LASSO to measure the predictive strength of various energy-consuming sectors and non-renewable energy and evaluate the influence of each indicator in out-of-sample tests, with a particular focus on their forecasting accuracy across business cycles and during crises. Furthermore, we divide investor sentiment into negative and positive levels to further evaluate how varying investor sentiment affects stock market volatility forecasting. Thus, this research paper parallels Baker and Wurgler (2006), pioneering the use of investor sentiment indices to forecast stock market volatility.

Overall, the contributions made herein hold substantial significance for the forecasting of market volatility, offering substantial insights for investors and policymakers, enabling a more profound comprehension of the determinants influencing stock market volatility. Concurrently, these contributions are instrumental in enhancing the efficiency of financial market operations.

Several important findings emerge from this study. Firstly, our newly constructed energy indicators have satisfactory volatility forecasting performance. The non-renewable energy consumption of the industrial sector, in particular, exhibits a notably consistent and superior performance. Secondly, integrated with the LASSO technique, the MIDAS model fully aggregates the information and has the best performance of any prediction approach. These intriguing results are validated by various robustness checks. We conduct further analysis, revealing that the newly established energy indicators have good predictive values even in complex and volatile economic environments with varying investor sentiment. Overall, our study presents innovative insights for forecasting stock volatility and informs investment and risk management strategies.

The structure of this research is delineated as follows: section “Literature review” is a literature review, examining the theoretical connections between energy consumption and stock market volatility. Section “Methodology” and section “Data” provide details of the forecasting methodologies and dataset used, with a focus on the development of energy consumption indicators. Section “Empirical results” presents and discusses the empirical results. Section “Robustness check” describes several robustness checks. Section “Further analysis” presents further analysis, assessing the predictive efficacy of energy consumption indicators across business cycles, during crises, and under fluctuating investor sentiment. Section 8 offers concluding remarks.

Literature review

Energy consumption serves as a pivotal metric for gauging economic activity, closely related to industrial production, transportation, and the overall health of the economy (Pirlogea and Cicea, 2012). Macroeconomic variables affect market integration and expected stock returns, thereby influencing stock market volatility (Bekaert and Harvey, 1995). As a component of the macroeconomy, changes in energy consumption reflect the intensity of economic activity and potential inflationary pressures, both of which can be closely related to stock market volatility. Therefore, changes in energy consumption can serve as a leading indicator of changes in economic activity, subsequently affecting stock market volatility.

Firstly, the interplay between energy prices and stock market volatility is examined in previous studies. On one hand, energy price volatility has a direct bearing on corporate expenses and consumer expenditure, which in turn influences corporate profitability and investor expectations in the stock market (Kilian, 2008; Oberndorfer, 2009). Although our study focuses on energy consumption rather than energy price, an increase in consumption may signal the risk of rising prices, thus affecting market volatility. On the other hand, investors may not always process information rationally (Kahneman and Tversky, 2013). Changes in energy consumption may affect stock market volatility by influencing investor sentiment and behavior.

Secondly, under the efficient market hypothesis, asset prices quickly reflect all available information (Fama, 1970), which suggests that any new information, such as changes in energy consumption, is promptly incorporated into stock prices by traders.

Last but not least, from the perspective of volatility forecasting theory, an effective predictive model should be able to capture all relevant information that affects volatility (Engle and Patton, 2007). Our empirical investigation reveals that energy consumption emerges as a significant predictor of stock market volatility, fulfilling the comprehensiveness criterion inherent in volatility forecasting models.

Our findings align with the extant scholarly discourse in several dimensions. Firstly, the preponderance of prior research concentrates on the repercussions of energy price volatility within financial markets (Oberndorfer, 2009; Jarrett et al., 2019). In contrast, our study introduces a novel perspective by scrutinizing the influence of energy consumption, thereby augmenting the research of Fan et al. (2017), who investigate the implications of energy efficiency for financial market dynamics. Secondly, our results substantiate the efficacy of the MIDAS-LASSO model in stock market volatility predictability, supporting the positive evaluations of machine learning methods for financial forecasting as reported in existing scholarly works (Alaminos et al., 2023; Masini et al., 2023). This suggests that machine learning methods can provide additional value compared to traditional financial models. Furthermore, our findings are consistent with the research of Bollerslev and Todorov (2011), who emphasize the importance of evaluating predictive models under different economic conditions. Our study further confirms the stability and predictive power of energy consumption indicators during periods of economic recession and turmoil. Lastly, the implications of our research for policymakers and market participants are in harmony with discussions in the literature regarding policy interventions and market efficiency. For example, Ren et al. (2023) discuss the potential impact of policy uncertainty on energy markets, while our study provides specific avenues for policymakers to influence stock market volatility by regulating energy consumption.

Methodology

Stock return volatility

Conditional volatility modeling methods rely on the ex-post variance measurement. Consequently, adhering to the methodology outlined by Paye (2012), we ascertain the monthly realized volatility (hereafter, RV) through the aggregation of the squared daily returns. To be precise, the RV for the t-th month is computed as:

1

$${\mathrm{RV}}_t=\sum _{i=1}^{N_{\mathrm{t}}}{r}_{i,t}^2$$where $N_{\mathrm{t}}$ signifies the count of trading days within the t-th month, and $r_{i,t}$ denotes the i-th daily return for that same month. However, the RV calculated in Eq. (1) is typically non-Gaussian, leading to inaccurate statistical inference when using ordinary least squares (OLS) for volatility prediction. Consequently, in line with Paye’s (2012) research, we apply the natural logarithm transformation to the RV.

Predictive model

Benchmark and MIDAS extension

The AR model, a standard in financial volatility forecasting, serves as our benchmark, following precedents set by Paye (2012) and Bakas and Triantafyllou (2019). Echoing Ma et al. (2018), we opt for a third-order lag:

2

$${\mathrm{RV}}_t={\beta }_0+\sum _{i=0}^2{\beta }_i{\mathrm{RV}}_{t-i-1}+{\epsilon }_t,$$where ${\beta }_0$ and ${\beta }_i$ are constant terms and estimated parameters, i represents the lag order, and ${\epsilon }_t$ denotes the error term.

Compared with the AR model, the MIDAS model has a stronger predictive ability (Li et al., 2022). Following Marsilli (2014), the MIDAS’s lag order is similarly established at three, expressed as:

3

$${\mathrm{RV}}_t={\beta }_0+\sum _{i=0}^2{\beta }_{\mathrm{RV}}W\left(i+1,{\rho }_1^{\mathrm{RV}},{\rho }_2^{\mathrm{RV}}\right){\mathrm{RV}}_{t-i-1}+{\epsilon }_t,$$where $W\left(i+1,{\rho }_1^{\mathrm{RV}},{\rho }_2^{\mathrm{RV}}\right)$ is the weight term. Following Lu et al. (2020), we set ${\rho }_1^{\mathrm{RV}}$ to 1. Correspondingly, the weight term can be formulated as:

4

$$W\left(i+1,{\rho }_1^{\mathrm{RV}},{\rho }_2^{\mathrm{RV}}\right)=\frac{g\left(i+1/3,1,{\rho }_2^{\mathrm{RV}}\right)}{{\sum }_{i=0}^{2}g\left(i+1/3,1,{\rho }_2^{\mathrm{RV}}\right)},$$where $g\left(i+1/3,1,{\rho }_2^{\mathrm{RV}}\right)$ is an anomalous integral function, which, according to Lu et al. (2022), can be simplified to:

5

$$g\left(i+1/3,1,{\rho }_2^{\mathrm{RV}}\right)=\frac{{\left(2-i\right)}^{{\rho }_2^{\mathrm{RV}}-1}F\left(1+{\rho }_2^{\mathrm{RV}}\right)}{3^{{\rho }_2^{\mathrm{RV}}-1}F\left({\rho }_2^{\mathrm{RV}}\right)}.$$

If we let x represent $\frac{i+1}{3}$, a represents ${\rho }_2^{\mathrm{RV}}$, then $F\left(a\right)={\int }_0^{+\infty }{e}^{-x}{x}^{a-1}dx$.

To harness the predictive capabilities of energy consumption indicators and bolster forecasting accuracy, we formulate the MIDAS-X model as follows:

6

$${\mathrm{RV}}_t={\beta }_0+\sum _{i=0}^2{\beta }_{\mathrm{RV}}W\left(i+1,{\rho }_1^{\mathrm{RV}},{\rho }_2^{\mathrm{RV}}\right){\mathrm{RV}}_{t-i-1}+\sum _{i=0}^2{\beta }_{X}W\left(i+1,{\rho }_1^X,{\rho }_2^X\right)X_{t-i-1}+{\epsilon }_t,$$where $X_{t-i-1}$ represents the predictor, including 9 energy consumption indicators.

Combination forecast

In the preceding section, 9 univariate models are used to investigate the new indicators’ predictive potential. However, due to fluctuating market circumstances, long-term sustained performance is challenging for individual predictors (Ma et al., 2022). Accordingly, we follow Rapach et al. (2010) and consider four popular combination forecasting techniques (mean, median, trimmed mean and discount mean square prediction error) to enhance the stability of model performance.

7

$${\widehat{\mathrm{RV}}}_{c,t}=\sum _i^N{w}_{i,t}{\widehat{\mathrm{RV}}}_{i,t},$$where combination forecast ${\widehat{\mathrm{RV}}}_{c,t}$ is formed by the weighted combination of N individual forecasts and $w_{i,t}$ is the weight term. The mean and median take the mean and median of N individual predictions, respectively. The trimmed mean (TMC) omits the extreme values, with weights $w_{i,t}=\frac{1}{N-2}$. For the discounted mean square prediction error, weights are calculated as $w_{i,t}=\frac{1}{{\Psi }_{i,t}{\sum }_n^N{\Psi }_{n,t}^{-1}}$, with ${\Psi }_{n,t}={\sum }_{b=d+1}^t{\psi }^{t-b}{\left({\mathrm{RV}}_b-{\widehat{\mathrm{RV}}}_{b,n}\right)}^2$ representing the discounted sum of squared forecast errors. We set the discount factor $\psi$ to 1.0 and 0.9, labeled DMSPE-A and DMSPE-B.

Dimensionality reduction

To address the overfitting risk that may arise in conventional forecasting, we explore dimensionality reduction methods. Following Huang et al. (2022), we incorporate PCA, PLS, and SPCA for dimensionality reduction and develop the subsequent models:

8

$${\mathrm{RV}}_t={\beta }_0+\sum _{i=0}^2{\beta }_{\mathrm{RV}}W\left(i+1,{\rho }_1^{\mathrm{RV}},{\rho }_2^{\mathrm{RV}}\right){\mathrm{RV}}_{t-i-1}+\sum _{i=0}^2{\beta }_{C}W\left(i+1,{\rho }_1^C,{\rho }_2^C\right)X_{t-i-1}^C+{\epsilon }_t,$$

9

$${\mathrm{RV}}_t={\beta }_0+\sum _{i=0}^2{\beta }_{\mathrm{RV}}W\left(i+1,{\rho }_1^{\mathrm{RV}},{\rho }_2^{\mathrm{RV}}\right){\mathrm{RV}}_{t-i-1}+\sum _{i=0}^2{\beta }_{D}W\left(i+1,{\rho }_1^D,{\rho }_2^D\right)X_{t-i-1}^D+{\epsilon }_t,$$

10

$${\mathrm{RV}}_t={\beta }_0+\sum _{i=0}^2{\beta }_{\mathrm{RV}}W\left(i+1,{\rho }_1^{\mathrm{RV}},{\rho }_2^{\mathrm{RV}}\right){\mathrm{RV}}_{t-i-1}+\sum _{i=0}^2{\beta }_{E}W\left(i+1,{\rho }_1^E,{\rho }_2^E\right)X_{t-i-1}^E+{\epsilon }_t,$$where $X_{t-i-1}^C$, $X_{t-i-1}^D$, and $X_{t-i-1}^E$ denote the principal components derived from energy consumption indicators using PCA, PLS, and SPCA, respectively1. In line with Lu et al. (2022), we employ the first principal component of the three techniques.

MIDAS model applying LASSO

To make effective use of potential prediction information, machine learning provides a fresh method for volatility forecasting. The LASSO technique proposed by Tibshirani (1996) captures sparse and transitory information (Chinco et al., 2019), and is widely used in forecasting. Hence, we integrate the LASSO technique with the MIDAS framework to optimally harness the insights from energy consumption. The resulting model is:

11

$${\mathrm{RV}}_t={\beta }_0+\sum _{j=1}^{K+1}{\beta }_j{\sum }_{i=0}^{2}W\left(i+1,{\rho }_1^{{\mathrm{RV}}^j},{\rho }_2^{{\mathrm{RV}}^j}\right){\mathrm{RV}}_{t-i-1}^j+{\epsilon }_t,$$where K is the total number of predictors. Specifically, the MIDAS-LASSO model controls the penalty intensity by adjusting the regulation parameter $\lambda$, expressed as:

12

$$\left[\widehat{\beta },\widehat{\theta }\right]=\arg \underset{\beta ,\theta }{\min }{\sum }_t{\left[{\mathrm{RV}}_t-{\beta }_0-\sum _{j=1}^{K+1}{\beta }_j\sum _{i=0}^{2}W\left(i+1,{\rho }_1^{{\mathrm{RV}}^j},{\rho }_2^{{\mathrm{RV}}^j}\right){\mathrm{RV}}_{t-i-1}^j\right]}^2+\lambda {\sum }_j∣{\beta }_j∣,$$where $\hat{\beta }$ and $\hat{\theta }$ are estimated shrinkage values.

Evaluation methods

Out-of-sample R²

Our research is anchored by dual objectives. The primary aim is to ascertain the predictive efficacy of novel energy indicators for stock market volatility. The secondary aim is to determine which forecasting approach can adeptly extract and utilize information from these variables to bolster predictive accuracy. To achieve these goals, we concentrate on the out-of-sample performance, scrutinizing the forecasting capabilities of each energy consumption indicator and model. Specifically, we apply the $R_{\mathrm{OOS}}^2$ method introduced by Campbell and Thompson (2008), to quantitatively evaluate and juxtapose the out-of-sample efficacy between the benchmark and competing models. The formula is:

13

$$R_{\mathrm{OOS}}^2=1-\frac{{\mathrm{MSPE}}_A}{{\mathrm{MSPE}}_B},$$where ${\mathrm{MSPE}}_j=\frac{{\sum }_{t=1}^O\left({\mathrm{RV}}_t-{\widehat{\mathrm{RV}}}_t\right)}{O}$, and $j=A,\mathit{B}$. A is the focal forecasting model, while B is the benchmark. O represents the out-of-sample period length. As a popular predictive test method, $R_{\mathrm{OOS}}^2$ captures the predictive differences of various models (Rapach et al., 2010; Paye, 2012). If the $R_{\mathrm{OOS}}^2$ value is greater than 0, its performance is better than the benchmark. In accordance with Clark and West (2007), we use the adjusted MSPE (MSPE-adj.) metric to evaluate the forecasting performance.

Economic value

Investors are more concerned with a model’s economic value than its forecasting performance in order to improve risk management. Consequently, we employ the mean-variance utility framework proposed by Bollerslev et al. (2018) to assess the economic value, which mirrors the anticipated utility of investors when constructing portfolios comprising both risky and risk-free assets. According to Liu et al. (2022), the expected utility function is:

14

$$E_t\left[U\left(W_{t+1}\right)\right]=E_t\left(W_{t+1}\right)-\frac{\theta {\mathrm{Var}}_t\left(W_{t+1}\right)}{2},$$where $\theta =-\frac{U^{\prime \prime }}{U^{\prime }}$ represents the investor’s coefficient of risk aversion, and $W_{t+1}$ denotes the asset allocation strategy for period $t+1$. It is hypothesized that investors allocate a proportion ${\phi }_t$ of their investment portfolio to the S&P 500 index, anticipated to generate a return $r_{t+1}$, and the balance to risk-free assets, which are expected to yield $r_{t+1,f}$. Consequently:

15

$$\begin{array}{c}{W}_{t+1}=\left[{\phi }_t{r}_{t+1}+\left(1-{\phi }_t\right)r_{t+1,f}+1\right]W_t\hfill \\ =\left({\phi }_t\widehat{r_{t+1}}+r_{t+1,f}+1\right)W_t.\hfill \end{array}$$

In this context, $\widehat{r_{t+1}}$ signifies the excess stock return, defined as $\widehat{r_{t+1}}=r_{t+1}-r_{t+1,f}$. Consequently, the expected utility is:

16

$$U\left({\phi }_t\right)=\left[{\phi }_t{E}_t\left(\widehat{r_{t+1}}\right)-\frac{\theta {\phi }_t^2{\mathrm{Var}}_t\left(W_{t+1}\right)}{2}\right]W_t=\left[{\phi }_t{E}_t\left(\widehat{r_{t+1}}\right)-\frac{\theta {\phi }_t^2{E}_t\left({\mathrm{RV}}_{t+1}\right)}{2}\right]W_t.$$

Subsequently, the optimal ex-ante allocation to the stock index for day $t+1$ is:

17

$${\phi }_t=\frac{{\theta }^{-1}{E}_t\left(\widehat{r_{t+1}}\right)}{E_t\left({\mathrm{RV}}_{t+1}\right)},$$where $E_t\left(\widehat{r_{t+1}}\right)$ and $E_t\left({\mathrm{RV}}_{t+1}\right)$ denote the expected $\widehat{r_{t+1}}$ and ${\mathrm{RV}}_{t+1}$, respectively.

Volatility risk modeling has received much attention recently (Ma et al., 2018; Lu et al., 2020; Li et al., 2022; He et al., 2024). To this end, we introduce the conditional Sharpe Ratio (SR) as:

18

$$\mathrm{SR}=\frac{E_t\left(\widehat{r_{t+1}}\right)}{\sqrt{E_t\left({\mathrm{RV}}_{t+1}\right)}}.$$

To optimize utility, leveraging the optimal weight ${\phi }_t^{*}=\frac{\mathit{SR}}{\theta \sqrt{E_t\left({\mathrm{RV}}_{t+1}\right)}}$, we derive the maximum utility $U\left({\phi }_t^{*}\right)=\frac{{\mathrm{SR}}^2{W}_t}{2\theta }$. However, in practical application, $E_t\left({\mathrm{RV}}_{t+1}\right)$ is usually unavailable. Therefore, we take the forecasting value $\widehat{{\mathrm{RV}}_{t+1}}$ as a substitute:

19

$$U\left(\widehat{{\mathrm{RV}}_{t+1}}\right)=\left(\frac{\sqrt{{\mathrm{RV}}_{t+1}}}{\sqrt{\widehat{{\mathrm{RV}}_{t+1}}}}-\frac{{\mathrm{RV}}_{t+1}}{2\widehat{{\mathrm{RV}}_{t+1}}}\right)\frac{{\mathrm{SR}}^2}{\theta }.$$

Consequently, the mean expected utility is given by:

20

$$Ū\left(\widehat{{\mathrm{RV}}_{t+1}}\right)=O^{-1}{\sum }_{t=T-O}^{T-1}U\left(\widehat{{\mathrm{RV}}_{t+1}}\right).$$

Following the parameters established by Bollerslev et al. (2018), we assign values of 0.4 to SR and 2 to θ.

Direction-of-change test

We employ the direction-of-change (hereafter, DoC) test as a complementary metric. According to Liang et al. (2022), DoC succinctly encapsulates the pivotal trading strategies for market timing and asset allocation, thereby facilitating the evaluation of the effectiveness of volatility directional change. To this end, we introduce a binary variable $D_t$ such that it takes a value of 1 when the model correctly anticipates the directional change in ${\mathrm{RV}}_t$, and 0 in all other cases. This binary variable is given by:

21

$$D_t=\left\{\begin{array}{c}1,\mathrm{if}{\mathrm{RV}}_{t-1}>{\mathrm{RV}}_t\mathrm{and}{\mathrm{RV}}_{t-1}>{\widehat{\mathrm{RV}}}_t\hfill \\ 1,\mathrm{if}{\mathrm{RV}}_{t-1}<{\mathrm{RV}}_t\mathrm{and}{\mathrm{RV}}_{t-1}<{\widehat{\mathrm{RV}}}_t\hfill \\ 0,\mathrm{otherwise}\hfill \end{array}\right).$$

Therefore, the DoC ratio for correctly predicting the direction of volatility is:

22

$$\mathrm{DoC}=\frac{{\sum }_{t=1}^P{D}_t}{P},$$where P represents the count of out-of-sample forecast values. We use the non-parametric test proposed by Pesaran and Timmermann (2009) to evaluate the model’s direct forecasting accuracy. In essence, the DoC ratio serves as an indicator of the model’s predictive precision.

Forecast-encompassing check

Motivated by Rapach et al. (2010) and Neely et al. (2014), we implement the forecast-encompassing check to evaluate the comparative informational content of diverse predictors and models. Specifically, we examine which of the new energy indicators contain the most information. We assess whether there is a method that can contain information from not only energy indicators but also other forecasting methods. Mathematically, the forecast-encompassing check is:

23

$${\mathrm{RV}}_t=\left(1-\gamma \right){\widehat{\mathrm{RV}}}_{a,t}+\gamma {\widehat{\mathrm{RV}}}_{b,t},$$where ${\widehat{\mathrm{RV}}}_{b,t}$ and ${\widehat{\mathrm{RV}}}_{a,t}$ represent the forecasted values for the primary and comparative models, respectively. The condition $\gamma \left(1-\gamma \right)=0$ signifies an encompassing relationship between the two models. Specifically, $\gamma =0$ indicates that model a encompasses model b in terms of forecast information, whereas $\left(1-\gamma \right)=0$ suggests that model b contains the information of model a. Guided by this, we adopt the null hypothesis $H_0:\gamma \left(1-\gamma \right)=0$, as per Harvey et al. (1998), to ascertain whether the model in question encompasses the information from the alternative model.

Data

Our study examines the forecasting power of various energy consumption measures for stock market volatility. Considering the S&P 500 to be an objective and reliable reflection of market movements, we procure daily closing prices from investing.com to compute the RV. Concurrently with the RV of the S&P 500 index, we incorporate data pertaining to energy consumption for our analysis. This study uses monthly energy consumption data for the United States provided by the EIA2. We divide energy consumption into residential, commercial, and industrial sectors according to the sources. Specifically, we classify energy into primary and secondary levels. Primary energy is categorized into renewable and non-renewable subsets based on its capacity for short-term replenishment. For secondary energy, we select electricity, being the most prevalent energy form, as a representative indicator. We take sales of electricity to ultimate customers as a proxy to measure consumption, which accurately reflects production and business activities. Table 1 gives the details.

Table 1. Variable description.

Category		Variable	Detailed information
Forecasting target		RV	Calculated as the Napierian logarithm difference of the S&P 500 index realized variance.
Primary energy	Renewable energy	RER	Calculated as the change rate of residential renewable energy consumption.
		REC	Calculated as the change rate of commercial renewable energy consumption.
		REI	Calculated as the change rate of industrial renewable energy consumption.
	Non-renewable energy	NOR	Calculated as the change rate of residential non-renewable energy consumption.
		NOC	Calculated as the change rate of commercial non-renewable energy consumption.
		NOI	Calculated as the change rate of industrial non-renewable energy consumption.
Secondary energy: main source, electricity		ELR	Calculated as the change rate of residential electricity sales to ultimate customers.
		ELC	Calculated as the change rate of commercial electricity sales to ultimate customers.
		ELI	Calculated as the change rate of industrial electricity sales to ultimate customers.

This table delineates the specific details pertaining to realized volatility and indicators of energy consumption.

Taking into account the integrity and accessibility of data, our dataset encompasses a period of 498 months, from April 1980 to December 2021. This extensive timeframe offers ample data to scrutinize the interconnection between energy consumption and stock market volatility. The test set encompasses an out-of-sample duration, comprising 331 data points from April 1994 to December 2021. To mitigate the issue of data overlap, we employ a rolling window approach in our analysis. Figure 1 shows the logarithmic RV trend over the sample. Intuitively, the stock market volatility fluctuates violently, reaching its peak during the financial crisis.

Fig. 1 [Images not available. See PDF.]

Realized volatility of Standard & Poor’s 500 index.

Significantly, we exclude seasonal effects of energy consumption, making the statistics more realistic. In order to maintain the magnitude order, we use the energy consumption change rate as an indicator:

24

$$z_t=\frac{x_{t+1}-x_t}{x_t},$$where $z_t$ represents the monthly change rate, and $x_t$, $x_{t+1}$ indicates the energy consumption for consecutive months. Our analysis focuses on nine energy consumption indicators derived from these change rates3, as detailed in Table 2. The data series exhibit a right-skewed distribution and meet the augmented Dickey-Fuller (hereafter, ADF) test requirements, validating their use in the empirical analysis.

Table 2. Descriptive statistics.

Variable	Mean	Std. dev.	Skewness	Kurtosis	Jarque-Bera	Q(3)	Q(9)	ADF
RV	−6.516	0.872	0.424	0.396	17.814***	446.651***	843.408***	−10.636***
RER	0.004	0.300	−2.597	49.804	50870.876***	5.462	7.377	−24.525***
REC	0.039	0.289	−0.183	10.261	2134.608***	43.927***	49.283***	−30.086***
REI	0.026	0.597	1.455	11.546	2873.825***	62.952***	88.434***	−30.162***
NOR	0.062	1.207	1.148	6.725	1023.015***	66.099***	68.270***	−31.603***
NOC	0.035	0.839	0.764	5.326	621.211***	63.040***	65.054***	−31.125***
NOI	0.003	0.306	0.250	2.370	117.981***	68.064***	81.745***	−30.232***
ELR	0.022	0.414	0.248	1.112	29.626***	40.791***	45.080***	−27.126***
ELC	0.020	0.182	0.098	1.009	20.941***	54.416***	66.542***	−29.478***
ELI	0.006	0.141	−0.241	2.807	163.321***	36.150***	37.811***	−28.011***

The table displays descriptive statistics from April 1980 to December 2021. Std. dev. is the standard deviation. The Jarque-Bera test assesses normality. Q(n) is the Ljung-Box autocorrelation statistic. ADF is the augmented Dickey-Fuller test. The asterisk *** denotes the 1% significance level.

Empirical results

Primary results

$R_{\mathrm{OOS}}^2$ and economic value analysis

Table 3 shows the out-of-sample forecasting outcomes, encompassing the $R_{\mathrm{OOS}}^2$ and economic value metrics. Our initial assessment focuses on individual predictors to ascertain whether energy indicators hold predictive capabilities regarding stock market volatility. In Panel A, the industrial non-renewable energy consumption (NOI) emerges with the most substantial and significant $R_{\mathrm{OOS}}^2$ value, suggesting a relatively consistent forecasting efficacy. This could be attributed to the industrial sector’s pivotal role in the economic growth of a highly industrialized nation like the United States, as noted by Haraguchi et al. (2017). Additionally, non-renewable energy, given its extensive application and high demand, serves as a principal driver of economic development, as highlighted by Salim et al. (2014). Consequently, industrial non-renewable energy consumption is a reliable proxy for real economic activity and influences stock market dynamics. When juxtaposed with the benchmark and other predictors, NOI exhibits the highest average expected utility, underscoring its enhanced economic value and implications for investment management. It is noteworthy that not all energy consumption indicators yield positive and significant $R_{\mathrm{OOS}}^2$ values, a finding that aligns with Ma et al. (2022), who observe the challenge for individual predictors in sustaining stability amidst market volatility.

Table 3. Out-of-sample $R^2$ results and economic value.

Model	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value	Average expected utility
Panel A: Benchmark and energy consumption indicators
Benchmark	-	-	-	3.218
MIDAS-RER	−0.026	1.179	0.119	3.232
MIDAS-REC	0.604**	1.836	0.033	3.236
MIDAS-REI	0.757*	1.416	0.078	3.235
MIDAS-NOR	−0.992	0.105	0.458	3.209
MIDAS-NOC	−0.883	0.306	0.380	3.211
MIDAS-NOI	2.785***	2.946	0.002	3.239
MIDAS-ELR	−0.756	0.208	0.418	3.202
MIDAS-ELC	0.357	1.060	0.145	3.204
MIDAS-ELI	0.880*	1.547	0.061	3.224
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	1.085**	1.792	0.037	3.228
MEDIAN	0.907*	1.558	0.060	3.230
TMC	1.052**	1.694	0.045	3.228
DMSPE-A	1.087**	1.795	0.036	3.228
DMSPE-B	1.110**	1.820	0.034	3.228
MIDAS-PCA	-0.441**	0.439	0.330	3.210
MIDAS-PLS	3.163***	3.071	0.001	3.230
MIDAS-SPCA	2.865***	3.025	0.001	3.240
MIDAS-LASSO	9.074***	5.073	0.000	3.292

The table displays $R_{\mathrm{OOS}}^2$ and average expected utility, with the largest values in bold. Asterisks denote significance levels: ***, **, and * at 1%, 5%, and 10%, respectively.

Secondly, we examine whether there is a method capable of capturing energy consumption information comprehensively and effectively. Panel B reveals that the ensemble models outperform both the univariate model and the techniques for dimensionality reduction concerning their forecasting effectiveness. Our findings validate the “combination forecast puzzle”, which is that combination forecasting usually performs better and more consistently than univariate models or more complicated techniques (Stock and Watson, 2004; Timmermann, 2006). Of particular significance, our findings indicate that the $R_{\mathrm{OOS}}^2$ values for the MIDAS-LASSO model substantially exceed those of the other models under consideration. This finding demonstrates how the MIDAS model, in conjunction with the LASSO method, completely captures the energy consumption information, including unexpected, transient, and sparse information, so as to enhance the prediction ability (Chinco et al., 2019). Furthermore, the MIDAS-LASSO model exhibits the highest expected economic utility, specifically 3.292, signifying its potential to offer the most substantial economic value to investors by accounting for both risk and return. To summarize, the MIDAS-LASSO model adeptly distills predictive signals from energy consumption data, thereby demonstrating its prowess in forecasting stock market volatility.

Direction-of-change analysis

Table 4 presents the DoC results. Evidently, all DoC ratios are significant and accurately forecast the direction at the 1% significance level. As shown in Panel A, compared to the other energy indicators, NOI has the highest DoC ratio, and has a stronger effect on direction forecasting. Panel B shows that MIDAS-LASSO outperforms other forecasting techniques with the highest DoC ratio. In general, our newly constructed energy indicators and forecasting methods are accurate in the direction of stock market volatility prediction, with good application values.

Table 4. Direction-of-change test.

Model	DoC ratio	PT statistic	p-value
Panel A: Benchmark and energy consumption indicators
Benchmark	0.648***	5.441	0.000
MIDAS-RER	0.652***	5.547	0.000
MIDAS-REC	0.639***	5.114	0.000
MIDAS-REI	0.661***	5.883	0.000
MIDAS-NOR	0.639***	5.105	0.000
MIDAS-NOC	0.642***	5.230	0.000
MIDAS-NOI	0.667***	6.079	0.000
MIDAS-ELR	0.642***	5.240	0.000
MIDAS-ELC	0.661***	5.874	0.000
MIDAS-ELI	0.652***	5.547	0.000
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	0.655***	5.672	0.000
MEDIAN	0.655***	5.662	0.000
TMC	0.655***	5.672	0.000
DMSPE-A	0.655***	5.672	0.000
DMSPE-B	0.655***	5.672	0.000
MIDAS-PCA	0.652***	5.539	0.000
MIDAS-PLS	0.655***	5.625	0.000
MIDAS-SPCA	0.658***	5.744	0.000
MIDAS-LASSO	0.667***	6.072	0.000

The table displays the direction-of-change test results, with the largest values in bold. The asterisk *** denotes the 1% significance level.

Forecast-encompassing analysis

Tables 5 and 6 present the p-values from the forecast-encompassing tests for the energy indicators and forecasting methods, respectively. Among the energy indicators, NOI exhibits the most significant p-value, suggesting that industrial non-renewable energy consumption encompasses additional predictive content for stock market volatility. The MIDAS-LASSO model’s pre-eminence is further substantiated by its capacity to encompass more information relative to other forecasting approaches. The results of the encompassing tests reveal that the MIDAS-LASSO model demonstrates superior performance in the domain of volatility forecasting.

Table 5. Forecast-encompassing check with energy consumption indicators.

Model	MIDAS-RER		MIDAS-REC		MIDAS-REI		MIDAS-NOR		MIDAS-NOC		MIDAS-NOI		MIDAS-ELR		MIDAS-ELC		MIDAS-ELI
	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$
Panel A: Benchmark and energy consumption indicators
Benchmark	0.120	0.164	0.034	0.207	0.079	0.420	0.458	0.058	0.380	0.062	0.002	0.366	0.418	0.108	0.145	0.304	0.062	0.377
MIDAS-RER	-	-	0.051	0.177	0.074	0.308	0.410	0.055	0.366	0.059	0.003	0.323	0.370	0.101	0.136	0.242	0.065	0.289
MIDAS-REC	0.177	0.051	-	-	0.091	0.111	0.447	0.013	0.387	0.017	0.004	0.163	0.410	0.023	0.165	0.070	0.091	0.137
MIDAS-REI	0.308	0.074	0.111	0.091	-	-	0.739	0.011	0.660	0.016	0.010	0.392	0.686	0.034	0.344	0.172	0.160	0.220
MIDAS-NOR	0.055	0.410	0.013	0.447	0.011	0.739	-	-	0.242	0.398	0.001	0.581	0.185	0.332	0.045	0.593	0.014	0.682
MIDAS-NOC	0.059	0.366	0.017	0.387	0.016	0.660	0.398	0.242	-	-	0.001	0.542	0.216	0.291	0.051	0.534	0.016	0.591
MIDAS-NOI	0.323	0.003	0.163	0.004	0.392	0.010	0.581	0.001	0.542	0.001	-	-	0.507	0.002	0.331	0.005	0.428	0.012
MIDAS-ELR	0.101	0.370	0.023	0.410	0.034	0.686	0.332	0.185	0.291	0.216	0.002	0.507	-	-	0.063	0.652	0.027	0.589
MIDAS-ELC	0.242	0.136	0.070	0.165	0.172	0.344	0.593	0.045	0.534	0.051	0.005	0.331	0.652	0.063	-	-	0.127	0.318
MIDAS-ELI	0.289	0.065	0.137	0.091	0.220	0.160	0.682	0.014	0.591	0.016	0.012	0.428	0.589	0.027	0.318	0.127	-	-
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	0.625	0.056	0.257	0.091	0.450	0.169	0.988	0.001	0.968	0.003	0.012	0.387	0.944	0.009	0.589	0.110	0.341	0.202
MEDIAN	0.549	0.074	0.201	0.115	0.345	0.221	0.969	0.005	0.932	0.008	0.009	0.345	0.918	0.019	0.519	0.163	0.249	0.229
TMC	0.599	0.063	0.234	0.096	0.441	0.176	0.990	0.001	0.967	0.003	0.012	0.348	0.952	0.009	0.593	0.129	0.323	0.206
DMSPE-A	0.625	0.056	0.258	0.091	0.451	0.168	0.988	0.001	0.967	0.003	0.012	0.390	0.943	0.009	0.588	0.109	0.342	0.202
DMSPE-B	0.631	0.054	0.264	0.087	0.463	0.162	0.988	0.001	0.967	0.002	0.013	0.389	0.945	0.008	0.598	0.104	0.352	0.195
MIDAS-PCA	0.110	0.290	0.018	0.319	0.055	0.595	0.581	0.111	0.500	0.146	0.002	0.482	0.483	0.215	0.073	0.579	0.042	0.515
MIDAS-PLS	0.277	0.002	0.207	0.003	0.357	0.004	0.559	0.001	0.508	0.001	0.216	0.073	0.458	0.001	0.293	0.004	0.528	0.006
MIDAS-SPCA	0.276	0.002	0.151	0.003	0.357	0.007	0.619	0.001	0.573	0.001	0.277	0.185	0.496	0.001	0.306	0.004	0.464	0.009
MIDAS-LASSO	0.822	0.000	0.886	0.000	0.789	0.000	0.914	0.000	0.912	0.000	0.854	0.000	0.888	0.000	0.804	0.000	0.826	0.000

The table reports p-values for the forecast-encompassing test with energy consumption indicators, where $\gamma$ tests the null of column forecasts encompassing row forecasts, and $1-\gamma$ tests the reverse. p-values under 0.1 are bolded.

Table 6. Forecast-encompassing check with various forecasting methods.

Model	Combination forecast										Dimensionality reduction method						LASSO approach
	MEAN		MEDIAN		TMC		DMSPE-A		DMSPE-B		MIDAS-PCA		MIDAS-PLS		MIDAS-SPCA		MIDAS-LASSO
	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$	$\gamma$	$1-\gamma$
Panel A: Benchmark and energy consumption indicators
Benchmark	0.037	0.633	0.060	0.621	0.046	0.659	0.037	0.632	0.035	0.638	0.331	0.129	0.001	0.299	0.001	0.320	0.000	0.685
MIDAS-RER	0.056	0.625	0.074	0.549	0.063	0.599	0.056	0.625	0.054	0.631	0.290	0.110	0.002	0.277	0.002	0.276	0.000	0.822
MIDAS-REC	0.091	0.257	0.115	0.201	0.096	0.234	0.091	0.258	0.087	0.264	0.319	0.018	0.003	0.207	0.003	0.151	0.000	0.886
MIDAS-REI	0.169	0.450	0.221	0.345	0.176	0.441	0.168	0.451	0.162	0.463	0.595	0.055	0.004	0.357	0.007	0.357	0.000	0.789
MIDAS-NOR	0.001	0.988	0.005	0.969	0.001	0.990	0.001	0.988	0.001	0.988	0.111	0.581	0.001	0.559	0.001	0.619	0.000	0.914
MIDAS-NOC	0.003	0.968	0.008	0.932	0.003	0.967	0.003	0.967	0.002	0.967	0.146	0.500	0.001	0.508	0.001	0.573	0.000	0.912
MIDAS-NOI	0.387	0.012	0.345	0.009	0.348	0.012	0.390	0.012	0.389	0.013	0.482	0.002	0.073	0.216	0.185	0.277	0.000	0.854
MIDAS-ELR	0.009	0.944	0.019	0.918	0.009	0.952	0.009	0.943	0.008	0.945	0.215	0.483	0.001	0.458	0.001	0.496	0.000	0.888
MIDAS-ELC	0.110	0.589	0.163	0.519	0.129	0.593	0.109	0.588	0.104	0.598	0.579	0.073	0.004	0.293	0.004	0.306	0.000	0.804
MIDAS-ELI	0.202	0.341	0.229	0.249	0.206	0.323	0.202	0.342	0.195	0.352	0.515	0.042	0.006	0.528	0.009	0.464	0.000	0.826
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	-	-	0.627	0.215	0.517	0.370	0.426	0.568	0.180	0.807	0.939	0.009	0.008	0.358	0.010	0.371	0.000	0.877
MEDIAN	0.215	0.627	-	-	0.208	0.704	0.214	0.624	0.196	0.643	0.874	0.026	0.005	0.308	0.007	0.317	0.000	0.812
TMC	0.370	0.517	0.704	0.208	-	-	0.367	0.517	0.324	0.559	0.938	0.012	0.007	0.326	0.010	0.332	0.000	0.828
DMSPE-A	0.568	0.426	0.624	0.214	0.517	0.367	-	-	0.153	0.837	0.938	0.009	0.008	0.360	0.010	0.374	0.000	0.878
DMSPE-B	0.807	0.180	0.643	0.196	0.559	0.324	0.837	0.153	-	-	0.941	0.009	0.008	0.358	0.010	0.372	0.000	0.878
MIDAS-PCA	0.009	0.939	0.026	0.874	0.012	0.938	0.009	0.938	0.009	0.941	-	-	0.002	0.439	0.002	0.494	0.000	0.851
MIDAS-PLS	0.358	0.008	0.308	0.005	0.326	0.007	0.360	0.008	0.358	0.008	0.439	0.002	-	-	0.311	0.135	0.000	0.811
MIDAS-SPCA	0.371	0.010	0.317	0.007	0.332	0.010	0.374	0.010	0.372	0.010	0.494	0.002	0.135	0.311	-	-	0.000	0.862
MIDAS-LASSO	0.877	0.000	0.812	0.000	0.828	0.000	0.878	0.000	0.878	0.000	0.851	0.000	0.811	0.000	0.862	0.000	-	-

The table reports p-values for the forecast-encompassing test with the combination forecasts, dimensionality reduction methods, and the LASSO approach, where γ tests the null of column forecasts encompassing row forecasts, and $1-\gamma$ tests the reverse. p-values under 0.1 are bolded.

Discussion of results

According to Table 3, the newly constructed energy consumption indicators have predictive effects on stock market volatility, especially NOI. Compared to other forecasting methods, MIDAS-LASSO comprehensively captures the information on energy consumption and enhances the forecasting performance. To illustrate the temporal evolution of forecasting accuracy, we employ the cumulative mean square error (hereafter, CMSE) metric, defined as:

25

$${\mathrm{CMSE}}_i=\sum _{t=O+1}^T\left[{\left({\widehat{\mathrm{RV}}}_t^j-{\mathrm{RV}}_t\right)}^2-{\left({\widehat{\mathrm{RV}}}_t^i-{\mathrm{RV}}_t\right)}^2\right].$$

This approach provides a straightforward assessment of the model’s predictive performance throughout the out-of-sample period. ${\widehat{\mathrm{RV}}}_t^j$ signifies the forecast value of the model under consideration, and ${\widehat{\mathrm{RV}}}_t^i$ represents that of the i-th comparative model, with T indicating the overall sample size. A negative ${\mathrm{CMSE}}_i$ denotes the model’s superior performance over its competitor.

Figures 2 and 3 show the CMSE plots for NOI and MIDAS-LASSO, respectively. Intuitively, the early forecasting performance of NOI appears to be erratic. The model’s forecasting efficacy experiences a significant uptick over time, being particularly pronounced throughout the periods of the global financial turmoil and the Crimean conflict. The COVID-19 pandemic briefly weakened predictive power, yet NOI’s forecasting stability generally persisted. Compared to individual predictors, the MIDAS-LASSO model shows superior forecasting performance. Even in a turbulent environment, MIDAS-LASSO’s performance is superior due to its ability to combine energy consumption information. Figure 4 illustrates the forecasting differences of various energy indicators. It is evident that the energy indicators have the ability to forecast stock market volatility. However, individual predictors typically cannot maintain stable long-term predictive performance, in line with Rapach et al. (2010).

Fig. 2 [Images not available. See PDF.]

CMSE performance of NOI.

Fig. 3 [Images not available. See PDF.]

CMSE performance of MIDAS-LASSO.

Fig. 4 [Images not available. See PDF.]

Variable selection of energy consumption.

In conclusion, the novel indicators can forecast stock market volatility, especially NOI. Furthermore, MIDAS-LASSO analyses all aspects of energy consumption, outperforming the univariate model and other forecasting techniques. For the practical implications of our findings, the newly constructed energy indicators not only focus on the residential sector but also incorporate the production and operation activities of the commercial and industrial sectors, providing an expanded viewpoint of the economic terrain. The empirical results highlight the significant value of energy usage metrics for predicting stock market volatility.

Robustness check

Alternative stock market benchmark

Inspired by Lu et al. (2022), who use alternative markets to forecast natural gas price volatility, we proceed to scrutinize the forecasting efficacy of energy consumption indicators alongside a spectrum of forecasting methodologies on the Dow Jones 65 Composite Average volatility. In comparison to the S&P 500, encompassing the 500 leading companies with the highest market value and liquidity, the Dow Jones 65 covers 65 large companies, including industrial stocks, transportation stocks, and high-tech stocks, among others, making it an alternative stock benchmark indicator.

Table 7 details the out-of-sample performance metrics for the Dow Jones Industrial Average. Specifically, in Panel A, the NOI exhibits the highest and most statistically significant $R_{\mathrm{OOS}}^2$ value, highlighting its superior predictive ability. Panel B shows that, for the Dow Jones 65, MIDAS-LASSO continues to perform better at forecasting volatility. However, the out-of-sample performance shown in Table 3 is marginally better than that shown in Table 7, indicating the superior application value of the newly developed energy indicators in S&P 500 volatility forecasting. In conclusion, the innovative energy indicators are trustworthy regarding Dow Jones volatility forecasting.

Table 7. Alternative stock market.

Model	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value
Panel A: Energy consumption indicators
MIDAS-RER	−0.251	1.068	0.143
MIDAS-REC	1.110**	2.245	0.012
MIDAS-REI	0.962*	1.604	0.054
MIDAS-NOR	−0.583	0.441	0.330
MIDAS-NOC	−0.512	0.563	0.287
MIDAS-NOI	2.380***	2.598	0.005
MIDAS-ELR	−0.629	0.326	0.372
MIDAS-ELC	0.240	1.034	0.151
MIDAS-ELI	0.448*	1.410	0.079
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	1.177**	1.771	0.038
MEDIAN	1.017*	1.640	0.050
TMC	1.214**	1.781	0.037
DMSPE-A	1.169**	1.764	0.039
DMSPE-B	1.182**	1.773	0.038
MIDAS-PCA	−0.247	0.617	0.268
MIDAS-PLS	4.200***	3.418	0.000
MIDAS-SPCA	3.176***	2.978	0.001
MIDAS-LASSO	8.912***	4.847	0.000

The table displays the $R_{\mathrm{OOS}}^2$ results of the Dow Jones index stock market, with the largest values in bold. Asterisks denote significance levels: ***, **, and * at 1%, 5%, and 10%, respectively.

Alternative lag order

Previous research indicates that diverse lag structures can be used to appraise the reliability of out-of-sample performance (Andersen et al., 2003; Li et al., 2022). Consequently, adhering to the approach of Paye (2012), we specify a lag order of six to assess the predictive efficacy. Table 8 shows the results. Notably, altering the lag order enhances the overall effectiveness of each predictor and forecasting technique. While most energy indicators have positive and significant $R_{\mathrm{OOS}}^2$ values, NOI consistently has the greatest $R_{\mathrm{OOS}}^2$ value, indicating its superior forecasting effectiveness. Panel B reveals that, although both ensemble models and dimensionality reduction techniques exhibit positive and significant $R_{\mathrm{OOS}}^2$ values, the MIDAS-LASSO model stands out in volatility forecasting. This is attributed to its precise information extraction capabilities, a result that corroborates the overall out-of-sample findings.

Table 8. Alternative lag order.

Model	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value
Panel A: Energy consumption indicators
MIDAS-RER	0.149*	1.523	0.064
MIDAS-REC	1.322**	2.274	0.011
MIDAS-REI	0.500*	1.435	0.076
MIDAS-NOR	0.639***	2.361	0.009
MIDAS-NOC	0.405***	2.334	0.010
MIDAS-NOI	2.062***	2.609	0.005
MIDAS-ELR	−0.225	0.891	0.186
MIDAS-ELC	−0.367	0.889	0.187
MIDAS-ELI	0.785**	1.768	0.038
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	1.823***	2.707	0.003
MEDIAN	1.319**	2.142	0.016
TMC	1.558***	2.367	0.009
DMSPE-A	1.837***	2.723	0.003
DMSPE-B	1.852***	2.744	0.003
MIDAS-PCA	0.083*	1.340	0.090
MIDAS-PLS	3.072***	3.056	0.001
MIDAS-SPCA	2.192***	3.075	0.001
MIDAS-LASSO	10.389***	5.680	0.000

The table displays $R_{\mathrm{OOS}}^2$ results with an alternative lag order equal to 6, with the largest values in bold. Asterisks denote significance levels: ***, **, and * at 1%, 5%, and 10%, respectively.

Alternative forecasting window

In the domain of volatility forecasting, robustness analysis serves as a pivotal method for assessing the reliability of a model’s predictive capabilities. This approach involves altering certain parameters or conditions of the model to observe the stability and consistency of its predictive outcomes. In our study, the original in-sample window is from April 1980 to March 1994, encompassing a total of 166 months of data. This window is used for the model’s estimation phase, the in-sample estimation stage. Subsequently, the out-of-sample period is delineated as extending from April 1994 to December 2021, comprising 331 observations, employed to evaluate the model’s forecasting prowess. Rossi and Inoue (2012) show that different sample periods can lead to divergent predictive performances. Motivated by these findings, we ascertain the robustness of our model’s predictive capabilities by expanding the in-sample window from 166 to 249 months, effectively elongating the duration of the in-sample estimation phase. In tandem, the out-of-sample period is redefined from March 2001 to December 2021. This adjustment ensures seamless chronological alignment between the in-sample and out-of-sample datasets, precluding any data overlap. The primary objectives of conducting this robustness analysis in this paper are as follows: firstly, to examine the impact of the time span; a longer in-sample window may contain more information, helping the model better learn patterns from historical data; secondly, to assess the model’s generalization capability; changing the out-of-sample period can test whether the model can adapt to new market conditions, thereby evaluating its generalization ability; and lastly, to verify the consistency of the results; if the model’s predictive outcomes remain consistent across sample windows, this enhances the reliability of the model’s predictive power.

Table 9 presents the forecasting outcomes using an alternative rolling window specification. Firstly, even though the majority of energy consumption indicators have positive $R_{\mathrm{OOS}}^2$ values, NOI has the highest and most significant $R_{\mathrm{OOS}}^2$ value. This result indicates that non-renewable energy consumption from the industrial sector has the largest effect on stock market volatility forecasting, despite changing the window size. Secondly, compared to other forecasting techniques, the MIDAS-LASSO more effectively captures the information and consistently performs well, which is consistent with our main results. Accordingly, the newly constructed energy consumption indicators and MIDAS-LASSO model effectively for stock market volatility forecasting.

Table 9. Alternative forecasting window.

Model	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value
Panel A: Energy consumption indicators
MIDAS-RER	0.139	0.956	0.170
MIDAS-REC	0.633	1.281	0.100
MIDAS-REI	0.954*	1.568	0.058
MIDAS-NOR	0.327	0.919	0.179
MIDAS-NOC	0.360	1.087	0.138
MIDAS-NOI	2.932***	2.505	0.006
MIDAS-ELR	0.151	0.639	0.261
MIDAS-ELC	0.948	1.268	0.102
MIDAS-ELI	0.713	1.170	0.121
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	1.234**	1.848	0.032
MEDIAN	0.761*	1.335	0.091
TMC	1.071**	1.651	0.049
DMSPE-A	1.238**	1.853	0.032
DMSPE-B	1.254**	1.871	0.031
MIDAS-PCA	0.414	0.950	0.171
MIDAS-PLS	1.863**	1.869	0.031
MIDAS-SPCA	2.725***	2.373	0.009
MIDAS-LASSO	7.659***	3.639	0.000

The table displays the $R_{\mathrm{OOS}}^2$ results with an alternative forecasting window of 248 months, with the largest values in bold. Asterisks denote significance levels: ***, **, and * at 1%, 5%, and 10%, respectively.

Further analysis

Discussion of energy consumption

Various energy-consuming sectors

In the preceding analysis, we establish that the novel energy consumption indicators, particularly those pertaining to non-renewable energy from the industrial sector, exert a significant impact on the forecasting of stock market volatility. In this segment, we assess the predictive accuracy of total energy consumption from the perspective of three major energy-consuming sectors: industrial (IND), commercial (COM), and residential (RES). Table 10 shows the results. Panel A delineates the out-of-sample performance across the three energy-consuming sectors, with IND demonstrating the most efficacious forecasting capabilities. This outcome aligns with the United States’ status as a nation with a highly developed industrial base. As the primary driver of economic development, industrial energy consumption reflects actual economic activity, influencing stock market activity. Panel B demonstrates that MIDAS-LASSO provides the best prediction by fully aggregating information from the indicators.

Table 10. Various energy-consuming sectors.

Model	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value
Panel A: Energy-consuming sector
MIDAS-RES	−0.962	0.108	0.457
MIDAS-COM	−0.776	0.103	0.459
MIDAS-IND	2.658***	2.815	0.002
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	1.042**	1.708	0.044
MEDIAN	−0.125	0.603	0.273
TMC	−0.125	0.603	0.273
DMSPE-A	1.056**	1.722	0.043
DMSPE-B	1.096**	1.757	0.039
MIDAS-PCA	−0.135	0.769	0.221
MIDAS-PLS	2.251***	2.587	0.005
MIDAS-SPCA	2.474***	2.782	0.003
MIDAS-LASSO	6.413***	4.185	0.000

The table displays the $R_{\mathrm{OOS}}^2$ results from various energy-consuming sectors, with the largest values in bold. Asterisks denote significance levels: *** and ** at 1% and 5%, respectively.

To further illustrate the influence of each energy-consuming sector on stock market volatility, we design a variable selection diagram using the LASSO technique and present the results in Fig. 5. It is evident that the impact of energy consumption on stock market volatility varies across distinct temporal intervals. From the global financial crisis outbreak, IND has shown a steadily bigger and stronger detrimental influence. Except for a few rare instances, energy consumption from the three sectors consistently influences stock market volatility. The growth theory, which holds that energy consumption enables economic growth (Apergis and Tang, 2013), and thus influences stock market activity, is also supported by our results.

Fig. 5 [Images not available. See PDF.]

Variable selection of total energy consumption from various consuming sectors.

Subdivision of non-renewable energy

Our principal investigation reveals that, among the recently formulated energy consumption indicators, non-renewable energy consumption by the industrial sector persistently exhibits enhanced predictive strength regarding stock market volatility. Given its high energy density and widespread utilization, non-renewable energy constitutes a pivotal factor in economic expansion, as noted by Salim et al. (2014). Consequently, we categorize non-renewable energy sources into coal, petroleum, and natural gas to delve deeper into their individual predictive impacts. The subdivided non-renewable energy statistics are updated to December 2007, and the information represented by each indicator is presented in Table 11. Table 12 summarizes the predictive results.

Table 11. Variable description of subdivided non-renewable energy consumption.

Category		Variable	Detailed information
Subdivided non-renewable energy	Coal	CLR	Calculated as the change rate of residential coal consumption.
		CLC	Calculated as the change rate of commercial coal consumption.
		CLI	Calculated as the change rate of industrial coal consumption.
	Petroleum	PMR	Calculated as the change rate of residential petroleum consumption.
		PMC	Calculated as the change rate of commercial petroleum consumption.
		PMI	Calculated as the change rate of industrial petroleum consumption.
	Natural gas	NGR	Calculated as the change rate of residential natural gas consumption.
		NGC	Calculated as the change rate of commercial natural gas consumption.
		NGI	Calculated as the change rate of industrial natural gas consumption.
Total energy consumption from various energy-consuming sectors		RES	Calculated as the change rate of residential energy consumption.
		COM	Calculated as the change rate of commercial energy consumption.
		IND	Calculated as the change rate of industrial energy consumption.

This table provides details of the supplementary variables.

Table 12. Subdivided non-renewable energy consumption.

Model	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value
Panel A: Subdivided non-renewable energy consumption
MIDAS-CLR	−1.108	0.995	0.160
MIDAS-CLC	−1.102	0.703	0.241
MIDAS-CLI	1.212**	1.897	0.029
MIDAS-PMR	−0.432	0.695	0.244
MIDAS-PMC	0.455	1.123	0.131
MIDAS-PMI	−0.208	0.754	0.226
MIDAS-NGR	−1.381	0.272	0.393
MIDAS-NGC	−0.330	0.774	0.220
MIDAS-NGI	1.433**	1.664	0.048
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	1.168*	1.282	0.100
MEDIAN	0.944	1.146	0.126
TMC	0.999	1.181	0.119
DMSPE-A	1.151	1.271	0.102
DMSPE-B	1.138	1.266	0.103
MIDAS-PCA	−0.541	0.912	0.181
MIDAS-PLS	3.232***	2.373	0.009
MIDAS-SPCA	2.119**	1.986	0.023
MIDAS-LASSO	13.358***	5.015	0.000

The table displays the $R_{\mathrm{OOS}}^2$ results with subdivided non-renewable energy consumption, with the largest values in bold. Asterisks denote significance levels: ***, **, and * at 1%, 5%, and 10%, respectively.

Panel A indicates that the industrial sector’s coal and natural gas consumption patterns, denoted as CLI and NGI, respectively, yield positive and statistically significant $R_{\mathrm{OOS}}^2$ values. There are two plausible explanations for this intriguing finding. Firstly, as a well-developed industrial country, the United States benefits from the world’s richest coal reserves. Due to the characteristics of high density, low cost, and easy combustion, coal is the primary energy source used. Secondly, natural gas, a clean and environmentally friendly energy, has recently gained popularity (Obama, 2017). As a key economic driver, the industrial sector significantly impacts economic growth and, by extension, stock market volatility, according to Haraguchi et al. (2017). The forecasting performance is enhanced by MIDAS-LASSO’s powerful information capture mechanism, although the subdivided non-renewable energy indicators’ predictive impacts are generally weaker than those shown in Table 3.

We examine the long-term impact of each non-renewable energy indicator on stock market volatility. Figure 6 presents the variable selection diagram. It is logical that various non-renewable energy indicators exert distinct influences on stock market volatility over time. Beyond the case of CLI, these energy indicators typically bear a negative relationship with stock market volatility. One probable explanation is that, being the most frequently used energy source, increasing coal consumption indicates economic growth and hence boosts stock market activity.

Fig. 6 [Images not available. See PDF.]

Variable selection of non-renewable energy consumption.

Business cycle

Paye (2012) establishes a robust relationship between business cycles and stock market volatility. Consequently, this study assesses the performance of energy indicators across economic recessions and expansions. The findings are presented in Table 13. Overall, both the energy indicators and forecasting methods perform better in recession periods. Panel A shows that NOI continuously outperforms all other energy indicators, with the greatest $R_{\mathrm{OOS}}^2$ value of 11.815% during the recession. Furthermore, during the recession, all forecasting techniques exhibit positive $R_{\mathrm{OOS}}^2$ values, which is better than the performance during expansions. In particular, due to its ability to capture dynamic information, MIDAS-LASSO achieves the best performance. There are several reliable reasons for these results. Firstly, due to the fact that even slight changes in economic activity can generate significant variations during the recession, market volatility tends to be more sensitive. Secondly, during recessions, market participants seek higher risk premiums while avoiding higher market risks, resulting in more dramatic volatility. Finally, since we generated 49 recessions and 282 expansions, the different sample lengths contribute to the performance variation.

Table 13. Out-of-sample performance during the business cycle.

Model	Recession			Expansion
	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value
Panel A: Energy consumption indicators
MIDAS−RER	0.274	0.443	0.329	−0.088	1.100	0.136
MIDAS-REC	1.539	1.052	0.146	0.409*	1.524	0.064
MIDAS-REI	1.513	0.877	0.190	0.600	1.118	0.132
MIDAS-NOR	−2.105	−0.765	0.778	−0.761	0.345	0.365
MIDAS-NOC	−3.515	−1.149	0.875	−0.337	0.717	0.237
MIDAS-NOI	11.815***	2.387	0.008	0.910**	1.847	0.032
MIDAS-ELR	−2.193	−1.005	0.843	−0.458	0.442	0.329
MIDAS-ELC	3.513	1.129	0.129	−0.298	0.536	0.296
MIDAS-ELI	1.507	0.783	0.217	0.749*	1.348	0.089
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	2.770**	1.673	0.047	0.735*	1.298	0.097
MEDIAN	0.192	0.300	0.382	1.055*	1.548	0.061
TMC	1.741	1.120	0.131	0.909*	1.395	0.081
DMSPE-A	2.813**	1.686	0.046	0.728*	1.294	0.098
DMSPE-B	2.901**	1.725	0.042	0.738*	1.302	0.097
MIDAS-PCA	0.361	0.407	0.342	−0.608	0.324	0.373
MIDAS-PLS	9.258**	1.893	0.029	1.898***	2.490	0.006
MIDAS-SPCA	13.431***	2.509	0.006	0.670**	1.842	0.033
MIDAS-LASSO	22.725***	2.891	0.002	6.240***	4.293	0.000

The table displays the $R_{\mathrm{OOS}}^2$ performance during the business cycle, with the largest values in bold. Asterisks denote significance levels: ***, **, and * at 1%, 5%, and 10%, respectively.

Multiple crisis periods

In this section, we examine whether innovative energy indicators and forecasting methods provide credible predictions in a volatile economic environment. The California power crisis broke out in June 2000, as the cost of producing energy increased and the disparity between supply and demand grew. The crisis affected not only the daily electricity consumption of residents but also industrial production and thus influenced economic activity. In addition, after the outbreak of the global financial crisis and COVID-19 pandemic came great macroeconomic shocks to the global economy, with financial markets severely challenged (Kuppuswamy and Villalonga, 2016; Pan et al., 2021). Accordingly, we assess the model’s performance specifically during the California power crisis, global financial crisis, and COVID-19 pandemic. Table 14 shows the results. Our findings indicate that the recently developed energy indicators produce favorable results, persisting even throughout periods of crisis. Specifically, NOI has a higher $R_{\mathrm{OOS}}^2$ value than that shown in Table 3 during crisis periods. MIDAS-LASSO also outperforms other forecasting models during crisis periods. In conclusion, the novel variables and models have excellent applicability during crises.

Table 14. Out-of-sample performance during crisis.

Model	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value
Panel A: Energy consumption indicators
MIDAS-RER	−2.993	0.138	0.445
MIDAS-REC	1.185	1.161	0.123
MIDAS-REI	0.681	0.838	0.201
MIDAS-NOR	−1.962	−0.331	0.630
MIDAS-NOC	−1.935	−0.184	0.573
MIDAS-NOI	6.256***	2.410	0.008
MIDAS-ELR	−1.925	−0.426	0.665
MIDAS-ELC	2.275	1.257	0.104
MIDAS-ELI	0.340	0.797	0.213
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	1.516	1.272	0.102
MEDIAN	−0.055	0.397	0.346
TMC	1.095	0.978	0.164
DMSPE-A	1.540*	1.287	0.099
DMSPE-B	1.635*	1.337	0.091
MIDAS-PCA	−0.592	0.230	0.409
MIDAS-PLS	4.627**	2.080	0.019
MIDAS-SPCA	6.863***	2.528	0.006
MIDAS-LASSO	15.716***	3.410	0.000

The table displays the $R_{\mathrm{OOS}}^2$ performance during the California power crisis, global financial crisis, and COVID-19 pandemic, with the largest values in bold. Asterisks denote significance levels: ***, **, and * at 1%, 5%, and 10%, respectively.

Investor sentiment variation

Baker and Wurgler (2006) discern that investor sentiment plays a role in shaping the cross-sectional variation in stock returns. In light of this, we investigate the potential effects of diverse investor sentiment states on the predictive modeling of stock market volatility. We use the composite sentiment index (SENT), which is derived by extracting the principal components from six constituent sentiment indicators4. We divide the SENT index into positive and negative sentiment periods as follows:

26

$$\left\{\begin{array}{c}\hfill \mathrm{positive},\mathrm{if}{\mathrm{SENT}}_t>0\hfill \\ \hfill \mathrm{negative},\mathrm{if}{\mathrm{SENT}}_t<0\hfill \end{array}\right),$$where ${\mathrm{SENT}}_t$ signifies the investor sentiment index for the t-th month.

Table 15 shows the results across diverse investor sentiment conditions. Generally, the newly developed energy indicators and forecasting approaches perform better during negative sentiment periods. It merits emphasis that, among the array of energy indicators, NOI alone maintains a consistently superior forecasting performance, uninfluenced by shifts in investor sentiment. MIDAS-LASSO continues to perform best among the several prediction techniques employed. This finding indicates MIDAS-LASSO’s ability to capture information accurately and support forecast stock market volatility when investor sentiment varies. The possible reasons for the better predictive performance of negative investor sentiment are as follows. The sentiment index, as formulated by Baker and Wurgler (2006), is correlated with macroeconomic conditions and consistent with the historical record of bubbles and crashes. In light of this, it is reasonable to expect that a weak economic environment will show when investor sentiment is negative. During economic downturns, financial markets experience significant volatility, prompting investors to focus on elevated risk premiums as a means to mitigate potential market risks. Consequently, our findings offer robust evidence for forecasting stock market volatility across varying investor sentiment environments.

Table 15. Out-of-sample performance with investor sentiment.

Model	Positive			Negative
	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value	$R_{\mathrm{OOS}}^2\left(\%\right)$	MSPE-adj.	p-value
Panel A: Energy consumption indicators
MIDAS-RER	−0.145	0.405	0.343	0.027	1.108	0.134
MIDAS-REC	−2.569	0.310	0.378	2.017**	2.069	0.019
MIDAS-REI	−1.054	0.123	0.451	1.564*	1.556	0.060
MIDAS-NOR	−3.282	−0.871	0.808	0.028	0.720	0.236
MIDAS-NOC	−3.197	−0.801	0.789	0.148	0.858	0.195
MIDAS-NOI	3.476***	2.451	0.007	2.477**	2.179	0.015
MIDAS-ELR	−2.398	−0.807	0.790	−0.025	0.594	0.276
MIDAS-ELC	0.789	0.878	0.190	0.165	0.767	0.222
MIDAS-ELI	3.854***	2.627	0.004	−0.446	0.489	0.312
Panel B: Combination forecasts, dimensionality reduction methods, and the LASSO approach
MEAN	0.508	0.766	0.222	1.342*	1.620	0.053
MEDIAN	0.812	1.030	0.151	0.949	1.266	0.103
TMC	0.715	0.919	0.179	1.202*	1.452	0.073
DMSPE-A	0.500	0.761	0.223	1.348*	1.626	0.052
DMSPE-B	0.512	0.775	0.219	1.377**	1.647	0.050
MIDAS-PCA	−1.040	−0.024	0.510	−0.174	0.530	0.298
MIDAS-PLS	5.723***	2.847	0.002	2.023**	2.022	0.022
MIDAS-SPCA	4.622***	2.763	0.003	2.081**	2.082	0.019
MIDAS-LASSO	6.257***	3.019	0.001	10.330***	4.142	0.000

The table displays the $R_{\mathrm{OOS}}^2$ performance during periods of positive and negative investor sentiment, with the largest values in bold. Asterisks denote significance levels: ***, **, and * at 1%, 5%, and 10%, respectively.

Conclusion

This study innovatively uses energy consumption as a novel indicator for forecasting stock market volatility, combining the MIDAS framework with machine learning techniques, thereby making significant theoretical and practical contributions to the field of financial forecasting. Our research not only confirms the significant correlation between energy consumption and stock market volatility but also demonstrates the superior predictive accuracy and robustness of MIDAS-LASSO. In comparison with the third-order AR model, which serves as a benchmark, and other MIDAS-based models such as MEAN, MEDIAN, TMC, DMSPE-A, DMSPE-B, MIDAS-PCA, MIDAS-PLS, and MIDAS-SPCA, the MIDAS-LASSO model demonstrates a notable enhancement in $R_{\mathrm{OOS}}^2$ values. Additionally, it exhibits exceptional performance with respect to MSPE-adj. and expected economic utility. Moreover, our analysis reveals that the non-renewable energy consumption indicator within the industrial sector consistently demonstrates enhanced predictive performance regarding stock market volatility. These findings provide important reference value for participants in the financial markets and offer new perspectives and methods for future research.

In order to probe the predictive efficacy of energy consumption on stock market volatility, we conduct an in-depth assessment of the forecasting prowess of non-renewable energy across diverse sectors and categories. We use the LASSO technique to facilitate a clearer interpretation of the indicators’ influence. Additionally, we investigate the forecasting performance under business cycles and various crisis conditions, showing that the new energy consumption indicators maintain stable predictive power even during economic recessions or periods of economic turmoil. We also examine the model performance during shifts in investor sentiment, discovering that the predictive factors and models have greater application value during periods of negative expectations. Our innovative approach, starting from the perspective of energy consumption and integrating machine learning methods, provides new evidence for stock market volatility prediction. It provides novel insights into the nexus between energy consumption and stock market volatility while contributing to the literature by enhancing our understanding of the predictive role of macroeconomic indicators. Our research underscores the potential of changes in energy consumption in financial forecasting and confirms the effectiveness of machine learning techniques in enhancing predictive accuracy.

From a theoretical standpoint, our study enhances the theoretical underpinnings for forecasting financial market volatility, specifically within the scholarly discourse that examines the influence of macroeconomic variables on financial markets. By incorporating energy consumption as a predictive variable, we offer a novel perspective for understanding market fluctuations and provide a new theoretical foundation for future research. From a managerial standpoint, our findings provide risk managers with new tools to better understand and forecast market volatility. The enhanced performance of the MIDAS-LASSO model provides substantial validation for investment decision-making and risk management strategies.

For policymakers, our findings highlight the interconnection between energy consumption strategies and the stability of financial markets. Policymakers can influence stock market volatility by adjusting energy consumption policies, thereby promoting financial stability. Our study specifically underscores the pivotal role of non-renewable energy consumption in the industrial sector, offering tailored insights for the development of policy directives. Moreover, the research indicates that encouraging the industrial sector to use more efficient energy sources or transition to renewable energy not only promotes environmental protection but may also exert a positive influence on stock market volatility. This provides a basis for designing relevant incentive measures.

Future research could further explore the predictive capabilities of different energy types and industry indices, especially in the context of current energy transitions and sustainable development.

Acknowledgements

This study was funded by the Service Science and Innovation Key Laboratory of Sichuan Province, Chengdu, China.

Author contributions

Fei Lu: conceptualization, methodology, software, investigation, formal analysis, writing—original draft; Feng Ma: conceptualization, funding acquisition, resources, supervision, writing—original draft; Elie Bouri: writing—original draft; writing—review & editing.

Data availability

The datasets generated for the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare competing interests.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent

This article does not contain any studies with human participants performed by any of the authors.

1

An additional explanation of dimensionality reduction methods can be found in Huang et al. (2022).

2

https://www.eia.gov/petroleum/data.php

3

In Section 7.1, we extend our analysis to encompass a supplementary examination of energy consumption indicators. This includes the aggregation of total energy consumption indicators across various sectors and the stratification of non-renewable energy consumption indicators.

4

The statistics are collected from Jeffrey Wurgler’s website (https://pages.stern.nyu.edu/~jwurgler/). Details of the underlying sentiment indicators are detailed in Baker and Wurgler (2006).

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1057/s41599-024-04130-x.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

Alaminos, D; Salas, MB; Fernández-Gámez, MÁ. Quantum Monte Carlo simulations for estimating FOREX markets: a speculative attacks experience. Humanit Soc Sci Commun; 2023; 10, 1 pp. 1-21. [DOI: https://dx.doi.org/10.1057/s41599-023-01836-2]

Andersen, TG; Bollerslev, T; Diebold, FX; Labys, P. Modeling and forecasting realized volatility. Econometrica; 2003; 71, 2 pp. 579-625.MathSciNet ID: 1958138[DOI: https://dx.doi.org/10.1111/1468-0262.00418]

Apergis, N; Tang, CF. Is the energy-led growth hypothesis valid? New evidence from a sample of 85 countries. Energy Econ; 2013; 38, pp. 24-31. [DOI: https://dx.doi.org/10.1016/j.eneco.2013.02.007]

Bakas, D; Triantafyllou, A. Volatility forecasting in commodity markets using macro uncertainty. Energy Econ; 2019; 81, pp. 79-94. [DOI: https://dx.doi.org/10.1016/j.eneco.2019.03.016]

Baker, M; Wurgler, J. Investor sentiment and the cross‐section of stock returns. J Financ; 2006; 61, 4 pp. 1645-1680. [DOI: https://dx.doi.org/10.1111/j.1540-6261.2006.00885.x]

Behrendt, S; Schmidt, A. The Twitter myth revisited: Intraday investor sentiment, Twitter activity and individual-level stock return volatility. J Bank Financ; 2018; 96, pp. 355-367. [DOI: https://dx.doi.org/10.1016/j.jbankfin.2018.09.016]

Bekaert, G; Harvey, CR. Time‐varying world market integration. J Financ; 1995; 50, 2 pp. 403-444.

Bekaert, G; Hoerova, M. The VIX, the variance premium and stock market volatility. J Econ; 2014; 183, 2 pp. 181-192.MathSciNet ID: 3276387[DOI: https://dx.doi.org/10.1016/j.jeconom.2014.05.008]

Bollerslev, T; Todorov, V. Tails, fears, and risk premia. J Financ; 2011; 66, 6 pp. 2165-2211. [DOI: https://dx.doi.org/10.1111/j.1540-6261.2011.01695.x]

Bollerslev, T; Hood, B; Huss, J; Pedersen, LH. Risk everywhere: Modeling and managing volatility. Rev Financ Stud; 2018; 31, 7 pp. 2729-2773. [DOI: https://dx.doi.org/10.1093/rfs/hhy041]

Campbell, JY; Thompson, SB. Predicting excess stock returns out of sample: Can anything beat the historical average?. Rev Financ Stud; 2008; 21, 4 pp. 1509-1531. [DOI: https://dx.doi.org/10.1093/rfs/hhm055]

Cheung, YW; Ng, LK. International evidence on the stock market and aggregate economic activity. J Empir Financ; 1998; 5, 3 pp. 281-296. [DOI: https://dx.doi.org/10.1016/S0927-5398(97)00025-X]

Chinco, A; Clark‐Joseph, AD; Ye, M. Sparse signals in the cross‐section of returns. J Financ; 2019; 74, 1 pp. 449-492. [DOI: https://dx.doi.org/10.1111/jofi.12733]

Clark, TE; West, KD. Approximately normal tests for equal predictive accuracy in nested models. J Econ; 2007; 138, 1 pp. 291-311.MathSciNet ID: 2380700[DOI: https://dx.doi.org/10.1016/j.jeconom.2006.05.023]

Engle, RF; Ng, VK. Measuring and testing the impact of news on volatility. J Financ; 1993; 48, 5 pp. 1749-1778. [DOI: https://dx.doi.org/10.1111/j.1540-6261.1993.tb05127.x]

Engle RF, Patton AJ (2007) What good is a volatility model? Forecasting volatility in the financial markets. Butterworth-Heinemann. pp. 47–63

Engle, RF; Ghysels, E; Sohn, B. Stock market volatility and macroeconomic fundamentals. Rev Econ Stat; 2013; 95, 3 pp. 776-797. [DOI: https://dx.doi.org/10.1162/REST_a_00300]

Fama, EF. Efficient capital markets. J Financ; 1970; 25, 2 pp. 383-417. [DOI: https://dx.doi.org/10.2307/2325486]

Fan, LW; Pan, SJ; Liu, GQ; Zhou, P. Does energy efficiency affect financial performance? Evidence from Chinese energy-intensive firms. J Clean Prod; 2017; 151, pp. 53-59. [DOI: https://dx.doi.org/10.1016/j.jclepro.2017.03.044]

Haraguchi, N; Cheng, CFC; Smeets, E. The importance of manufacturing in economic development: has this changed?. World Dev; 2017; 93, pp. 293-315. [DOI: https://dx.doi.org/10.1016/j.worlddev.2016.12.013]

Harvey, DI; Leybourne, SJ; Newbold, P. Tests for forecast encompassing. J Bus Econ Stat; 1998; 16, 2 pp. 254-259. [DOI: https://dx.doi.org/10.1080/07350015.1998.10524759]

He, M; Zhang, Y; Wang, Y; Wen, D. Modelling and forecasting crude oil price volatility with climate policy uncertainty. Humanit Soc Sci Commun; 2024; 11, 1 pp. 1-10. [DOI: https://dx.doi.org/10.1057/s41599-024-03561-w]

Huang, D; Jiang, F; Li, K; Tong, G; Zhou, G. Scaled PCA: A new approach to dimension reduction. Manag Sci; 2022; 68, 3 pp. 1678-1695. [DOI: https://dx.doi.org/10.1287/mnsc.2021.4020]

Jarrett, U; Mohaddes, K; Mohtadi, H. Oil price volatility, financial institutions and economic growth. Energy Policy; 2019; 126, pp. 131-144. [DOI: https://dx.doi.org/10.1016/j.enpol.2018.10.068]

Kahneman D, Tversky A (2013) Prospect theory: An analysis of decision under risk. Handbook of the Fundamentals of Financial Decision Making: Part I, pp 99–127

Kilian, L. The economic effects of energy price shocks. J Econ Lit; 2008; 46, 4 pp. 871-909. [DOI: https://dx.doi.org/10.1257/jel.46.4.871]

Kuppuswamy, V; Villalonga, B. Does diversification create value in the presence of external financing constraints? Evidence from the 2007-2009 financial crisis. Manag Sci; 2016; 62, 4 pp. 905-923. [DOI: https://dx.doi.org/10.1287/mnsc.2015.2165]

Li, X; Liang, C; Chen, Z; Umar, M. Forecasting crude oil volatility with uncertainty indicators: New evidence. Energy Econ; 2022; 108, [DOI: https://dx.doi.org/10.1016/j.eneco.2022.105936]

Liang, C; Umar, M; Ma, F; Huynh, TL. Climate policy uncertainty and world renewable energy index volatility forecasting. Technol Forecast Soc Change; 2022; 182, [DOI: https://dx.doi.org/10.1016/j.techfore.2022.121810]

Liu, L; Geng, Q; Zhang, Y; Wang, Y. Investors’ perspective on forecasting crude oil return volatility: Where do we stand today?. J Manag Sci Eng; 2022; 7, 3 pp. 423-438.[COI: 1:CAS:528:DC%2BB3sXjsVKqt7w%3D]

Lu F, Ma F, Li P, Huang D (2022) Natural gas volatility predictability in a data-rich world. Int Rev Financ Anal 102218. https://doi.org/10.1016/j.irfa.2022.102218

Lu, X; Ma, F; Wang, J; Wang, J. Examining the predictive information of CBOE OVX on China’s oil futures volatility: Evidence from MS-MIDAS models. Energy; 2020; 212,[COI: 1:CAS:528:DC%2BB3cXhslynt7nI] [DOI: https://dx.doi.org/10.1016/j.energy.2020.118743] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32904908]

Ma, F; Liu, J; Wahab, MIM; Zhang, Y. Forecasting the aggregate oil price volatility in a data-rich environment. Econ Model; 2018; 72, pp. 320-332. [DOI: https://dx.doi.org/10.1016/j.econmod.2018.02.009]

Ma, F; Lu, F; Tao, Y. Geopolitical risk and excess stock returns predictability: New evidence from a century of data. Financ Res Lett; 2022; 50, [DOI: https://dx.doi.org/10.1016/j.frl.2022.103211]

Marsilli C (2014) Variable selection in predictive MIDAS models (November). Banque de France Working Paper No. 520, Available at SSRN: https://ssrn.com/abstract=2531339

Masini, RP; Medeiros, MC; Mendes, EF. Machine learning advances for time series forecasting. J Econ Surv; 2023; 37, 1 pp. 76-111. [DOI: https://dx.doi.org/10.1111/joes.12429]

Neely, CJ; Rapach, DE; Tu, J; Zhou, G. Forecasting the equity risk premium: the role of technical indicators. Manag Sci; 2014; 60, 7 pp. 1772-1791. [DOI: https://dx.doi.org/10.1287/mnsc.2013.1838]

Obama, B. The irreversible momentum of clean energy. Science; 2017; 355, 6321 pp. 126-129.2017Sci..355.126O[COI: 1:CAS:528:DC%2BC2sXhsFyku78%3D] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28069665][DOI: https://dx.doi.org/10.1126/science.aam6284]

Oberndorfer, U. Energy prices, volatility, and the stock market: Evidence from the Eurozone. Energy Policy; 2009; 37, 12 pp. 5787-5795. [DOI: https://dx.doi.org/10.1016/j.enpol.2009.08.043]

Pan, WF; Wang, X; Wu, G; Xu, W. The COVID-19 pandemic and sovereign credit risk. China Financ Rev Int; 2021; 11, 3 pp. 287-301. [DOI: https://dx.doi.org/10.1108/CFRI-01-2021-0010]

Paye, BS. Déjà vol’: Predictive regressions for aggregate stock market volatility using macroeconomic variables. J Financ Econ; 2012; 106, 3 pp. 527-546. [DOI: https://dx.doi.org/10.1016/j.jfineco.2012.06.005]

Pesaran, MH; Timmermann, A. Testing dependence among serially correlated multicategory variables. J Am Stat Assoc; 2009; 104, 485 pp. 325-337.MathSciNet ID: 2663043[COI: 1:CAS:528:DC%2BD1MXktFKrtLo%3D] [DOI: https://dx.doi.org/10.1198/jasa.2009.0113]

Pirlogea, C; Cicea, C. Econometric perspective of the energy consumption and economic growth relation in European Union. Renew Sustain Energy Rev; 2012; 16, 8 pp. 5718-5726. [DOI: https://dx.doi.org/10.1016/j.rser.2012.06.010]

Rapach, DE; Strauss, JK; Zhou, G. Out-of-sample equity premium prediction: Combination forecasts and links to the real economy. Rev Financ Stud; 2010; 23, 2 pp. 821-862. [DOI: https://dx.doi.org/10.1093/rfs/hhp063]

Ren, X; Li, J; He, F; Lucey, B. Impact of climate policy uncertainty on traditional energy and green markets: Evidence from time-varying granger tests. Renew Sustain Energy Rev; 2023; 173, 113058. [DOI: https://dx.doi.org/10.1016/j.rser.2022.113058]

Rossi, B; Inoue, A. Out-of-sample forecast tests robust to the choice of window size. J Bus Econ Stat; 2012; 30, 3 pp. 432-453.MathSciNet ID: 2969225[DOI: https://dx.doi.org/10.1080/07350015.2012.693850]

Salim, RA; Hassan, K; Shafiei, S. Renewable and non-renewable energy consumption and economic activities: Further evidence from OECD countries. Energy Econ; 2014; 44, pp. 350-360. [DOI: https://dx.doi.org/10.1016/j.eneco.2014.05.001]

Stock, JH; Watson, MW. Combination forecasts of output growth in a seven‐country data set. J Forecast; 2004; 23, 6 pp. 405-430. [DOI: https://dx.doi.org/10.1002/for.928]

Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodological 58(1):267–288

Timmermann, A. Forecast combinations. Handb Econ Forecast; 2006; 1, pp. 135-196. [DOI: https://dx.doi.org/10.1016/S1574-0706(05)01004-9]

Wang, X; Xiao, W; Yu, J. Modeling and forecasting realized volatility with the fractional Ornstein–Uhlenbeck process. J Econ; 2023; 232, 2 pp. 389-415.MathSciNet ID: 4539492[DOI: https://dx.doi.org/10.1016/j.jeconom.2021.08.001]

Wen, F; Gong, X; Cai, S. Forecasting the volatility of crude oil futures using HAR-type models with structural breaks. Energy Econ; 2016; 59, pp. 400-413. [DOI: https://dx.doi.org/10.1016/j.eneco.2016.07.014]

Zhang, YJ. Interpreting the dynamic nexus between energy consumption and economic growth: empirical evidence from Russia. Energy Policy; 2011; 39, 5 pp. 2265-2272. [DOI: https://dx.doi.org/10.1016/j.enpol.2011.01.024]

Zhao, L; Liu, W; Zhou, M; Wen, F. Extreme event shocks and dynamic volatility interactions: The stock, commodity, and carbon markets in China. Financ Res Lett; 2022; 47, [DOI: https://dx.doi.org/10.1016/j.frl.2021.102645]