Sustainability, Accuracy, Fairness, and Explainability (SAFE) Machine Learning in Quantitative Trading

Abstract

The paper investigates the application of advanced machine learning (ML) methodologies, with a particular emphasis on state-of-the-art deep learning models, to predict financial market dynamics and maximize profitability through algorithmic trading strategies. The study compares the predictive capabilities and behavioral characteristics of traditional machine learning approaches, such as logistic regression and support vector machines, with those of highly sophisticated deep learning architectures, including Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, and Gated Recurrent Units (GRUs). The findings underscore the fundamental distinctions between these methodologies, with deeply trained models exhibiting markedly different predictive behaviors and performance, particularly in capturing complex temporal patterns within financial data. A cornerstone of the paper is the introduction and rigorous analysis of a framework to evaluate models, by means of the SAFE framework (Sustainability, Accuracy, Fairness, and Explainability). The framework is designed to address the opacity of black-box ML models by systematically evaluating their behavior across a set of critical dimensions. It also demonstrates how models’ predictive outputs align with the observed data, thereby reinforcing their reliability and robustness. The paper leverages historical stock price data from International Business Machines Corporation (IBM). The dataset is partitioned into a training phase during which the models are calibrated, and a validation phase, used to evaluate the predictive performance of the generated trading signals. The study addresses two primary machine learning tasks: regression and classification. Classical models are utilized for classification tasks, with their outputs directly interpreted as trading signals, while advanced deep learning models are employed for regression, with predictions of future stock prices further processed into actionable trading strategies. To evaluate the effectiveness of each strategy, rigorous backtesting is conducted, incorporating visual representations such as equity curves to assess profitability and key risk metrics like maximum drawdown for risk management. Supplementary performance indicators, including hit rates and the incidence of false positions, are analyzed alongside the equity curves to provide a holistic assessment of each model’s performance. This comprehensive evaluation not only highlights the superiority of cutting-edge deep learning models in predicting financial market trends but also demonstrates the pivotal role of the SAFE framework in ensuring that machine learning models remain trustworthy, interpretable, and aligned with ethical considerations.

1. Background: Motivation and Literature Review

This paper aims to evaluate the integration of advanced machine learning techniques, particularly neural networks with core mathematical and statistical methods, to develop advanced trading strategies in financial markets. Neural networks are positioned as potent tools for uncovering complex patterns in market data that traditional statistical approaches may fail to detect. This study aims to utilize these capabilities to improve the accuracy of market predictions and trading decisions. Specifically, the objectives of this paper include the development of a framework that combines neural networks with statistical methodologies, such as time series analysis and regression models, to forecast asset prices and identify profitable trading opportunities. Furthermore, this research investigates the adaptability of these models to varying market conditions, such as different volatility regimes, and assess their capacity to generalize across diverse asset classes, including equities, commodities, and cryptocurrencies. A key objective is to evaluate the performance of the models in both short-term trading (high-frequency trading) and long-term investment contexts, thereby providing a thorough assessment of their practical utility. Additionally, the study focuses on the interpretability of these models, ensuring that the predictions generated by the neural networks are comprehensible and reliable for human traders and analysts. The integration of neural networks with conventional mathematical and statistical frameworks is expected to provide a dual advantage: the ability of neural networks to process and extract insights from large, unstructured datasets, alongside the theoretical foundation provided by mathematical models rooted in established financial theory.

Previous studies, such as those by Hu et al. (2020) [1] and Hiransha et al. (2018) [2], have shown that such hybrid approaches can lead to enhanced predictive performance and the development of more robust trading strategies.

We now review the most important machine learning algorithms for financial time series analysis. According to Shalev and Ben (2014) [3], the most effective approach to understanding a machine learning algorithm is through the characterization of its hypothesis class. A hypothesis class serves as a mathematical formalization of the algorithm, in which the algorithm’s behavior is rigorously defined. Furthermore, the hypothesis is intrinsically linked to other critical components, such as the loss function. For example, the hypothesis class for logistic regression can be expressed as follows:

(1)$\begin{array}{ccc}\hfill {\mathcal{H}}_{lr}& =& \sigma •{\mathcal{Q}}_d(\theta ,x,b)\hfill \end{array}$

(2)$\begin{array}{ccc}\hfill {\mathcal{Q}}_d(\theta ,x,b)& =& {\displaystyle \{h_{\theta }(x)=\sum _{i=1}^d{x}_i{\theta }_i+b,\forall x,\theta \in {\mathbb{R}}^d,b\in \mathbb{R}\}}\hfill \end{array}$

(3)$\begin{array}{ccc}\hfill \sigma (u)& =& {\displaystyle \frac{1}{1+e^{-u}},\forall u\in \mathbb{R}}\hfill \end{array}$

(4)$\begin{array}{ccc}\hfill \ell (\theta ,x,y)& =& log(1+e^{-y(x^{\prime }\theta +b)})\hfill \end{array}$

where ${\mathcal{H}}_{lr}$ is the hypothesis class of logistic regression, notation • is a chained operand between functions, ${\mathcal{Q}}_d(\theta ,x,b)$ is a linear combination from two d-dimensional vector $x,\theta$ and the intercept b, $\sigma (u)$ is an activation, and $\ell (\theta ,x,y)$ is a loss function. Our objective is to optimize the generalized loss function (or objective function) subject to several conditions, including risk minimization, structural risk minimization, and regularized risk minimization. The primary distinction among these conditions lies in the formulation of the objective function and its associated components. In the following, we elucidate key theoretical concepts and foundational issues involved in the design of machine learning algorithms.

Univariate time-series data do not inherently conform to a supervised learning framework, rendering the direct application of supervised machine learning algorithms impractical. However, such time-series data—whether univariate or multivariate—can be transformed into a tabular format suitable for supervised learning. To this end, consider a univariate time series indexed from 1 to 10, with a lag of 3. These data can be restructured into a supervised learning format using the following logic: observations indexed 1, 2, and 3 are used to predict the value indexed by 4; similarly, observations indexed 2, 3, and 4 predict the value indexed by 5, and so on. Mathematically, the univariate time-series data can ultimately be represented as a matrix $\mathbb{X}$ and a vector y where $\mathbb{X}$ corresponds to the input variables and y corresponds to the output variables in the supervised learning context. A summary of this pseudo-input and output could be described as follows (Table 1):

A critical distinction exists between the re-framed matrix $\mathbb{X}$ and a vector y; as noted by Matthew et al. (2020) [4], these quantities are sequentially ordered, which precludes the possibility of randomizing, shuffling, or bootstrapping to augment data diversity. This constraint represents a significant challenge in the context of financial applications. Each combination of trainable parameters $\theta$ and hyperparameters constitutes a distinct supervised model. Consequently, if these parameters are not appropriately constrained, an infinite number of possible models may arise. As a result, the problem ultimately becomes one of fine-tuning both the trainable parameters and hyperparameters. Shalev and Ben (2014) [3] highlight two commonly employed approaches for addressing this issue: Structural Risk Minimization (SRM) and cross-validation. SRM approach is particularly effective when the hypothesis class can be partitioned into a countable set of sub-hypothesis classes ${\mathcal{H}}_1,{\mathcal{H}}_2\cdots ,{\mathcal{H}}_n$. Under these conditions, the optimal model $h^{*}$ could be characterized as follows:

(5)$\begin{array}{ccc}\hfill h^{*}& \in & argmi{n}_{\forall h\in \mathcal{H}}[{\mathcal{L}}_S(\theta )+{ϵ}_{n(h)}(m,w_{n(h)}\zeta )]\hfill \end{array}$

where $\zeta$ is a confidence parameter. Weights $w_i$, $\forall i=1,\cdots ,m$ satisfy ${\sum }_{i=1}^m{w}_i\le 1$, and ${ϵ}_n(m,w_n\zeta )=su{p}_{h,\theta \in {\mathcal{H}}_n}|{\mathcal{L}}_D(\theta )$ − ${\mathcal{L}}_S(\theta )|$. ${ϵ}_n(m,w_n\zeta )$ is the supreme value of the sequence of the difference between the true loss and the training loss given function h.

Shalev and Ben (2014) [3] identified a notable application of Structural Risk Minimization (SRM), namely, the Minimum Description Length (MDL) principle, which is extensively utilized, particularly in algorithms such as Decision Trees. Through a judicious selection of the weight parameters, the right-hand side of Equation (5) could be simplified. John, Peter, Robert, and Martin (1996) [5] presented theoretical frameworks and examples of SRM; however, these proved challenging to implement due to their mathematical complexity. While SRM can be effective in certain contexts, the practical application of Equation (5) often leads to significant pessimism. As such, the search for optimal models is more pragmatically conducted through approaches such as cross-validation. Matthew et al. (2020) [4] further define cross-validation as a method for hyperparameter optimization, achieved through the rotation of k folds within the training dataset. In this approach, models are trained on k − 1 folds independently, and validation is performed on the remaining fold. It is critical that the k-fold procedure respects the sequential ordering of the data. Figure 1 illustrates a pseudo cross-validation scheme for sequential data. In this study, we propose two variations of cross-validation tailored to time-series data within a supervised learning framework. Both methods ensure that the validation set remains independent of the training set, while preserving the temporal sequence inherent in the original dataset.

The capacity to generalize in unseen environments must be tightly constrained in order to prevent overfitting and enhance the robustness of the model. Shalev and Ben (2014) [3] illustrate that a model is considered stable if a small perturbation in the input does not lead to a disproportionately large change in the output. Formally, let $\mathcal{A}$ be an algorithm, $\mathcal{S}=\{z_i=(x_i,y_i)\}$ $\forall i=1,\cdots ,m$ an m-tuple training set, and let $\mathcal{A}(\mathcal{S})$ denote the outcome of an algorithm $\mathcal{A}$ with the training set $\mathcal{S}$ as input. Given a training set $\mathcal{S}$, an additional observation $z^{*}$, ${\mathcal{S}}_i$ denotes the training set with a replacement $z^{*}$ for $z_i$. The stability of a model is then measured by comparing the loss value of the outcome $\mathcal{A}(\mathcal{S})$ on $z_i$ to the loss value of $\mathcal{A}({\mathcal{S}}_i)$ on $z_i$. For clarity, model stability is quantified as $\ell (\mathcal{A}({\mathcal{S}}_i,z_i))$ − $\ell (\mathcal{A}(\mathcal{S}),z_i)$. Shalev and Ben (2014) [3] have proposed that the stability of a model is closely linked to the phenomenon of overfitting, as described below:

(6)$\begin{array}{ccc}\hfill \mathbb{E}({\mathcal{L}}_{\mathcal{D}}(\mathcal{A}(\mathcal{S}))-{\mathcal{L}}_{\mathcal{S}}(\mathcal{A}(\mathcal{S})))& =& \mathbb{E}(\ell (\mathcal{A}({\mathcal{S}}_i,z_i)-\ell (\mathcal{A}(\mathcal{S})),z_i))\hfill \end{array}$

where ${\mathcal{L}}_{\mathcal{D}}(\mathcal{A}(\mathcal{S}))$ and ${\mathcal{L}}_{\mathcal{S}}(\mathcal{A}(\mathcal{S}))$ are the true and training loss given algorithm $\mathcal{A}$. The proof is given in the appendix.

The issue of stability ultimately reduces to bounding the expected value of the loss function: $\mathbb{E}(\ell (\mathcal{A}({\mathcal{S}}_i,z_i)$ − $\ell (\mathcal{A}(\mathcal{S})),z_i))$. To address this, the loss function is endowed with the strong properties of a Lipschitz continuous or smoothed function and is trained under an ${\ell }_2$ norm constraint. Further, the objective function comprises the training loss function, augmented by a regularization term involving the ${\ell }_2$ norm of the variable $\theta$:

(7)$\begin{array}{c}\hfill {\displaystyle {\mathcal{L}}_{\mathcal{S}}(\subseteq )+\frac{\gamma }{2}{\parallel \subseteq \parallel }^2=\frac{1}{m}\sum _{i=1}^m\ell (\theta ,x_i,y_i)+\frac{\gamma }{2}{\parallel \theta \parallel }^2}\end{array}$

where $\gamma$ is a regularization hyperparameter, controlling the stability of the model.

We now focus on the specific models that we considered in the empirical analysis of the paper.

1.1. Logistic Regression

Logistic regression is a machine learning algorithm learning from $\mathbb{X}$ to the interval $[0,1]$.

(8)$\begin{array}{ccc}\hfill {\mathcal{H}}_{lr}& =& \sigma •{\mathcal{Q}}_d(\theta ,x,b)\hfill \end{array}$

(9)$\begin{array}{ccc}\hfill {\mathcal{Q}}_d(\theta ,x,b)& =& {\displaystyle \{h_{\theta }(x)=\sum _{i=1}^d{x}_i{\theta }_i+b,\forall x,\theta \in {\mathbb{R}}^d,b\in \mathbb{R}\}}\hfill \end{array}$

(10)$\begin{array}{ccc}\hfill \sigma (u)& =& {\displaystyle \frac{1}{1+e^{-u}},\forall u\in \mathbb{R}}\hfill \end{array}$

(11)$\begin{array}{ccc}\hfill {\ell }_{lr}(\theta ,x,y)& =& log(1+e^{-y(x^{\prime }\theta +b)})\hfill \end{array}$

The loss function, ${\ell }_{lr}(\theta ,x,y)$ in (13) is recognized as a convex function with respect to $\theta$, ensuring the effectiveness of the optimization process. To further enhance stability and mitigate the risk of overfitting, we introduce a regularization term to the objective function:

(12)$\begin{array}{ccc}\hfill {\displaystyle {\mathcal{L}}_{\mathcal{S}}(\subseteq )+\frac{\gamma }{2}{\parallel \subseteq \parallel }^2}& =& {\displaystyle \frac{1}{m}\sum _{i=1}^m{e}^{y_i(x_i^{\prime }\theta +b)}+\frac{\gamma }{2}{\parallel \theta \parallel }^2}\hfill \end{array}$

1.2. Support Vector Machines

A support vector machine (SVM) is a machine learning algorithm that employs either a linear or nonlinear hyperplane to partition observations into distinct classes. The algorithm classifies a new sample based on its features, assigning it to one side of the hyperplane. The SVM seeks to maximize the margin, defined as the distance between the hyperplane and the nearest observations, such that the training set is separated with the largest possible margin. This ensures that all observations not only lie on the correct side of the hyperplane but are also as distant from the margin as possible. In short, the hypothesis class of SVMs could be described as follows:

(13)$\begin{array}{ccc}\hfill {\mathcal{H}}_{svm}& =& \sigma •{\mathcal{Q}}_d(\theta ,x,b)\hfill \end{array}$

(14)$\begin{array}{ccc}\hfill {\mathcal{Q}}_d(\theta ,x,b)& =& {\displaystyle \{h_{\theta }(x)=\sum _{i=1}^d{x}_i{\theta }_i+b,\forall x,\theta \in {\mathbb{R}}^d,b\in \mathbb{R}\}}\hfill \end{array}$

(15)$\begin{array}{ccc}\hfill {\ell }_{svm}(\theta ,x,y)& =& max\{0,1-y(x^{\prime }\theta +b)\}\hfill \end{array}$

where ${\ell }_{hinge}(\theta ,x,y)=max\{0,1$-$y(x^{\prime }\theta +b)$ is the hinge loss function. To enhance stability and mitigate the risk of overfitting, the objective function may be formulated as follows:

(16)$\begin{array}{ccc}\hfill {\displaystyle {\mathcal{L}}_{\mathcal{S}}(\subseteq )+\frac{\gamma }{2}{\parallel \subseteq \parallel }^2}& =& {\displaystyle \frac{1}{m}\sum _{i=1}^{m}max\{0,1-y_i(x_i^{\prime }\theta +b)\}+\frac{\gamma }{2}{\parallel \theta \parallel }^2}\hfill \end{array}$

1.3. Deep Learning

Deep learning is a subfield of machine learning leveraging a diverse array of neural network architectures in conjunction with advanced optimization techniques. It has been successfully applied across various domains, from social sciences to finance, demonstrating its broad utility and effectiveness. Deep learning has gained significant prominence in the analysis of financial data due to its capacity to model complex, nonlinear relationships within large datasets. Financial data, often characterized by a low signal-to-noise ratio, present challenges that traditional statistical methods may not adequately address. Deep learning models, particularly those based on neural network architectures such as Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks, are particularly well suited for time-series forecasting. Furthermore, deep learning benefits from the integration of vast and diverse data sources, including historical prices, trading volumes, and even alternative data. Nevertheless, the application of deep learning in finance necessitates careful attention to issues of model interpretability and the risk of overfitting, given the potential consequences for trading strategies and investment decisions, as highlighted by Fischer and Krauss (2018) [6].

Deep learning has fundamentally transformed numerous fields, including computer vision, medicine, recommendation systems, and quantitative finance. Its applications span a wide range of tasks, from human speech recognition and image detection to stock price prediction. This section focuses on the general architecture of Recurrent Neural Networks (RNNs) and their variants, all of which are primarily concerned with the analysis of financial time series and the generation of investment signals. Deep learning has gained considerable traction in time-series modeling, as evidenced by seminal works such as those of Paul (1988) [7], Zaiyong, Chrys, and Paul (1991) [8], Julian and Crhis (1998) [9], Zhang and Min (2005) [10], Sepp and Jurgen (1997) [11], or Guoqiang, Patuwo, and Michael (1998) [12] who have proposed neural network-based time-series models within the context of financial and commodity markets. In this research, RNNs are conceptualized as a nonlinear extension of classical financial econometric models, such as ARIMA(p, d, q) and AR(p).

In short, an RNN is described as a function of sequences $x_t,y_t$ as follows:

(17)$\begin{array}{ccc}\hfill y_{t+h}& =& f({\mathbb{X}}_{t,t-j}),h\ge 1,j\ge 1\hfill \end{array}$

(18)$\begin{array}{ccc}\hfill {\mathbb{X}}_{t,t-j}& =& \{{\mathbb{X}}_t,{\mathbb{X}}_{t-1},\cdots ,{\mathbb{X}}_{t-j}\}\hfill \end{array}$

where $y_{t+h}$ is the output of the model, h$((h\ge 1))$ implies the number of steps ahead forecast in the future. ${\mathbb{X}}_{t,t-j}$ is the collection of lagged values of the input.

An RNN offers greater flexibility than a simple linear function, as it can employ activation functions such as tanh or other nonlinear functions. Mathematically, the formula of an RNN could be expressed as follows:

(19)$\begin{array}{ccc}\hfill y_{t+h}& =& {\sigma }^{(2)}(W_z^{(2)}{z}_t+b^{(2)})+{ϵ}_t\hfill \end{array}$

(20)$\begin{array}{ccc}\hfill z_{t-j}& =& {\sigma }^{(1)}(W_x^{(1)}{x}_{t-j}+W_z^{(1)}{z}_{t-j-1})+b^{(1)})\hfill \end{array}$

where $z_{t-j}$, $x_{t-j}$, and $y_{t+h}$ are hidden state, input, and output variables, respectively. h indicates the number of steps ahead forecast in the future. $W_x^{(1)}$ is a connection matrix between the input $x_{t-j}$ and the hidden state $z_{t-j}$, $W_z^{(1)}$ connects the hidden state $z_{t-j}$ and its lagged value, and $W_z^{(2)}$ bridges the hidden state $z_t$ and the output $y_{t+h}$. $W_x^{(1)},W_z^{(1)},$ and $W_z^{(2)}$ are time-invariant. ${\sigma }^{(1)},{\sigma }^{(2)}$ are nonlinear activation functions, ${ϵ}_t$ as a noise.

Figure 2 illustrates the architecture of an RNN. A key distinguishing feature of the RNN, as compared to other fully connected neural networks, is its mechanism of sharing trainable parameters. It is evident from Figure 2 that the same matrix of coefficients is used to map the input $x_{t-i}$ to the hidden states $z_{t-i}$ as well as for the connections between the hidden states and their lagged values. This parsimony in the number of trainable parameters is a principal motivation for modifying the structure of classical fully connected neural networks. Additionally, a reduction in the number of trainable parameters typically results in a lower propensity for overfitting and a decrease in the training time required.

The Long Short-Term Memory (LSTM) network is a variant of the Recurrent Neural Network (RNN) model. While it retains the foundational features of the RNN, it introduces the novel concept of a long-term memory cell $c_t$. In addition to the memory cell, LSTM incorporates four gates: input, output, and forget gates, which regulate the flow of information within the network. In the absence of these gates, the LSTM effectively reduces to a standard RNN, as noted by Aurélien (2019) [13]. The architecture of the LSTM is illustrated in Figure 3. The detailed structure of the LSTM can be described as follows:

(21)$\begin{array}{ccc}\hfill i_t& =& \sigma (W_{xi}{x}_t+W_{hi}{h}_{t-1}+b_i)\hfill \end{array}$

(22)$\begin{array}{ccc}\hfill f_t& =& \sigma (W_{xf}{x}_t+W_{hf}{h}_{t-1}+b_f)\hfill \end{array}$

(23)$\begin{array}{ccc}\hfill o_t& =& \sigma (W_{xo}{x}_t+W_{ho}{h}_{t-1}+b_o)\hfill \end{array}$

(24)$\begin{array}{ccc}\hfill g_t& =& tanh(W_{xg}{x}_t+W_{hg}{h}_{t-1}+b_g)\hfill \end{array}$

(25)$\begin{array}{ccc}\hfill c_t& =& f_t\otimes c_{t-1}+i_t\otimes g_t\hfill \end{array}$

(26)$\begin{array}{ccc}\hfill y_t& =& h_t=o_t\otimes tanh(c_t)\hfill \end{array}$

where $W_{xi}$ and $W_{hi}$ are the connection matrices between the input $x_t$, the previous hidden state $h_{t-1}$, and the input gate $i_t$, respectively. The coefficient $b_i$ could be the intercept of the input gate $x_t$, the previous hidden state $h_{t-1}$, and the output gate $o_t$ with the intercept $b_o$. $W_{xg},W_{hg}$ are the connection matrices between input information $x_t$, the previous hidden state $h_{t-1}$, and the main gate $g_t$ with the intercept $b_g$. $W_{xi},W_{hi},W_{xf},W_{hf},W_{xo},W_{ho},W_{xg},W_{hg},b_i,b_f,b_o$, and $b_g$ are all time-invariant. $\sigma (\theta )$ and $tanh(\theta )$ are a sigma and a tanh activation function, and the mathematical notation ⊗ is an element-wise product.

In Figure 3, the information of the long-term memory cell $c_t$ is connected to its preceding value $c_{t-1}$ via the forget gate $f_t$, and the product of the information from the main gate, $g_t$, and the product of the information from the main gate, $i_t$. Specifically, the information from the previous memory cell is filtered through the forget gate, and then enriched with additional relevant information from the input gate and the main gate. Subsequently, the cell state $c_t$ undergoes further transformation through a tanh activation function, before being modulated by the output gate $o_t$ and ultimately passed to the output $y_t$.

The Gated Recurrent Unit (GRU) is a simplified variant of the Long Short-Term Memory (LSTM) model. While both LSTM and GRU share common features, such as the forget and control mechanisms that regulate the retention or modification of salient information from historical data, the GRU does not incorporate a separate long-term memory cell $c_t$ as in LSTM. Instead, the GRU utilizes a set of gating mechanisms, specifically, the update gate $z_t$, reset gate $r_t$, and main gate $g_t$. As described by Aurélien (2019) [13], the structure of the GRU can be outlined as follows (Figure 4):

Aurélien (2019) [13] also proposed the mathematical underpinnings of the GRU model as follows:

(27)$\begin{array}{ccc}\hfill z_t& =& \sigma (W_{xz}{x}_t+W_{hz}{h}_{t-1}+b_z)\hfill \end{array}$

(28)$\begin{array}{ccc}\hfill r_t& =& \sigma (W_{xr}{x}_t+W_{hr}{h}_{t-1}+b_r)\hfill \end{array}$

(29)$\begin{array}{ccc}\hfill g_t& =& tanh(W_{xg}{x}_t+W_{hg}(r_t\otimes h_{t-1})+b_g)\hfill \end{array}$

(30)$\begin{array}{ccc}\hfill h_t& =& z_t\otimes h_{t-1}+(1-z_t)\otimes g_t\hfill \end{array}$

where $W_{xz}$ and $W_{hz}$ are the connection matrices between the input information $x_t$ and the previous hidden state $h_{t-1}$. The coefficient $b_z$ is an intercept of that gate. The matrices $W_{xr}$ and $W_{hr}$ connect the input information $x_t$ and the previous hidden state $h_{t-1}$ with an intercept $b_r$. $W_{xg}$ and $W_{hg}$ are the connection matrices between the input information $x_t$ and a part of the previous hidden state $h_{t-1}$ filter by the value of reset gate $r_t$. $W_{xz},W_{hz},W_{xr},W_{hr},W_{xg},W_{hg},b_z,b_r$, and $b_g$ are time-invariant. $\sigma (\theta )$ and $tanh(\theta )$ are a sigma and a tanh activation function. The mathematical notation ⊗ is an element-wise product.

In the empirical results section of this paper we compare the above models on a real dataset, and we also compare them with another state-of-the-art deep learning model, a convolutional network (CNN), whose details we omit, for lack of space. The rest of the paper is organized as follows. In the next section, we present the data we employed in our analysis. In Section 3, we show the results deriving from the application of machine learning models, in terms of quantitative trading. In Section 4, we introduce and formalize the SAFE framework, which is then applied to the data in Section 5. Section 6 concludes this paper.

2. Data

For the purposes of this research, the selected dataset consisted of the stock prices of International Business Machines (IBM), with the time period under consideration spanning from January 2000 to March 2021. The training phase of the model extended from the beginning of the dataset until February 2020, while the testing phase covered the period from February 2020 to March 2021. Table 2, Table 3 and Table 4 present statistical summaries for the entire dataset, the training period, and the testing period, respectively. Notably, Table 4 provides a summary of the standard deviation (volatility) for the testing period, which was $8.05$.

Figure 5 presents 5322 observations, of which 4789 observations (representing $95\%$ the dataset) were allocated for training, and the remaining 533 observations (constituting $5\%$ of the dataset) were reserved for testing. As illustrated, the training period exhibited greater stock price volatility when compared to the testing period.

In this paper, data analysis was performed through Python (latest v. 3.13.1), which has shown its value in several financial deep learning projects (see, e.g., Chollet (2018) [14] and Paszke et al. (2019) [15]). As a result, Python has become the preferred language for deep learning, empowering researchers and practitioners to advance the frontiers of artificial intelligence.

3. Results: Machine Learning Models

To analyze the traded financial products with machine learning models, we propose an eight-step procedure, as follows:

• Step 1: Split a time-series data into training and testing parts;

• Step 2: Scale both training and testing parts by min-max or normal measurement;

• Step 3: Convert a time-series data in both parts into a supervised learning form;

• Step 4: Perform cross-validation on the training part to seek the optimal hyperparameters;

• Step 5: Apply ${\ell }_2$ norm regularization to prevent overfitting;

• Step 6: Train and validate the model on the training and testing part, respectively;

• Step 7: Construct a trading strategy from the results from the testing part;

• Step 8: Backtest the trading strategy.

Step 1 involves partitioning the time-series data into distinct training and testing subsets, ensuring that the model is validated independently and mitigating the risk of look-ahead bias. This division also facilitates the evaluation of the model’s performance within a real-world testing environment. Step 2 outlines a method for scaling the original data to ensure consistency and improve model efficiency. Steps 3 and 4 detail the process of transforming the time-series data into a tabular format suitable for supervised learning, as well as the optimization of hyperparameters. Cross-validation is performed rigorously and effectively during these steps to enhance model robustness and accuracy. As highlighted by Shalev and Ben (2014) [3], one issue in the design of machine learning models is overfitting. Step 5 addresses this issue by introducing a regularization technique aimed at mitigating overfitting. It is important to note that the regularization parameter, denoted as $\gamma$, is highly sensitive and functions as a trade-off factor that balances the training loss ${\mathcal{L}}_S(\theta )$ with the true loss $\mathcal{L}D(\theta )$. In Step 6, the model is trained in a manner that is agnostic to the specific task, allowing it to be applied to either regression or classification. In Step 7, the procedure generates a full set of pseudo-trading signals, which are considered simulated trading signals for the purpose of evaluation. Finally, Step 8 assesses the performance of each strategy through intraday vectorized backtesting. The results are visualized on equity curves, which depict the cumulative profit or loss over time. Additionally, the drawdown indicator, an essential measure in quantitative trading, is also examined. The drawdown represents the maximum loss experienced by the strategy during the testing period and serves as a key metric for evaluating the model’s risk management effectiveness.

Model estimation is a fundamental step in the development of statistical learning methodologies. In traditional machine learning models, such as linear regression, logistic regression, and support vector machines, well-established estimation techniques such as Maximum Likelihood Estimation (MLE) and Ordinary Least Squares (OLS) are employed. These techniques operate by optimizing a predefined loss function to identify the optimal model parameters. In contrast, deep learning models, including neural networks, necessitate more sophisticated estimation processes due to their highly nonlinear structures and the large number of parameters involved. These models typically utilize gradient-based optimization methods, such as Stochastic Gradient Descent (SGD) and its variants (e.g., Adam), to iteratively adjust weights and biases in order to minimize the loss function. Furthermore, regularization techniques, such as $l2$ regularization and dropout, are incorporated to mitigate the risk of overfitting during the estimation process, thereby enhancing the model’s ability to generalize to unseen data, as discussed by Goodfellow et al. (2016) [16] and LeCun et al. (2015) [17]. In this context, we distinguish between two primary types of model-based signal strategies: those based on deep learning models and those grounded in classical machine learning techniques.

The deep learning models employed in this research were trained to produce one-step-ahead forecasts, predicting the subsequent stock price based on historical price data, and subsequently interpreting these forecasts into actionable trading signals. In contrast, the classical models were designed to generate strategy-specific signals. In the present research, “short-selling” was excluded from the strategy, and the trading approach was restricted to long-only positions. While the backtesting procedure served to demonstrate the efficacy of the strategy, it is important to note that transaction costs, which are crucial in real-world investment scenarios, were not incorporated into the evaluation framework. To mitigate look-ahead bias, the prediction period was distinctly separated from the training period, thereby ensuring the strategy’s practical applicability and robustness when deployed in real-world settings. This research presents a comprehensive analysis of IBM’s stock price prediction using Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, and Gated Recurrent Units (GRUs). Figure 6, Figure 7 and Figure 8 illustrate the stock price predictions generated by each model. Although the models exhibit similar predictive patterns, they differ in their respective levels of root-mean-square error (RMSE). While none of the models fully capture the exact movements of the stock price, they demonstrate a notable capacity to identify and track the primary trends in the price dynamics.

Figure 6 presents the one-step-ahead forecast of the stock price, utilizing the previous 10 data points as input. This approach ensured that a complete set of predictions was generated for the period from February 2019 to March 2021, thereby effectively mitigating the risk of look-ahead bias. The root-mean-square error (RMSE) associated with the predictions, as illustrated in Figure 6, Figure 7 and Figure 8, was $$2.40$, $$2.34$, and $$2.36$ for the RNN, LSTM, and GRU, respectively.

The summary of the root-mean-square error of each model is given below (Table 5):

From Figure 9, Figure 10 and Figure 11, it is evident that during the testing period, the RNN, LSTM, and GRU-based strategies placed 373, 363, and 498 long orders, respectively, out of a total of 523 orders. The corresponding placement rates for long orders were $71.33\%$, $69.40\%$, and $95.20\%$, with the GRU-based strategy exhibiting the highest rate of long positions, followed by the RNN- and LSTM-based strategies (Table 6). Further analysis, as shown in Figure 9, indicates that the RNN-based strategy correctly identified 194 out of 279 “ideal” long positions, yielding a hit rate of $70\%$. Similarly, Figure 10 and Figure 11 reveal that the LSTM-based strategy achieved a hit rate of $69\%$ with 191 accurate long positions out of 279, while the GRU-based strategy attained the highest hit rate of $96.40\%$, correctly identifying 269 of the 279 ideal long positions.

However, a significant number of “false positions” could be observed, which arose from inaccuracies in the predicted prices. As depicted in Figure 9, Figure 10 and Figure 11, several instances of downturn periods are evident where the models continued to place long positions despite the market’s adverse conditions. This suggests that the models, while capturing general trends, may not have fully accounted for market corrections or downturns, leading to suboptimal decisions during periods of price decline.

Figure 12, Figure 13 and Figure 14 present the equity curves for the deep learning model-based signals in comparison to the buy-and-hold strategy. These equity curves illustrate the daily cumulative profit or loss over the testing period, with performance measured via backtesting. A careful inspection of the graphs reveals that the cumulative profit can be divided into three distinct phases: the initial phase from February 2019 to early March 2020, a second phase around March 2020, and the final phase from March 2020 to the end of the period. In particular, the first phase demonstrates the outperformance of the buy-and-hold strategy, followed by a notable decline in profitability for both strategies during the second phase. However, the last phase reveals a marked improvement in the performance of the deep learning model-based strategies, which ultimately outperformed the buy-and-hold approach.

Table 7 presents the summary of the rate of false positions in each model.

From Table 7, it is observed that the GRU-based signals resulted in the highest rate of false positions, followed by the RNN- and LSTM-based strategies. This discrepancy highlights the varying accuracy of the models in generating trading signals. To further assess the efficacy of each strategy, the equity curves were analyzed. The equity curve was derived directly from backtesting, and more specifically, it was computed by aggregating the daily returns associated with the pseudo-long positions generated by the respective models. The equity curve thus provides a clear representation of the cumulative performance of each strategy over the testing period.

Figure 12 presents the equity curves derived from two vectorized backtests: one based on the RNN-generated signals and the other corresponding to the buy-and-hold strategy. Both simulations cover the trading period from February 2019 to February 2021. From February 2019 to March 2020, the RNN-based strategy demonstrated a relatively modest profit, underperforming compared to the buy-and-hold strategy. During that initial period, the maximum profit of the buy-and-hold strategy reached approximately 25%, while the RNN-based strategy achieved only around 5%. However, the situation changed dramatically after a significant drop in IBM’s stock price around May 2020. As shown in Figure 12, while the equity curve of the buy-and-hold strategy declined to below −25%, the RNN-based strategy experienced a less severe dip, reaching only −15%. This divergence in performance indicates that the RNN-based strategy was more resilient during periods of sharp market downturns. Following the recovery in stock prices after March 2020, the RNN-based strategy outperformed the buy-and-hold strategy. By the end of the period, the RNN-based strategy achieved an accumulated profit of 14.3%, corresponding to a compound annual growth rate (CAGR) of 10%, while the buy-and-hold strategy realized a modest 0.6% return, corresponding to a CAGR of 0.4%.

The key summaries of both strategies can be described below (Table 8):

Similarly, in Figure 13, we present two equity curves derived from the backtesting of the LSTM-based strategy and the buy-and-hold strategy. While the buy-and-hold strategy posted higher profits from February 2019 to March 2020, the gap between the LSTM-based strategy’s profit and the buy-and-hold profit was narrower compared to the gap observed between the RNN-based strategy and the buy-and-hold strategy. Figure 12 and Figure 13 also demonstrate that the maximum loss of the RNN-based strategy during that period was approximately −10.20%, whereas the maximum loss of the LSTM-based strategy was −7.62%, indicating that the LSTM-based signals outperformed the RNN-based signals during the first phase of the period. Furthermore, during the dramatic downturn in IBM’s stock price, the maximum loss of the LSTM-based strategy was limited to −13.5%, while the corresponding losses for the buy-and-hold strategy and the RNN-based strategy were −25.15% and −18.07%, respectively. At the end of the period, Figure 13 reveals that the cumulative profit for the LSTM-based strategy reached 21.77%, corresponding to a compound annual growth rate (CAGR) of 10.50%, whereas the cumulative profit of the buy-and-hold strategy was only 0.6%, with a CAGR of 0.4%. Moreover, the drawdown of the LSTM-based strategy was also more favorable than that of the buy-and-hold strategy. The key summary of both strategies is outlined below (Table 9):

Figure 14 also presents two equity curves derived from the daily backtesting of the GRU-based strategy and the buy-and-hold strategy. From February 2019 to August 2019, the cumulative profits of both the GRU-based and the buy-and-hold strategies were nearly identical. However, after August 2019, the equity curve of the buy-and-hold strategy began to rise, outperforming the GRU-based strategy. The maximum accumulated profit of the buy-and-hold strategy reached 28.50%, while the GRU-based strategy’s peak was limited to 21.80%. The situation changed dramatically starting in March 2020. During that period, the cumulative profits of both strategies dropped significantly, although at different rates. The GRU-based strategy experienced its largest loss, dipping to approximately −12%, which was the lowest recorded drawdown for any of the strategies tested. In contrast, the buy-and-hold strategy’s cumulative profit collapsed by more than −25%, reflecting a more severe downturn.

At the end of the evaluation period, the accumulated profit of the GRU-based strategy reached 26.70%, corresponding to a compound annual growth rate (CAGR) of 18.65%. In contrast, the final accumulated profit of the buy-and-hold strategy for IBM was limited to 0.6%, which translates to a CAGR of 0.40%. The key summary of both strategies is provided follow (Table 10):

We now turn to the prediction of IBM’s stock price using two classical machine learning models. Figure 15 presents the long orders generated by the logistic regression (LR) model. A total of 325 long orders were placed out of 523 during the testing period from February 2019 to March 2020, corresponding to a long order rate of 62.15%. Similarly, Figure 16 illustrates the long orders issued by the support vector machine (SVM) model, which placed 324 long orders out of 523, resulting in a long order rate of 61.95%. Notably, the long-order placement rates for both the LR and SVM models were comparable, distinguishing them from deep learning models, which typically exhibited more variability in order placement. In the context of trading, two important performance metrics emerge: first, the accuracy of the long positions placed, and second, the rate of false long positions. Figure 15 and Figure 16 further indicate that, of the 279 “ideal” long positions, the LR model accurately predicted 182 long positions, yielding a hit rate of 65.25%. The SVM model, similarly, predicted 180 accurate long positions out of 279, resulting in a hit rate of 64.5%. An “ideal” long position is defined here as a buy order that would yield a positive return, i.e., an increment in the stock price.

The summary of the hit rate of the LR- and SVM-based model is given below (Table 11):

The summary of the rate of the false position in each model can be described below:

Table 12 presents the rate of false long positions for each classical machine learning-based strategy, which was approximately 44.4%. Notably, this rate was generally lower than that observed for the deep learning models. Furthermore, the equity curves derived from the backtesting of the logistic regression (LR) and support vector machine (SVM)-based strategies are illustrated below:

Figure 17 presents the equity curves of the logistic regression (LR)-based signals and the buy-and-hold strategy over the trading period from February 2019 to February 2021, encompassing approximately 523 trading days. Upon review, the equity curve in Figure 17 illustrates that, from February 2019 to March 2020, the accumulative profit of the LR-based signals closely tracked that of the buy-and-hold strategy. However, during that period, the LR-based model consistently outperformed the buy-and-hold strategy. Specifically, the maximum accumulative profit of the LR-based strategy reached approximately 25%, which was on par with the buy-and-hold strategy. The situation changed markedly after the significant decline in IBM’s stock price in March 2020, which fell by more than 38%, from 146 to 90. Consequently, the equity curve of the buy-and-hold strategy plunged below −25%, while the LR-based strategy experienced a more moderate decline of around −12%. This indicates that, in terms of risk management, the LR-based strategy demonstrated a superior performance in mitigating losses during the downturn. However, the LR-based strategy failed to sustain this relative outperformance as the period progressed. By the end of the evaluation period, the LR-based strategy achieved an accumulative profit of 8.75%, corresponding to a compound annual growth rate (CAGR) of 4.24%, which, while positive, was significantly lower than the results of the deep learning-based strategies. The key performance comparisons between the two strategies are summarized as follows (Table 13):

Figure 18 presents the equity curves for the SVM-based strategy and the buy-and-hold strategy. In detail, up until February 2020, the SVM-based strategy exhibited a slightly higher level of profit compared to the buy-and-hold strategy; however, the difference between the two equity curves was not substantial. Around February 2020, a sharp increase in the equity curve of the buy-and-hold strategy was observed, reaching a peak with an accumulated profit of approximately 25%, whereas the SVM-based strategy’s accumulated profit plateaued at around 7%. However, the situation shifted dramatically following a significant drop in the stock price. Consequently, both equity curves—those of the SVM-based strategy and the buy-and-hold strategy—declined, with the former reaching a drawdown of approximately −30.80% and the latter −25.15%. Remarkably, all model-based strategies outperformed the buy-and-hold strategy over the observed period.

However, due to the upward movement in the stock price from June 2020 until the end of the period, the equity curve of the SVM-based strategy continued to rise, while the buy-and-hold strategy experienced more fluctuation. Ultimately, the equity curve of the SVM-based strategy achieved a cumulative profit of 6.15%, corresponding to a compound annual growth rate (CAGR) of 2.97%. This result was modest when compared to the performance of the other model-based strategies (Table 14).

In summary, this study provides an overview of algorithmic trading strategies derived from machine learning models. The results demonstrate that model-based strategies exhibit superior efficacy relative to the traditional buy-and-hold approach. Deeply trained models demonstrate superior capability in managing the downside risk of the equity curve. Consequently, these models can assist investors in making more informed decisions regarding long positions. However, there remains room for improvement in minimizing the detection of false signals. A summary of the performance of all strategies is provided in Table 15.

4. SAFE Machine Learning Models

Machine learning has emerged as a transformative force across numerous disciplines, revolutionizing fields such as healthcare, finance, and transportation. In healthcare, these technologies enable advanced predictive modeling for patient outcomes and treatment recommendations. In finance, they facilitate credit risk analysis and fraud detection, while in transportation, machine learning systems underpin autonomous vehicle operations, striving to enhance both safety and efficiency. Despite these achievements, critical concerns persist regarding the safety, reliability, and ethical implications of machine learning applications. A central challenge arises from the opaque nature of many machine learning models, colloquially termed the “black-box” problem. This opacity complicates the interpretation of model outputs and decision-making processes, raising significant issues such as unintended biases, erroneous predictions, and diminished user trust. As these technologies increasingly permeate high-stakes domains, addressing these limitations is imperative. Ensuring their reliability necessitates rigorous validation protocols and a robust theoretical framework to evaluate their ethical and technical integrity.

The SAFE machine learning framework is an evolving framework aimed at mitigating the risks associated with the deployment of AI systems, particularly in sensitive fields like finance. Machine learning models, known for their predictive power, also present challenges such as lack of transparency, fairness, and robustness, often referred to as the “black-box” nature of AI. According to Giudici et al. (2024) [18], the SAFE framework encompassing Sustainability, Accuracy, Fairness, and Explainability provides essential metrics to assess the trustworthiness of AI systems. This is especially crucial in finance, where models can inadvertently introduce biases, creating unfair advantages or risks. The Rank Graduation Box (RGB) metrics proposed by Babaei et al. (2025) [19] measure these aspects through model-agnostic tools, ensuring that AI systems in finance remain robust and fair under various stress scenarios. However, the inherent opacity of machine learning systems raises significant concerns in the financial sector. A critical issue is the inability to explain decisions made by complex models such as deep learning or random forests. This lack of explainability can lead to substantial risks in high-stakes environments like banking or insurance, where decisions affect financial stability and individual livelihoods. Regulatory bodies like the European Union have responded by introducing guidelines such as the AI Act (2021) [20], which mandates that AI systems be not only accurate but also interpretable and fair (European Commission, 2021 [20]).

The financial sector is particularly vulnerable to these risks, as AI-driven decisions often involve sensitive data and have far-reaching consequences. A failure to address fairness, for instance, can lead to discriminatory practices, especially in areas like lending or credit scoring. Giudici et al. (2023) [21] underscore that AI models in finance must be robust against extreme events, such as market crashes or cyberattacks, and ensure that biases do not disproportionately affect disadvantaged groups. This is where the Rank Graduation Fairness (RGF) metric comes into play, as it measures how well AI systems perform across different demographic groups, thus mitigating unfair outcomes. Moreover, Giudici and Raffinetti (2024) [22] highlight that explainability is another critical factor in ensuring the safety of machine learning models. For AI systems to be truly trustworthy, they must offer clear reasoning for their decisions, which is especially difficult for black-box models. The Rank Graduation Explainability (RGE) metric helps by offering a quantitative measure of how understandable a model’s decisions are, ensuring transparency without sacrificing accuracy. Finally, the development of safe machine learning systems is vital for the responsible use of AI in finance. With frameworks like SAFE and tools such as the Rank Graduation Box, it is possible to measure and control the risks associated with AI, ensuring that systems remain accurate, fair, and transparent.

4.1. Theoretical Background

The foundation of our proposal lies in the utilization of the Lorenz curve, along with the associated concepts of the dual Lorenz and the Concordance curve. These statistical tools are extensively employed to encapsulate and analyze the distribution of income and wealth.

Let $Y^{*}$ and $Y^{**}$ represent two statistical distributions, which could be continuous, ordinal, or binary variables, each defined over a dataset comprising n observations. These distributions serve as the basis for our comparative analysis.

The values of $Y^{*}$ can be arranged in non-decreasing order to construct the Lorenz curve, denoted as $L_{Y^{*}}$. For i = 1, 2, …, n the Lorenz curve is defined by the pairs $\left({\displaystyle \frac{i}{n}},{\displaystyle \frac{{\sum }_{j=1}^i{y}_{r_j^{*}}^{*}}{n{\overline{y}}^{*}}}\right)$, where $r_j^{*}$ indicates the non-decreasing ranks of $Y^{*}$, and ${\overline{y}}^{*}$ indicates the mean of $Y^{*}$.

Similarly, the values of $Y^{*}$ could alternatively be arranged in non-increasing order to construct the dual Lorenz curve, denoted as $L_{Y^{*}}^{\prime }$. The dual Lorenz curve is formally defined by the coordinate pairs $\left({\displaystyle \frac{i}{n}},{\displaystyle \frac{{\sum }_{j=1}^i{y}_{r_{n+1-j}^{*}}^{*}}{n{\overline{y}}^{*}}}\right)$, where $r_{n+1-j}^{*}$ corresponds to the ranks of $Y^{*}$.

The Concordance curve, denoted as C is formally constructed by ordering the $Y^{*}$ values according to the ranks of $Y^{**}$. Let $r_j^{**}$ represent the non-decreasing ranks of $Y^{**}$; for i = 1, 2, …, n, the Concordance curve is formally defined by the coordinate pairs $\left({\displaystyle \frac{i}{n}},{\displaystyle \frac{{\sum }_{j=1}^i{y}_{r_j^{**}}^{*}}{n{\overline{y}}^{*}}}\right)$.

There are potentially four distinct scenarios that can arise in the relationship between the Concordance curve, the Lorenz curve, and the dual Lorenz curve. These scenarios can be analyzed as follows:

• (a). When $r_i^{**}=r_i^{*}$ for all $i=1,\dots ,n$, the Concordance curve C coincides with the Lorenz curve, representing a state of full concordance between the two distributions.

• (b). When $r_i^{**}=r_{n+1-i}^{*}$ for all $i=1,\dots ,n$, the Concordance curve C aligns with the dual Lorenz curve, indicating a state of full discordance between the distributions.

• (c). When all $r_i^{**}$ values are identical for $i=1,\dots ,n$, the Concordance curve C coincides with the 45-degree line, signifying no association between the two distributions. In this scenario, as all $Y^{**}$ values are tied, the $Y^{*}$ values are replaced by their mean, ${\overline{y}}^{*}$.

• (d). More generally, the distance between the Concordance curve C and the Lorenz curve quantifies the degree to which the ranks of $Y^{**}$ diverge from those of $Y^{*}$, reflecting the extent of dissimilarity between the two distributions in terms of their ranked values.

Figure 19 illustrates that the Concordance curve C is consistently situated between the Lorenz curve $L_{Y^{*}}$ and the dual Lorenz curve $L_{Y^{*}}^{\prime }$. This observation can be utilized to construct a summary measure quantifying the distance between the Concordance curve and the two Lorenz curves, which will serve as the foundation for the SAFE risk metrics.

In order to normalize this distance, we can divide the area between the Concordance curve C and the dual Lorenz curve by its maximum value, which corresponds to the area between the Lorenz and dual Lorenz curves. This normalization guarantees that the metric is standardized and lies within the interval $(0,1)$. More formally, the measure is defined as:

(31)$\begin{array}{cc}\hfill RG& ={\displaystyle \frac{{\sum }_{i=1}^n\left\{\frac{1}{n{\overline{y}}^{*}}\left({\sum }_{j=1}^i{y}_{r_{n+1-j}^{*}}^{*}-{\sum }_{j=1}^i{y}_{r_j^{**}}^{*}\right)\right\}}{{\sum }_{i=1}^n\left\{\frac{1}{n{\overline{y}}^{*}}\left({\sum }_{j=1}^i{y}_{r_{n+1-j}^{*}}^{*}-{\sum }_{j=1}^i{y}_{r_j^{*}}^{*}\right)\right\}}}\hfill \\ & ={\displaystyle \frac{{\sum }_{i=1}^{n}i{y}_{r_i^{**}}^{*}-{\sum }_{i=1}^{n}i{y}_{r_{n+1-i}^{*}}^n}{{\sum }_{i=1}^{n}i{y}_{r_i^{*}}^{*}-{\sum }_{i=1}^{n}i{y}_{r_{n+1-i}^{*}}^{*}}}\hfill \end{array}$

The proposed RG measure serves as the foundational basis for the SAFE metrics described in the following.

We begin by addressing the concept of “Accuracy”. Accuracy necessitates that the output generated by an AI application aligns closely with the observed or anticipated outcomes. Considering a target variable Y and a set of K predictors, a machine learning model can be employed to produce predictions, denoted as $\widehat{Y}$. Subsequently, the metric accuracy can be derived formally by assigning $Y^{*}=Y$ and $Y^{**}=\widehat{Y}$.

Figure 19 can be interpreted in the context of accuracy by substituting $Y^{*}$ with Y and $Y^{**}$ with $\widehat{Y}$. The optimal scenario [a] arises when the predicted ranks of the response variable Y precisely match the observed ranks, resulting in the Concordance curve C perfectly coinciding with the Lorenz curve $L_Y$ (top-left graph). Conversely, the worst-case scenario [b] occurs when the predicted ranks align with the reversed observed ranks, causing the Concordance curve C to exactly overlap with the dual Lorenz curve $L_{Y^{*}}^{\prime }$ (top-right graph).

The scenario of no association [c] is observed when the model generates random predictions, leading the Concordance curve C to align with the 45-degree line (bottom-left graph). Finally, in the general scenario [d] (bottom-right graph), the Concordance curve C is positioned within the area bounded by the Lorenz curve $L_Y$ and its dual, $L_{Y^{*}}^{\prime }$

It can be demonstrated that the RGA metric satisfies $0\le \mathrm{RGA}\le 1$. A value of RGA = 1 indicates a perfectly concordant model, where the Concordance curve C coincides with the Lorenz curve $L_{Y^{*}}$.

Conversely, RGA = 0 corresponds to a perfectly discordant model, where the Concordance curve C overlaps with the dual Lorenz curve $L_{Y^{*}}^{\prime }$. For random predictions, RGA = 0.5 as the Concordance curve C aligns with the 45-degree line.

The RGA metric, initially proposed by Raffinetti (2023) [23] was designed as a measure of accuracy, independent of the nature of the response variable. Notably, in the case of a binary response, the RGA metric aligns with the Area Under the Curve (AUC) metric (see, for instance, Giudici & Raffinetti, 2024 [22]). However, this metric can be equivalently computed for ordinal and continuous response variables. In the remainder of this subsection, we demonstrate how the conceptual foundation of the RGA metric can be generalized to other measures, thereby establishing a unified framework for evaluating the compliance and risk of AI systems, grounded in the RG metric.

We now turn our attention to the concept of S: “Sustainability”, which is intrinsically linked to the robustness of AI systems. Robustness pertains to the resilience of AI applications in the presence of extreme data, whether originating internally or as a result of cyberattacks, both of which have the potential to distort the input data and, consequently, the generated outputs. The evaluation of robustness is commonly performed by quantifying an appropriate measure of distance between the predictions produced by the model under normal conditions and those generated when the input data are subject to (either intentional or unintentional) perturbations. The RG metric offers a particularly valuable distance measure in this context, as it relies on rank-based distances, which are inherently more robust to the presence of outliers and extreme data compared to value-based distances, such as the root-mean-square error.

To assess the robustness of AI applications, we propose the utilization of a “Rank Graduation Robustness” (RGR) measure. This metric is defined by specifying in Equation (31) $Y^{*}=\widehat{Y}$ and $Y^{**}={\widehat{Y}}_p$, where $\widehat{Y}$ represents the predicted values derived from the unperturbed data, and ${\widehat{Y}}_p$ denotes the predicted values obtained from the perturbed data.

As illustrated in Figure 19, maximum robustness [a] is achieved when the ranks of the predicted response values $\widehat{Y}$ perfectly correspond to the ranks of the predicted values derived from the perturbed data. In this scenario, the Concordance curve C coincides precisely with the Lorenz curve $L_{\widehat{Y}}$. Conversely, the worst-case scenario [b] arises when the ranks of the predicted response values $\widehat{Y}$ are inversely aligned with the ranks of the predicted values from the perturbed data. There, the Concordance curve C perfectly aligns with the dual Lorenz curve $L_{\widehat{Y}}^{\prime }$.

Case [c] arises when the Concordance curve C coincides with the 45-degree line, indicating that perturbations in the input data result in random predictions. The general case [d], on the other hand, occurs when the Concordance curve C lies within the region bounded by the Lorenz curve $L_{\widehat{Y}}$ and its dual, $L_{\widehat{Y}}^{\prime }$. It can be obviously demonstrated that the RGR measure is confined to the closed interval [0, 1]. Specifically, RGR = 1 corresponds to a fully robust model, where $C\equiv L_{\widehat{Y}}$. RGR = 0 represents a fully perturbed model, where $C\equiv L_{\widehat{Y}}^{\prime }$, and RGR = 0.5 indicates that the perturbations result in a model producing just random predictions.

We now focus on the measurement of “Explainability”, with the standard metrics in this domain being Shapley values and their variants, such as Shapley–Lorenz values. Although these metrics are commonly used, they are computationally intensive, and their calculation may become infeasible when a large number of input variables are involved.

In this paper, we propose a novel approach to address this limitation by adapting Equation (31) for the measurement of explainability. Specifically, we define $Y^{*}=\widehat{Y}$, and $Y^{**}={\widehat{Y}}^{-X_k}$, where $\widehat{Y}$ represents the predicted values from a full model that includes all k predictors, and ${\widehat{Y}}^{-X_k}$ denotes the predicted values from a reduced model, which excludes the kth predictor under evaluation. It is noteworthy that the greater the influence of a variable $X_k$ in explaining Y, the more pronounced the divergence between the ranks of the predicted values from the full model and those associated with the reduced model. Therefore, let $Y^{*}=\widehat{Y}$ and $Y^{**}={\widehat{Y}}^{-X_k}$; we then formally define RGE = 1-RG.

Interpreting Figure 19 within the context of explainability, it is observed that the case of minimal explainability [a] arises when the ranks of the predicted response values $\widehat{Y}$ align with the ranks of the predicted values ${\widehat{Y}}^{-X_k}$, resulting in the Concordance curve C perfectly coinciding with the Lorenz curve $L_{\widehat{Y}}$ (left graph). Conversely, the case of maximal explainability [b] occurs when the ranks of the predicted response values $\widehat{Y}$ inversely aligned with those of ${\widehat{Y}}^{-X_k}$, with the Concordance curve C perfectly overlapping the dual Lorenz curve $L_{\widehat{Y}}^{\prime }$. The scenario [c] is characterized by the Concordance curve C coinciding with the 45-degree line, indicating that the ${\widehat{Y}}^{-X_k}$ predictions are entirely random. Finally, the general case [d] occurs when the Concordance curve C resides within the region bounded by the Lorenz curve $L_{\widehat{Y}}$ and its dual, $L_{\widehat{Y}}^{\prime }$. It is established that $0\le \mathrm{RGE}\le 1$. Specifically, RGE = 1 if the kth predictor provides the maximal explanatory power. RGE = 0 if the kth predictor contributes nothing to the explanation of the response, and RGE = 0.5 if the model without the kth predictor corresponds to a random model.

Fairness is commonly assessed using parity measures, which compare the AI outputs generated for distinct population groups. This group-based comparison is unconditional on the values of other input variables, making it insufficient for identifying the underlying causes of unfairness. Moreover, such measures may give rise to paradoxes: the output may appear fair at a marginal level, but not when conditioned on a control variable. For instance, a credit score may be fair with respect to the borrower’s gender on an overall basis, yet not when conditioned on the borrower’s income level. To address the limitations of parity measures, we propose adapting Formula (31) to the fairness context, resulting in the development of the “Rank Graduation Fairness” (RGF) metric.

More specifically, given a set G of group variables, which are transformed into G binary variables. The metric fairness is computed by defining $Y^{*}=\widehat{Y}$, and $Y^{**}={\widehat{Y}}^{-X_g}$, where $\widehat{Y}$ represents the predicted values obtained using a full model that includes all G groups of variables, and ${\widehat{Y}}^{-X_g}$ denotes the predicted values derived from a reduced model that excludes the gth group of binary variables under evaluation.

We can now interpret Figure 19 within the context of the RGF measure. The scenario of maximum fairness [a] occurs when the ranks of the predicted response values $\widehat{Y}$ align with the ranks of the predicted values ${\widehat{Y}}^{-X_g}$, resulting in the Concordance curve C perfectly overlapping the Lorenz curve $L_{\widehat{Y}}$ (left graph). The condition of maximum unfairness [b] arises when the ranks of the predicted response values $\widehat{Y}$ are inversely aligned with those of ${\widehat{Y}}^{-X_g}$, leading to the Concordance curve C coinciding with the dual Lorenz curve $L_{\widehat{Y}}^{\prime }$. In case [c], the Concordance curve C coincides with the 45-degree line, meaning the ${\widehat{Y}}^{-X_g}$ correspond to a random model. In the intermediate scenario [d], the Concordance curve C lies within the region between the Lorenz curve $L_{\widehat{Y}}$ and its counterpart dual, $L_{\widehat{Y}}^{\prime }$. It could be clearly illustrated that $0\le \mathrm{RGF}\le 1$. Particularly, RGF = 1 corresponds to the case of maximum fairness, RGR = 0 corresponds to the case of maximum unfairness, and RGF = 0.5 occurs when the model excluding the gth group of variables corresponds to a random model.

We finally remark that for all the above metrics, statistical tests based on resampling are available, leveraging Hoeffding’s Theorem and utilizing the Jackknife method, to yield a test statistic.

4.2. Computational Algorithm

Let us assume $(X,y)$ represent the feature data and target data. The entire workflow, including training, testing, and applying the SAFE methodology, is outlined in the following pseudocode (Algorithm 1):

Algorithm 1 Workflow for training, testing, and applying the SAFE methodology

1:. Input: Feature data X, target data y, set of models P, SAFE metrics’ list SAFE_metrics.  2:. Output: Vectorized backtest results, SAFE metrics’ results, and combination statistics tests.  3:. Step 1: Split data  4:. $(X_{\mathrm{train}},X_{\mathrm{test}},y_{\mathrm{train}},y_{\mathrm{test}})\leftarrow \text{split\_data}(X,y,\text{test\_size}=0.2)$  5:. Steps 2–5: Train, test, backtest, compute SAFE metrics, and combinations  6:. for each model $m\in P$ do  7:.    $m.\mathrm{fit}(X_{\mathrm{train}},y_{\mathrm{train}})$ {Training phase}  8:.    $\mathrm{predictions}\left[m\right]\leftarrow m.\mathrm{predict}(X_{\mathrm{test}})$ {Testing phase}  9:.    $\text{backtest\_results}\left[m\right]\leftarrow \text{perform\_vectorized\_backtest}(m,X_{\mathrm{test}},y_{\mathrm{test}})$ {Backtest} 10:.    for each metric $\mathrm{metric}\in \text{SAFE\_metrics}$ do 11:.      $\text{safe\_metrics\_results}\left[m\right]\left[\mathrm{metric}\right]\leftarrow \text{compute\_safe\_metric}(m,\mathrm{metric},X_{\mathrm{test}},y_{\mathrm{test}})$ 12:.    end for 13:. end for 14:. Step 6: Compute combination statistics tests. 15:. $\mathrm{combinations}\leftarrow \text{generate\_all\_2\_combinations}(P)$ 16:. for each pair $(m_1,m_2)\in \mathrm{combinations}$ do 17:.    $\text{combination\_stats}\left[(m_1,m_2)\right]\leftarrow \text{compute\_statistics}(m_1,m_2,\widehat{Y},y_{\mathrm{test}})$ 18:. end for 19:. Return: $\{\text{predictions, backtest\_results, safe\_metrics\_results, combination\_stats}\}$

5. Results: SAFE Machine Learning Models

In this section, we applied the SAFE machine learning methods to our data. The compared models were developed utilizing historical data spanning both 10 and 20 years, respectively. In addition to the five models discussed, this section provide an in-depth analysis of the architecture of Convolutional Neural Networks (CNNs), which are widely regarded as a state-of-the-art framework in both deep learning and time-series analysis. It is worth highlighting that all models in this research, ranging from sophisticated deep learning algorithms to more traditional machine learning approaches, were constructed exclusively from the stock’s own time-series data. No external economic indicators or variables were incorporated into these models. This choice was deliberate, as the aim was to assess the stock’s historical performance to forecast future trends and, based on those predictions, make informed trading decisions. The focus on time-series data means that the models relied solely on past stock information to predict future behavior, emphasizing the intrinsic value of the stock’s own historical data. By excluding external economic factors, the analysis isolated the stock’s inherent patterns and trends. As a result, the models were more self-contained, relying on the stock’s own movements to drive predictions. This approach is particularly relevant in finance, where a stock-specific analysis can sometimes offer more precise insights than models that include broader, less directly relevant economic data. Although the overall methodology touches on multiple dimensions of AI model evaluation, including fairness, this section of the research placed particular emphasis on three main aspects: accuracy, sustainability (or robustness), and explainability. Accuracy is critical in ensuring that the models make reliable predictions based on the stock’s past performance. Sustainability, or robustness, assesses the models’ resilience to changes or disturbances in the input data, ensuring that predictions remain stable under varying conditions. Lastly, explainability is crucial for understanding how the models arrive at their predictions, offering transparency in decision-making processes.

The empirical evaluation of six models, namely, Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), support vector machine (SVM), and logistic regression (Logit), in terms of accuracy and robustness, underscored the pronounced superiority of deep learning methodologies over classical machine learning approaches. Specifically, the deep learning models (LSTM, GRU, RNN, and CNN) exhibited significantly higher accuracy compared to their classical counterparts (SVM and Logit). Using 20 years of historical data, the Long Short-Term Memory (LSTM) model achieved an accuracy of $0.9998$, the Gated Recurrent Unit (GRU) reached $0.9999$, the Recurrent Neural Network (RNN) attained $0.9996$, and the Convolutional Neural Network (CNN) recorded $0.99977$, demonstrating their exceptional precision in predictive performance. A similar trend was observed with models trained on 10 years of data, where the LSTM, GRU, RNN, and CNN achieved accuracies of $0.9994$, $0.9998$, $0.9992$, and $0.99973$, respectively. In contrast, when trained on 10 years of historical data, the classical models exhibited considerably lower accuracy, with the support vector machine (SVM) achieving $0.4935$ and logistic regression (Logit) attaining $0.5185$. With 20 years of historical data, the accuracy of these models increased slightly to $0.51377$ and $0.5477$, respectively. This significant discrepancy can be attributed to the ability of deep learning models to effectively capture complex, nonlinear relationships within the stock’s time-series data—patterns that classical models, such as SVM and Logit, are inherently limited in detecting. These findings align with the results of the backtesting analysis presented in the preceding section, wherein deep learning models consistently outperformed their classical counterparts. The superior accuracy demonstrated by the LSTM, GRU, and RNN models underscores their effectiveness in discerning the intricate, nonlinear dependencies inherent in stock price movements, Secondly, the robustness index further affirms the stability of the models when exposed to variations or perturbations within the data, with deep learning models once again surpassing classical machine learning approaches. The robustness scores for the deep learning models remained notably high when trained on 20 years of historical data, with the Long Short-Term Memory (LSTM) and Recurrent Neural Network (RNN) achieving scores of $0.8682$ and $0.8309$, respectively. The Gated Recurrent Unit (GRU) attained a score of $0.8369$, while the Convolutional Neural Network (CNN) achieved a particularly high score of $0.9472$. These results were consistent with those observed when training with a shorter period of 10 years, where the respective robustness scores were $0.8510$, $0.8633$, $0.8634$, and $0.9536$. In contrast, both classical models, support vector machine (SVM) and logistic regression (Logit), exhibited substantially lower robustness. When trained on 20 years of historical data, both models attained scores of $0.4653$ and $0.5241$, respectively. When trained with 10 years of data, the robustness scores for these models were $0.6400$ and $0.4500$, respectively. The observed similarity in patterns between accuracy and robustness further supports the notion that deep learning models not only achieve high accuracy but also retain their performance under suboptimal conditions, thereby reinforcing their superiority. The robustness of these models, particularly those trained with extensive data, can be primarily attributed to the application of regularization techniques, which are essential in mitigating overfitting. Regularization methods, such as $l2$ regularization (weight decay), dropout, and early stopping, function by imposing constraints or penalties on the model’s complexity. Consequently, deep learning models are better positioned to preserve both stability and accuracy when confronted with variations or noise in the input data. By reducing overfitting, regularization improves the model’s capacity to discern underlying, more generalizable patterns, thereby enhancing its robustness and reliability in practical, real-world settings.

To validate the above findings, statistical tests were conducted to compare the robustness of each model. At a $95\%$ confidence level, the results confirmed that the differences in robustness across the models were statistically significant. The p-values between models further illustrate these differences: LSTM–GRU ($2.67\times {10}^{-226}$), LSTM–RNN ($3.8\times {10}^{-198}$), LSTM–SVM $(0.0009)$, GRU–RNN $(0.015)$, GRU–SVM $(0.0346)$, CNN–LSTM ($3.82\times {10}^{-148}$), CNN–GRU ($1.16\times {10}^{-80}$), CNN–RNN ($3.82\times {10}^{-148}$), RNN–SVM $(0.015)$, SVM–Logit ($2.4\times {10}^{-5}$). These values indicate that statistically, the robustness of the deep learning models is significantly different from that of classical machine learning models, with deep learning showing much higher robustness scores.

We remark that, in the context of deploying AI models, explainability becomes a fundamental concern. The ability to understand and interpret how these models make decisions is not merely a technical necessity but a crucial factor in gaining and maintaining trust from users, traders, and stakeholders. To this aim, in the next figures, we compare the explainability of different machine learning models, for the IBM stock prices, in terms of their lagged values.

Figure 20, Figure 21, Figure 22, Figure 23, Figure 24 and Figure 25 depict the behavior of advanced deep models, in terms of explainability. They reveal that the models equally prioritized both historical and the most recent price data, when making predictions. This balanced approach is particularly valuable in trading environments, where current price information can provide more actionable insights compared to historical data alone. In contrast, classical machine learning models tended to focus more heavily on past data. This methodological difference can account for the observed superiority in accuracy and robustness of advanced deep learning models over traditional machine learning approaches. The advanced models’ ability to integrate and weigh both past and present information allows them to adapt more effectively to changing conditions, thereby enhancing their predictive performance and reliability in dynamic environments.

6. Concluding Remarks

This paper investigated the implications of trading predictions based on machine learning algorithms applied to IBM’s stock price data spanning from January 2000 to March 2021. Specifically, the models were trained using data from January 2000 to February 2019 and evaluated on the subsequent period. The algorithms analyzed included logistic regression, support vector machine, Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). Each model adhered rigorously to the comprehensive eight-step procedure outlined in the methodology section, with model-based strategies independently validated during the testing phase. The effectiveness of each strategy was assessed across four key metrics: cumulative profit, drawdown, hit rate, and false position rate. To facilitate comparison, the equity curves for each strategy were presented. Notably, the drawdown metric is central to risk management, as it reflects the maximum loss tolerated by each strategy in response to fluctuations in the underlying asset’s price.

The results presented in this study indicated that deep learning models outperformed classical machine learning models in both profit generation and risk management. Among the deep learning models, the GRU-based strategy demonstrated the most exceptional performance, achieving an accumulated profit of 26.71%, equivalent to a compound annual growth rate (CAGR) of 18.65%. The LSTM-based and RNN-based strategies also delivered favorable outcomes, with the LSTM-based strategy attaining an accumulated profit of 21.77%, corresponding to a CAGR of 10.50%, and the RNN-based strategy generating an accumulated profit of 14.30%, which equated to a CAGR of 6.00%. In contrast, the default buy-and-hold strategy realized only a modest gain of 0.6%, or a CAGR of 0.4%. Furthermore, the classical machine learning models, logistic regression (LR)-based and support vector machine (SVM)-based strategies, yielded accumulated profits of 8.80% (CAGR of 6.00%) and 6.15% (CAGR of 2.97%), respectively. Although these strategies performed better than the buy-and-hold strategy, their results were considerably more moderate compared to those of the deep learning models.

The paper also considered how to evaluate the reliability of machine learning models for trading, using the SAFE AI framework. In addition to evaluating model accuracy, the framework provides users, traders, and stakeholders with valuable insights on the “hidden dynamics” of deeply trained models, assessing their robustness, enhancing interpretability, and fostering greater transparency in their application.

In conclusion, the findings of this study provide a valuable contribution to the application of machine learning models in quantitative trading. The results indicate that despite areas for potential improvement, deep learning models along with a SAFE evaluation framework can offer a promising and effective approach to both profit generation and risk management in quantitative investment, outperforming traditional model-based strategies as well as the default buy-and-hold strategy.

Author Contributions

Writing—original draft, P.T.D. and P.G. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

No new data were created or analyzed in this study.

Acknowledgments

The paper is a result of a close collaboration between the two authors, especially in contextual design. While D.P.T. applied the models and drafted the paper, P.G. supervised and revised the work.

Conflicts of Interest

There are no interests to declare.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. The pseudo-cross-validation of sequential tabular form.

Figure 2. The unfolding architecture of the Recurrent Neural Network.

Figure 3. The unfolding architecture of the Long Short-Term Memory.

Figure 4. The unfolding architecture of the Gated Recurrent Unit.

Figure 5. The original stock price of IBM from January 2000 to March 2021.

Figure 6. The original stock price of IBM and its predicted value by an RNN.

Figure 7. The original stock price of IBM and its prediction by LSTM.

Figure 8. The original stock price of IBM and its prediction by a GRU.

Figure 9. The long position from the RNN-based strategy.

Figure 10. The long position from the LSTM-based strategy.

Figure 11. The long position from the GRU-based strategy.

Figure 12. Equity curve of the buy-and-hold and RNN-based strategies.

Figure 13. The equity curve of the buy-and-hold and LSTM-based strategies.

Figure 14. The equity curve of the buy-and-hold and GRU-based strategies.

Figure 15. The long position by the logistic regression-based strategy.

Figure 16. The long position from the SVM-based strategy.

Figure 17. The equity curve of the buy-and-hold and logistic regression-based strategies.

Figure 18. The equity curve of the buy-and-hold and SVM-based strategies.

Figure 19. Possible four case of Lorenz, dual Lorenz and Concordance curve.

Figure 20. Features’ explanation of LSTM model.

Figure 21. Features’ explanation of RNN model.

Figure 22. Features’ explanation of GRU model.

Figure 23. Features’ explanation of CNN model.

Figure 24. Features’ explanation of SVM model.

Figure 25. Features’ explanation of logistic regression model.

Table 1

The pseudo-supervised learning from converted time series.

	Input		Output
1	2	3	4
2	3	4	5
…	…	…	…
7	8	9	10

Source: Authors.

Table 2

The statistical summary of IBM’s stock prices during the whole period.

Statistics	Value	Statistics	Value
Mean	$96.50$	1st quantile	$62.50$
Standard deviation	$34.10$	2nd quantile	$91.70$
Min	$35.10$	3rd quantile	$127.60$
Max	159.50	4th quantile	$159.50$

Source: Author.

Table 3

The statistical summary of IBM’s stock prices during the training dataset.

Statistics	Value	Statistics	Value
Mean	$93.50$	1st quantile	$60.75$
Standard deviation	$34.70$	2nd quantile	$84.05$
Min	$35.10$	3rd quantile	$128.02$
Max	159.50	4th quantile	$159.50$

Source: Author.

Table 4

The statistical summary of IBM’s stock prices during the testing dataset.

Statistics	Value	Statistics	Value
Mean	$112.92$	1st quantile	$118.95$
Standard deviation	$8.05$	2nd quantile	$123.60$
Min	$88.78$	3rd quantile	$127.01$
Max	147.15	4th quantile	$147.15$

Source: Author.

Table 5

The root-mean-square error of the forecast of IBM’s stock price using an RNN, LSTM, and GRU model.

Model	RMSE
RNN	$2.40$
LSTM	$2.34$
GRU	$2.36$

Source: Author.

Table 6

The hit rate of the RNN-, LSTM-, and GRU-based strategies.

Strategy	The Hit Rate
RNN-based strategy	$70.0\%$
LSTM-based strategy	$68.5\%$
GRU-based strategy	$96.5\%$

Source: Authors.

Table 7

The rate of false positions of the RNN-, LSTM-, GRU-based strategies.

Strategy	False Position Rate
RNN-based strategy	$73.60\%$
LSTM-based strategy	$70.70\%$
GRU-based strategy	$94.20\%$

Source: Authors.

Table 8

Key summary of the RNN-based and buy-and-hold strategies.

Strategy	Accumulated Profit	Drawdown
RNN-based strategy	$14.30\%(6.00\%)$	−18.05%
Buy-and-Hold strategy	$0.60\%(0.40\%)$	−25.15%

Source: Authors. Note: CAGR in parenthesis.

Table 9

Key summary of the LSTM-based and buy-and-hold strategies.

Strategy	Accumulated Profit	Drawdown
LSTM-based strategy	$21.77\%(10.50\%)$	−13.50%
Buy-and-hold strategy	$0.60\%(0.40\%)$	−25.15%

Source: Authors. Note: CAGR in parenthesis.

Table 10

Key summary of the GRU-based and buy-and-hold strategies.

Strategy	Accumulated Profit	Drawdown
GRU-based strategy	$26.71\%(18.65\%)$	−11.58%
Buy-and-hold strategy	$0.60\%(0.40\%)$	−25.15%

Source: Author. Note: CAGR in parenthesis.

Table 11

The hit rate of the LR- and SVM-based strategies.

Strategy	The Hit Rate
LR-based strategy	$62.5\%$
SVM-based strategy	$64.5\%$

Source: Author.

Table 12

The rate of false position of the LR- and SVM-based strategies.

Strategy	False Position Rate
LR-based strategy	$44.40\%$
SVM-based strategy	$44.40\%$

Source: Authors.

Table 13

Key summary of the LR-based and buy-and-hold strategy.

Strategy	Accumulated Profit	Drawdown
LR-based strategy	$8.80\%(6.00\%)$	−18.05%
Buy-and-hold strategy	$0.60\%(0.40\%)$	−25.15%

Source: Authors. Note: CAGR in parenthesis.

Table 14

Key summary of the SVM-based and buy-and-hold strategy.

Strategy	Accumulated Profit	Drawdown
SVM-based strategy	$6.15\%(2.97\%)$	−30.08%
Buy-and-Hold strategy	$0.60\%(0.40\%)$	−25.15%

Source: Authors. Note: CAGR in parenthesis.

Table 15

Key summaries of all trading strategies.

Strategy	Accumulated Profit	Drawdown
SVM-based strategy	$6.15\%(2.97\%)$	−30.08%
LR-based strategy	$8.80\%(6.00\%)$	−18.05%
RNN-based strategy	$14.30\%(6.00\%)$	−18.05%
LSTM-based strategy	$21.77\%(10.50\%)$	−13.50%
GRU-based strategy	$26.71\%(18.65\%)$	−11.58%
Buy-and-hold strategy	$0.60\%(0.40\%)$	−25.15%

Source: Authors. Note: CAGR in parenthesis.

References

1. Hu, Y.; Liu, X.; Zhang, L. Application of deep learning in financial markets: A review. J. Financ. Data Sci.; 2020; 6, pp. 107-127.

2. Hiransha, M.; Gopalakrishnan, E.A.; Menon, V.K.; Soman, K.P. NSE stock market prediction using deep-learning models. Procedia Comput. Sci.; 2018; 132, pp. 1351-1362.

3. Shalev, S.S.; Ben, D.S. Understanding Machine Learning From Theory to Algorithms; Cambridge University Press: New York, NY, USA, 2014.

4. Matthew, F.D.; Igor, H.; Paul, B. Machine Learning in Finance From Theory to Practice; Springer: Cham, Switzerland, 2020.

5. John, S.T.; Peter, L.B.; Robert, C.W.; Martin, A. Citation: Structural Risk Minimization over Data-Dependent Hierarchies. Neural Comput. Learn.; 1996; 8556, pp. 18-26.

6. Fischer, T.; Krauss, C. Deep learning with long short-term memory networks for financial market predictions. Eur. J. Oper. Res.; 2018; 270, pp. 654-669. [DOI: https://dx.doi.org/10.1016/j.ejor.2017.11.054]

7. Paul, J.W. Citation: Generalization of backpropagation with application to a recurrent gas market model. Neural Netw.; 1988; 1, pp. 339-356.

8. Zaiyoung, T.; Chrys, D.A.; Paul, A.F. Citation: Time series forecasting using neural networks vs box-jenkins methodology. Simulation; 1991; 57, pp. 303-310. [DOI: https://dx.doi.org/10.1177/003754979105700508]

9. Julian, F.; Chris, C. Citation: Time series forecasting with neural networks: A comparative study using the air line data. J. R. Stat. Soc. Ser. C (Appl. Stat.); 1998; 47, pp. 231-250.

10. Zhang, G.P.; Min, Q. Citation: Neural network forecasting for seasonal and trend time series. Eur. J. Oper. Res.; 2005; 160, pp. 501-514. [DOI: https://dx.doi.org/10.1016/j.ejor.2003.08.037]

11. Sepp, H.; Jurgen, S. Citation: Long Short-Term Memory. Neural Comput.; 1997; 9, pp. 1735-1780.

12. Guoqiang, Z.; Patuwo, B.E.; Michael, Y.H. Citation: Forecasting with artificial neural networks: The state of the art. Int. J. Forecast.; 1998; 14, pp. 35-62.

13. Aurélien, G. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow; O’Reilly Media: Sebastopol, OT, Canada, 2019.

14. Chollet, F. Deep Learning with Python; Manning Publications: New York, NY, USA, 2018.

15. Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L. et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Adv. Neural Inf. Process. Syst.; 2019; 32, pp. 8026-8037.

16. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016.

17. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature; 2015; 521, pp. 436-444. [DOI: https://dx.doi.org/10.1038/nature14539] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26017442]

18. Giudici, P.; Centurelli, M.; Turchetta, S. Artificial intelligence risk measurement. Expert Syst. Appl.; 2024; 235, 121220. [DOI: https://dx.doi.org/10.1016/j.eswa.2023.121220]

19. Babaei, G.; Giudici, P.; Raffinetti, E. A rank graduation Box for SAFE Artificial intelligence. Expert Syst. Appl.; 2025; 59, 125239. [DOI: https://dx.doi.org/10.1016/j.eswa.2024.125239]

20. European Commission. Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonized Rules on Artificial Intelligence (Artificial Intelligence Act) Andamending Certain (Union Legislative Acts); European Commission: Brussels, Belgium, 2021.

21. Giudici, P.; Raffinetti, E. SAFE artificial intelligence in finance. Financ. Res. Lett.; 2023; 13, 104088. [DOI: https://dx.doi.org/10.1016/j.frl.2023.104088]

22. Giudici, P.; Raffinetti, E. RGA: A unified approach of predictive accuracy. Adv. Data Anal. Appl.; 2024; in press [DOI: https://dx.doi.org/10.1007/s11634-023-00574-2]

23. Raffinetti, E. A rank graduation accuracy measure to mitigate artificial intellgence risks. Qual. Quant.; 2023; 57, pp. 131-150. [DOI: https://dx.doi.org/10.1007/s11135-023-01613-y]