Criticality Assessment Method for Automated Driving Systems by Introducing Fictive Vehicles and Variable Criticality Thresholds

Abstract

The safety approval and assessment of automated driving systems (ADS) are becoming sophisticated and challenging tasks. Because the number of traffic scenarios is vast, it is essential to assess their criticality and extract the ones that present a safety risk. In this paper, we are proposing a novel method based on the time-to-react (TTR) measurement, which has advantages in considering avoidance possibilities. The method incorporates the concept of fictive vehicles and variable criticality thresholds (VCTs) to assess the overall scenario’s criticality. By introducing variable thresholds, a criticality scale is defined and used for criticality calculation. Based on this scale, the presented method determines the criticality of the lanes adjacent to the ego vehicle. This is performed by placing fictive vehicles in the adjacent lanes, which represent copies of the ego. The effectiveness of the method is demonstrated in two highway scenarios, with and without trailing vehicles. Results show different criticality for the two scenarios. The overall criticality of the scenario with trailing vehicles is higher due to the decrease in avoidance possibilities for the ego vehicle.

1. Introduction

The assessment of ADS is essential for the approval and testing procedures. Due to a high scenario space and the complexity of ADS that needs to be tested for safety approval, scenario-based methods are promising and widely used approaches in the field of research. Recent studies, for example, [1,2], are focused on simulation-based testing and the generation of scenarios to reduce the testing effort for ADS. The reduction is achieved by identifying relevant safety-critical scenarios (SCS) using a variety of safety metrics and scenario-generation methods. A comprehensive review of current studies related to scenario-based approaches is presented in [3]. A scenario in the context of this work represents a series of actions occurring over a period of time, as defined in [4]. Other similar definitions are introduced and can be found, for example, in [5,6]. After extracting scenarios from a synthesized or real dataset, the scenarios are analyzed and assessed on their criticality. There are different definitions of what the criticality of a certain situation means. According to [7], accident probability combined with severity defines the criticality of a certain situation. The probability which is used here takes into account the avoidance of a particular situation. This approach enables a better interpretation of a specific situation. To calculate accurate avoidance possibilities, environment parameters, which are composed of static and dynamic objects, should be considered for evaluating the criticality of the situation. The available probabilistic methods, such as those in [8,9,10], calculate possible evasive maneuvers or accident probabilities; based on these, they estimate whether a situation is evasive, critical, or fulfills safety criteria. This consideration is advantageous on the one hand because it takes into account a large number of possibilities and the probability or feasibility of these possibilities in the calculation. On the other hand, due to complex solutions to mathematical problems, real-time capabilities and unique solutions are only sometimes given. To overcome such issues, we introduce a novel assessment method that introduces fictive ego vehicles on the left and right lanes adjacent to the tested ego vehicle. These fictive ego vehicles represent the exact copy of the ego vehicle, i.e., the vehicle configuration and the automated driving functions are the same. In this constellation, the ego vehicle and the neighboring fictive ego vehicles are assessed individually in the respective lane according to the introduced criticality scale. Combining the criticalities of the ego and fictive vehicle, an overall criticality of the situation can be calculated. To calculate the criticality, we take the avoidability of a collision into account using the TTR metric. The TTR metric consists of the time-to-brake (TTB) and time-to-steer (TTS) measurements, which represent the times left to avoid a critical situation by braking or steering, respectively. For SCS assessment, usually, the fixed metric thresholds are defined. However, metrics often depend on the kinematic relations between the ego and surrounding objects, such as distances and velocities. Although time-based metrics capture the influence of both distance and velocities, thresholds remain changeable depending on the object states. To overcome this issue with TTR, we are introducing variable criticality thresholds (VCTs) that determine a criticality scale. Finally, combining the concept of fictive ego vehicles, the TTR, and the VCTs, we derived the novel assessment method presented in this work. In the first Section 2, we present a detailed analysis of safety metrics used in scenario-based approaches for the assessment of SCS and a detailed description of the usage of the TTR. Using the TTR the criticality definition and the assessment method are presented in Section 3. As the last step of the work, we demonstrate the effectiveness of the presented method based on a safety-relevant test example.

2. Safety Metrics

According to [11], safety metrics are classified into temporal-, distance- and deceleration-based safety metrics. In [12], additional categorizations are introduced: the statistics-based, potential-field-based, and unexpected-driving-behavior-based metrics. These reviews show that temporal safety metrics are often and widely used to assess scenarios. The leading representative that is often mentioned in the context of the criticality of scenarios is the TTC. The main advantage of the TTC lies in its simplicity and efficiency for rear-end scenarios. The main disadvantages of the TTC criteria are the insensitivity to lateral collision risks and the fact that it cannot reflect the potential risk when the relative speed between two vehicles is zero. This implies that a possible intervention of a driver in safety-critical situations is not taken into account by the TTC. Therefore, various adjustments and adaptations of the classic TTC time were made; for example, a modified time-to-collision (MTTC) measure was introduced in [13]. Compared with the conventional TTC, the MTTC considers relative acceleration and relative velocity to improve collision prediction. In [14], the worst-time-to-collision (WTTC) metric was introduced. The WTTC considers the influence of vehicle dynamics and avoidance possibilities of the ego vehicle with the worst-case assumption regarding vehicle behavior. Another approach based on the TTC is presented in [15], where a time-exposed TTC (TET) and the time-integrated TTC (TIT) are introduced. These two metrics are extensions of the TTC and take the whole trajectory of vehicles over space and time into account. In [16], the TTR was introduced for the development and evaluation of forward-collision-avoidance systems, which consider a TTS, a TTS, and a TTK. In general, these reaction times determine the remaining time for successful collision avoidance by braking, steering, or acceleration. A similar approach is described in [17]; here, in addition to the TTS, a requested acceleration is introduced and used for the assessment of certain scenarios for ADS. The main advantages of these approaches compared with conventional temporal metrics such as the TTC and extensions of the TTC [18,19] are in consideration of avoidance possibilities and kinematic behavior of vehicles. The later the driver reacts successfully to an impending collision, the more the system is forced to exploit the vehicle’s physical limits. Using the TTR as a safety metric offers the possibility to interpret and define the criticality in a wide range of safety-relevant scenarios. The present work represents a continuation of previous studies of [16,17,20], where longitudinal and lateral kinematics are taken into account for a criticality assessment of ADS.

Time-to-react is defined as TTR = max{TTB, TTS}, where TTB and TTS are the time-to-brake and time-to-steer metrics, respectively. They represent time reserves to the last point to brake (LPTB) and last point to steer (LPTS) [21]. The points at which the ego has to perform braking or steering to avoid a collision with a vehicle in front, also called a target vehicle. For practical reasons, the times at which TTB and TTS are equal to 0 will be denoted as ${\tau }_{\mathrm{b}}$ and ${\tau }_{\mathrm{s}}$, respectively. In their general and simplest forms, they are defined by (1) and (2). The remaining distances between the ego and the target vehicle at ${\tau }_{\mathrm{b}}$ and ${\tau }_{\mathrm{s}}$ are defined as $d_{\mathrm{b}}$ and $d_{\mathrm{s}}$ by (3) and (4), respectively. The $d_{\mathrm{b}}$ is the distance that ego travels while reducing its velocity ($v_{\mathrm{ego}}$) to the target’s velocity $v_{\mathrm{t}}$ by applying the maximum braking $a_{\min ,\mathrm{x}}$ [21]. It is worth mentioning that $a_{\min ,\mathrm{x}}$ stands for the maximum longitudinal deceleration, wherein we respect the positive and negative signs of acceleration in the longitudinal direction. Here, $d_{\mathrm{s}}$ is calculated as equivalent to the evasion time $t_{\mathrm{ev}}$ (5) needed to change a lane with maximum lateral acceleration, $a_{\max ,\mathrm{y}}$ [17,21]. Unlike the longitudinal direction, in the lateral direction, we do not consider negative signs; therefore, $a_{\max ,\mathrm{y}}$ denotes the maximum acceleration, regardless of direction. These temporal and spatial relations are depicted in Figure 1, while all the parameters used to calculate the metrics and their thresholds are summarized in Table 1. Figure 1 illustrates that ${\tau }_{\mathrm{s}}$ occurs at a later point in time than ${\tau }_{\mathrm{b}}$, which indicates the steering as the last emergency intervention to avoid a collision; however, this is not always the case. Figure 2 shows the dependency of the remaining distances $d_{\mathrm{b}}$ and $d_{\mathrm{s}}$ on the relative velocity between the ego vehicle and target vehicle. Here, braking leaves more time to avoid a collision at low velocities, but at higher relative velocities, steering would be more appropriate.

(1)$\begin{array}{c}\hfill {\tau }_{\mathrm{b}}=\frac{d_0-d_{\mathrm{b}}}{v_{\mathrm{ego}}-v_{\mathrm{t}}}\end{array}$

(2)$\begin{array}{c}\hfill {\tau }_{\mathrm{s}}=\frac{d_0-d_{\mathrm{s}}}{v_{\mathrm{ego}}-v_{\mathrm{t}}}\end{array}$

(3)$\begin{array}{c}\hfill d_{\mathrm{b}}=-\frac{v_{\mathrm{rel}}^2}{2a_{\min ,\mathrm{x}}}\end{array}$

(4)$\begin{array}{c}\hfill d_{\mathrm{s}}=\sqrt{\frac{2d_{\mathrm{y}}}{a_{\max ,\mathrm{y}}}}{v}_{\mathrm{rel}}\end{array}$

(5)$\begin{array}{c}\hfill t_{\mathrm{ev}}=\sqrt{\frac{2d_{\mathrm{y}}}{a_{\max ,\mathrm{y}}}}\end{array}$

Figure 1

Visualization of steering and braking possibilities.

[Figure omitted. See PDF]

Table 1

Parameters used for safety metrics and thresholds.

State parameters	Description
$v_{\mathrm{ego}}$	Velocity of the ego vehicle
$v_{\mathrm{t}}$	Velocity of the target vehicle
$v_{\mathrm{rel}}$	Relative velocity between the ego vehicle and the target vehicle in front
$a_{\mathrm{ego},\mathrm{x}}$	Current longitudinal acceleration of the ego vehicle
$a_{\min ,\mathrm{x}}$	Maximum longitudinal deceleration
$a_{\max ,\mathrm{y}}$	Maximum lateral acceleration
$a_{\mathrm{req},\mathrm{x}}$	Required longitudinal deceleration of the ego vehicle
$a_{\mathrm{req},\mathrm{y}}$	Required lateral deceleration of the ego vehicle
Time parameters
${\tau }_{\mathrm{b}}$	Time to reach LPTB (TTB $=0$)
${\tau }_{\mathrm{s}}$	Time to reach LPTS (TTS $=0$)
${\tau }_{\mathrm{b},\mathrm{th}}$	Threshold for the ${\tau }_{\mathrm{b}}$
${\tau }_{\mathrm{s},\mathrm{th}}$	Threshold for the ${\tau }_{\mathrm{s}}$
$t_{\mathsf{\rho }}$	Response time of the ego vehicle
$t_{\mathrm{ev}}$	Evasion time needed to change a lane
Distance parameters
$d_0$	Initial distance between the ego and target vehicle at the point t=0
$d_{\mathrm{b}}$	Distance to target vehicle at ${\tau }_{\mathrm{b}}$
$d_{\mathrm{s}}$	Distance to target vehicle at ${\tau }_{\mathrm{s}}$
$d_{\mathrm{y}}$	Lateral evasion distance
$d_{\mathrm{b},\min }$	Minimum safety distance for braking maneuver
$d_{\mathrm{s},\min }$	Minimum safety distance for steering maneuver
Remaining parameters
g	Gravitational constant
$\mu$	Road friction coefficient

Figure 2

Remaining distance of an ego vehicle to a target vehicle for avoiding a rear-end collision in longitudinal traffic for a pure braking maneuver and a pure steering maneuver.

[Figure omitted. See PDF]

The previously defined relations in (1)–(4) consider only the constant velocities of the ego and target vehicles representing only a special case and resulting in a reduced precision in the presence of variable velocities. To increase the prediction precision while keeping it feasible regarding its complexity, a straight road and constant velocity $v_{\mathrm{t}}$ is assumed, while the ego vehicle keeps constant longitudinal acceleration $a_{\mathrm{ego},\mathrm{x}}$ up to the point where evasive braking or evasive steering actions must be executed. Furthermore, considering ideal conditions, the possible maximum braking and lateral acceleration are $a_{\min ,\mathrm{x}}=-\mu g$ and $a_{\max ,\mathrm{y}}=\mu g$, respectively, where $\mu$ represents the maximum available tire road friction. Considering such steady states, the remaining distances $d_{\mathrm{b}}$ and $d_{\mathrm{s}}$ are given by (6) and (7):

(6)$\begin{array}{c}\hfill d_{\mathrm{b}}=-\frac{{(v_{\mathrm{rel}}+a_{\mathrm{ego},\mathrm{x}}{\tau }_{\mathrm{b}})}^2}{2a_{\min ,\mathrm{x}}}\end{array}$

(7)$\begin{array}{c}\hfill d_{\mathrm{s}}=\sqrt{\frac{2d_{\mathrm{y}}}{a_{\max ,\mathrm{y}}}}(v_{\mathrm{rel}}+a_{\mathrm{ego},\mathrm{x}}{\tau }_{\mathrm{s}}),\end{array}$

while ${\tau }_{\mathrm{b}}$ and ${\tau }_{\mathrm{s}}$ are determined by solving the following two quadratic equations.

(8)$\begin{array}{c}\hfill \frac{1}{2}\left(a_{\mathrm{ego},\mathrm{x}}-\frac{a_{\mathrm{ego},\mathrm{x}}^2}{a_{\min ,\mathrm{x}}}\right){\tau }_b^2+\left(v_{\mathrm{rel}}-\frac{v_{\mathrm{rel}}{a}_{\mathrm{ego},\mathrm{x}}}{a_{\min ,\mathrm{x}}}\right){\tau }_b-d_0-\frac{v_{\mathrm{rel}}^2}{2a_{\min ,\mathrm{x}}}=0\end{array}$

(9)$\begin{array}{c}\hfill \frac{1}{2}{a}_{\mathrm{ego},\mathrm{x}}{\tau }_{\mathrm{s}}^2+\left(v_{\mathrm{rel}}+a_{\mathrm{ego},\mathrm{x}}\sqrt{2\frac{d_{\mathrm{y}}}{a_{\max ,\mathrm{y}}}}\right){\tau }_{\mathrm{s}}-d_0+v_{\mathrm{rel}}\sqrt{2\frac{d_{\mathrm{y}}}{a_{\max ,\mathrm{y}}}}=0\end{array}$

The metrics described by (6)–(9) are used below to establish the thresholds and assess criticality.

3. Assessment Method

3.1. Concept of Fictive Vehicles

Figure 3 shows the novel concept of fictive vehicles on vehicle configurations taken from PEGASUS [22,23], where nine vehicles from the immediate surrounding of the ego vehicle are considered as safety relevant for scenario development or analysis. Furthermore, the vehicles ${\mathrm{EGO}}_{\mathrm{f}}^{\mathrm{r}}$ and ${\mathrm{EGO}}_{\mathrm{f}}^{\mathrm{l}}$ represent fictive duplicates of the ego vehicle and are needed for the assessment method described below. However, there is one condition under which a fictive vehicle can be placed in adjacent lanes—the spaces at the neighboring lanes next to the ego must be unobstructed. For better interpretability, the environment around the ego vehicle is divided into trailing traffic, leading traffic, and the part adjacent to the ego vehicle, which consists of fictive vehicles.

3.2. Definition of the Criticality

The criticality definition consists of two parts. The first part deals with the definition of variable criticality threshold (VCT) based on the metrics introduced in Section 2. The second part deals with the quantification of the criticality based on the VCT.

3.2.1. Definition of Variable Criticality Thresholds

To determine the VCT, we build upon the required accelerations $a_{\mathrm{req},\mathrm{x}}$ and $a_{\mathrm{req},\mathrm{y}}$ for braking and steering, respectively, as well as on minimum safety distances for braking ($d_{\mathrm{b},\min }$) and steering ($d_{\mathrm{s},\min }$). Distances are calculated based on the mathematical model of responsibility-sensitive safety (RSS) [24], that accounts for the worst-case scenario, including the response time of the ego and the maximum theoretical braking to the standstill of a target vehicle. However, we introduced slight modifications for calculating $d_{\mathrm{b},\min }$, which are shown in (10). Unlike the RSS, we do not consider the maximum longitudinal acceleration of the ego during the response time; therefore, we use the current ego’s acceleration instead. Another adjustment is regarding the ego’s deceleration after the response time, where we use $a_{\mathrm{req},\mathrm{x}}$, which denotes braking but is not limited only to small values.

(10)$\begin{array}{c}\hfill d_{\mathrm{b},\min }=v_{\mathrm{ego}}{t}_{\mathsf{\rho }}+\frac{1}{2}{a}_{\mathrm{ego},\mathrm{x}}{t}_{\mathsf{\rho }}^2-\frac{{(v_{\mathrm{ego}}+t_{\mathsf{\rho }}{a}_{\max ,\mathrm{acc}})}^2}{2a_{\mathrm{req},\mathrm{x}}}+\frac{v_{\mathrm{t}}^2}{2a_{\min ,\mathrm{x}}}\end{array}$

The safety distance for steering, $d_{\mathrm{s},\min }$, is determined in a similar manner (11). The only difference is that the braking distance of the ego is substituted with the required distance to perform a lane change.

(11)$\begin{array}{c}\hfill d_{\mathrm{s},\min }=v_{\mathrm{ego}}{t}_{\mathsf{\rho }}+\frac{1}{2}{a}_{\mathrm{ego},\mathrm{x}}{t}_{\mathsf{\rho }}^2+\sqrt{2\frac{d_{\mathrm{y}}}{a_{\mathrm{req},\mathrm{y}}}}(v_{\mathrm{ego}}+a_{\mathrm{ego},\mathrm{x}}{t}_{\mathsf{\rho }})+\frac{v_{\mathrm{t}}^2}{2a_{\min ,\mathrm{x}}}\end{array}$

Substituting the initial distance $d_0$ in (8) and (9) with the minimum distances $d_{\mathrm{b},\min }$ and $d_{\mathrm{s},\min }$, the threshold values for TTB (12) and TTS (13) can be determined. In this way, the safety distance is transferred into the time domain and steered by $a_{\mathrm{req},\mathrm{x}}$ and $a_{\mathrm{req},\mathrm{y}}$. With $a_{\mathrm{req},\mathrm{x}}$ and $a_{\mathrm{req},\mathrm{y}}$, it is possible to interpret the thresholds and link them to objective or subjective safety and comfort investigations. Now, at a certain ${\tau }_{\mathrm{b}}$, when $d_0=d_{\mathrm{b},\min }$, we can claim—if there is sudden maximum braking of the target vehicle—that the ego will need to brake with $a_{\mathrm{req},\mathrm{x}}$ to avoid a collision; the same applies for steering.

(12)$\begin{array}{c}\hfill \frac{1}{2}\left(a_{\mathrm{ego},\mathrm{x}}-\frac{a_{\mathrm{ego},\mathrm{x}}^2}{a_{\min ,\mathrm{x}}}\right){\tau }_{\mathrm{b},\mathrm{th}}^2+\left(v_{\mathrm{rel}}-\frac{v_{\mathrm{rel}}{a}_{\mathrm{ego},\mathrm{x}}}{a_{\min ,\mathrm{x}}}\right){\tau }_{\mathrm{b},\mathrm{th}}-d_{\mathrm{b},\min }-\frac{v_{\mathrm{rel}}^2}{2a_{\min ,\mathrm{x}}}=0\end{array}$

(13)$\begin{array}{c}\hfill \frac{1}{2}{a}_{\mathrm{ego},\mathrm{x}}{\tau }_{\mathrm{s},\mathrm{th}}^2+\left(v_{\mathrm{rel}}+a_{\mathrm{ego},\mathrm{x}}\sqrt{2\frac{d_{\mathrm{y}}}{a_{\max ,\mathrm{y}}}}\right){\tau }_{\mathrm{s},\mathrm{th}}-d_{\mathrm{s},\min }+v_{\mathrm{rel}}\sqrt{2\frac{d_{\mathrm{y}}}{a_{\max ,\mathrm{y}}}}=0\end{array}$

In the present work, different safety thresholds were defined according to the different criticality values determined by the required accelerations $a_{\mathrm{req},\mathrm{x}}$ and $a_{\mathrm{req},\mathrm{y}}$, which were obtained from the literature and are provided in Table 2. Driving comfort was taken as the criterion to determine the lowest deceleration threshold level. Several studies, such as [25,26], have correlated the subjective evaluation of driving comfort to objective values, and one of the most significant is acceleration. Taking the relevant literature into account, $-2{\mathrm{m}/\mathrm{s}}^2$ was considered as the lowest threshold for longitudinal acceleration. The other extreme is the highest threshold, which corresponds to emergency braking and depends on the current friction coefficient. In this study, we defined the emergency level by $-\mu g$. Inbetween these limits and regarding the criticality levels defined below, we defined two intermediate levels based on the partial braking of adaptive cruise control (ACC) [27]. Therefore, the lower intermediate level is $-3{\mathrm{m}/\mathrm{s}}^2$ and the higher intermediate level is $-5{\mathrm{m}/\mathrm{s}}^2$. Furthermore, the lateral direction ($a_{\mathrm{req},\mathrm{y}}$) is determined based on the lane change duration. For a comfortable lane change, we have chosen 6 s, which is an acceptable value based on the driving simulator research in [28]. The intermediate values are an educated guess based on the analysis of the naturalistic driving data derived in [29]. According to [29], the lane change duration varies from 0.7 s to 16.1 s, with a mean of 3.81 s and a standard deviation of 2.03 s. For the first intermediate step, we have chosen 4 s because it fits with the studies conducted in [28,29]. The second intermediate step is 2 s, which is based on the standard deviation presented in [29]. The emergency maneuver is set as previously defined with $a_{\max ,\mathrm{y}}=7$ ${\mathrm{m}/\mathrm{s}}^2$, which also fits the order size of the lower values in [29]. It should be highlighted that the values for lateral acceleration in Table 2 are explicitly derived from subjective studies on lane change and calculated using a simplified equation for the evasion time $t_{\mathrm{ev}}$ (5), where the lateral evasion distance, $d_{\mathrm{y}}$, is the lane width 3.5 m. Therefore, different lateral accelerations can be expected if other studies or advanced equations for $t_{\mathrm{ev}}$ are considered.

3.2.2. Criticality Scale

Based on the VCT, the criticality is quantified with four criticality values, $c_i$. This procedure is illustrated in Figure 4, where the index $i\in \{1,2,3,4\}$ represents the values for a certain criticality level. To calculate those values, the relations from (8), (9), (12) and (13), and the TTRs are needed. Using the longitudinal and lateral acceleration levels from Table 2 as the required deceleration for the safety distance $d_{\mathrm{b},\min }$ and $d_{\mathrm{s},\min }$ in (10) and (11), the distance relations are calculated for the longitudinal and lateral direction in (14).

• Longitudinal direction

  $\begin{array}{cc}\hfill d_{\mathrm{b},\min }^{\mathrm{L}}& =d_{\mathrm{b},\min }(a_{\mathrm{req},\mathrm{x}}=-2{\mathrm{m}/\mathrm{s}}^2)\hfill \\ \hfill d_{\mathrm{b},\min }^{\mathrm{int},1}& =d_{\mathrm{b},\min }(a_{\mathrm{req},\mathrm{x}}=-3{\mathrm{m}/\mathrm{s}}^2)\hfill \\ \hfill d_{\mathrm{b},\min }^{\mathrm{int},2}& =d_{\mathrm{b},\min }(a_{\mathrm{req},\mathrm{x}}=-5{\mathrm{m}/\mathrm{s}}^2)\hfill \\ \hfill d_{\mathrm{b},\min }^{\mathrm{H}}& =d_{\mathrm{b},\min }(a_{\mathrm{req},\mathrm{x}}=-\mu g)\hfill \end{array}$

• Lateral direction

  (14)$\begin{array}{cc}\hfill d_{\mathrm{s},\min }^{\mathrm{L}}& =d_{\mathrm{s},\min }(a_{\mathrm{req},\mathrm{y}}=0.2{\mathrm{m}/\mathrm{s}}^2)\hfill \\ \hfill d_{\mathrm{s},\min }^{\mathrm{int},1}& =d_{\mathrm{s},\min }(a_{\mathrm{req},\mathrm{y}}=0.5{\mathrm{m}/\mathrm{s}}^2)\hfill \\ \hfill d_{\mathrm{s},\min }^{\mathrm{int},2}& =d_{\mathrm{s},\min }(a_{\mathrm{req},\mathrm{y}}=1.9{\mathrm{m}/\mathrm{s}}^2)\hfill \\ \hfill d_{\mathrm{s},\min }^{\mathrm{H}}& =d_{\mathrm{s},\min }(a_{\mathrm{req},\mathrm{y}}=\mu g)\hfill \end{array}$

The minimum distances $d_{\mathrm{b},\min }^j$ and $d_{\mathrm{s},\min }^j$ with $j\in \{\mathrm{L},\mathrm{int},1,\mathrm{int},2,\mathrm{H}\}$ from (14) represent the minimum distance between the ego and target vehicle at which evasive maneuver must be performed with corresponding $a_{\mathrm{req},\mathrm{x}}$ or $a_{\mathrm{req},\mathrm{y}}$ to avoid a collision. Applying these distances in the threshold relation (12) and (13), the VCTs are defined. The VCT levels are calculated according to (15).

• TTB ≥ TTS

  $\begin{array}{cc}\hfill {\tau }_{\mathrm{th}}^{\mathrm{L}}& ={\tau }_{\mathrm{b},\mathrm{th}}(d_{\mathrm{b},\min }=d_{\mathrm{b},\min }^{\mathrm{L}})\hfill \\ \hfill {\tau }_{\mathrm{th}}^{\mathrm{int},1}& ={\tau }_{\mathrm{b},\mathrm{th}}(d_{\mathrm{b},\min }=d_{\mathrm{b},\min }^{\mathrm{int},1})\hfill \\ \hfill {\tau }_{\mathrm{th}}^{\mathrm{int},2}& ={\tau }_{\mathrm{b},\mathrm{th}}(d_{\mathrm{b},\min }=d_{\mathrm{b},\min }^{\mathrm{int},2})\hfill \\ \hfill {\tau }_{\mathrm{th}}^{\mathrm{H}}& ={\tau }_{\mathrm{b},\mathrm{th}}(d_{\mathrm{b},\min }=d_{\mathrm{b},\min }^{\mathrm{H}})\hfill \end{array}$

• TTB < TTS

  (15)$\begin{array}{cc}\hfill {\tau }_{\mathrm{th}}^{\mathrm{L}}& ={\tau }_{\mathrm{s},\mathrm{th}}(d_{\mathrm{s},\min }=d_{\mathrm{s},\min }^{\mathrm{L}})\hfill \\ \hfill {\tau }_{\mathrm{th}}^{\mathrm{int},1}& ={\tau }_{\mathrm{s},\mathrm{th}}(d_{\mathrm{s},\min }=d_{\mathrm{s},\min }^{\mathrm{int},1})\hfill \\ \hfill {\tau }_{\mathrm{th}}^{\mathrm{int},2}& ={\tau }_{\mathrm{s},\mathrm{th}}(d_{\mathrm{s},\min }=d_{\mathrm{s},\min }^{\mathrm{int},2})\hfill \\ \hfill {\tau }_{\mathrm{th}}^{\mathrm{H}}& ={\tau }_{\mathrm{s},\mathrm{th}}(d_{\mathrm{s},\min }=d_{\mathrm{s},\min }^{\mathrm{H}})\hfill \end{array}$

These threshold values represent four intervals for the TTR. Depending on the range in which the TTR time is, a criticality value according to (16) is determined. The less time available for an evasive maneuver, the more critical the situation is. It is important to notice that threshold intervals are changeable over time and depend on the vehicle states, such as $v_{\mathrm{ego}}$, $a_{\mathrm{ego}}$ and $v_{\mathrm{t}}$.

(16)$c_i=\left\{\begin{array}{cc}{c}_1=1,\hfill & \mathrm{TTR}\ge {\tau }_{\mathrm{th}}^{\mathrm{L}}\hfill \\ c_2=2,{\tau }_{\mathrm{th}}^{\mathrm{L}}>\hfill & \mathrm{TTR}\ge {\tau }_{\mathrm{th}}^{\mathrm{int},1}\hfill \\ c_3=3,{\tau }_{\mathrm{th}}^{\mathrm{int},1}>\hfill & \mathrm{TTR}\ge {\tau }_{\mathrm{th}}^{\mathrm{int},2}\hfill \\ c_4=4,{\tau }_{\mathrm{th}}^{\mathrm{int},2}>\hfill & \mathrm{TTR}\ge {\tau }_{\mathrm{th}}^{\mathrm{H}}\hfill \end{array}\right.$

3.3. Criticality Assessment Method

To assess the overall criticality of the scenario we are combining the criticality definition from (16) with the concept of the fictive vehicles depicted in Figure 3. Considering the three-lane configuration from Figure 3, it is possible to place two fictive vehicles as the condition of unoccupied adjacent lanes is satisfied. For each of those fictive vehicles, as well as for ego, the criticality values according to (16) are determined. However, for the fictive vehicle, only braking as an evasive maneuver is considered which leads to TTR=TTB, while for the ego, TTR = max{TTB, TTS}. Now, the overall criticality $c_{\mathrm{s}}$ is determined by (17). This equation takes mean criticality values calculated between ego ($c_{\mathrm{ego}}$) and each fictive vehicle ($c_{\mathrm{ego},\mathrm{f}}^{\mathrm{r}}$ and $c_{\mathrm{ego},\mathrm{f}}^{\mathrm{l}}$). The mean values are rounded to the next higher value, and the lower is chosen as the overall criticality. In the case of equal mean values for the left and right lanes, the one with a higher TTR is taken.

(17)$\begin{array}{c}\hfill c_{\mathrm{s}}=\min \left\{⌈\frac{c_{\mathrm{ego}}+c_{\mathrm{ego},\mathrm{f}}^{\mathrm{r}}}{2}⌉,⌈\frac{c_{\mathrm{ego}}+c_{\mathrm{ego},\mathrm{f}}^{\mathrm{l}}}{2}⌉\right\}\end{array}$

Additionally, if a trailing vehicle exists, the time gaps ${\tau }_{\mathrm{gap},\mathrm{l}}$ and/or ${\tau }_{\mathrm{gap},\mathrm{r}}$ are calculated and considered for the criticality assessment. The time gap ${\tau }_{\mathrm{gap},\mathrm{l}}$ represents a time gap between the trailing vehicle at the left side and ${\tau }_{\mathrm{gap},\mathrm{r}}$ at the right side. The time gap itself represents the time interval between the fictive ego vehicles ${\mathrm{EGO}}_{\mathrm{f}}^{\mathrm{l}}$ and ${\mathrm{EGO}}_{\mathrm{f}}^{\mathrm{r}}$ and the trailing vehicles ${\mathrm{T}}_{\mathrm{l}}$ and ${\mathrm{T}}_{\mathrm{r}}$. The calculation of the time gaps for the left and right sight are presented in

(18)${\tau }_{\mathrm{gap},\mathrm{l}}=\frac{(s_{{\mathrm{T}}_{\mathrm{l}}}-s_{{\mathrm{ego}}_{\mathrm{f}}^{\mathrm{l}}})}{v_{{\mathrm{T}}_{\mathrm{l}}}},$

(19)${\tau }_{\mathrm{gap},\mathrm{r}}=\frac{(s_{{\mathrm{T}}_{\mathrm{r}}}-s_{{\mathrm{ego}}_{\mathrm{f}}^{\mathrm{r}}})}{v_{{\mathrm{T}}_{\mathrm{r}}}}$

with $s_{{\mathrm{T}}_{\mathrm{l}}}$ and $s_{{\mathrm{T}}_{\mathrm{r}}}$ being the trailing vehicle positions; $v_{{\mathrm{T}}_{\mathrm{l}}}$ and $v_{{\mathrm{T}}_{\mathrm{r}}}$ being the trailing vehicle velocities; $s_{{\mathrm{ego}}_{\mathrm{f}}^{\mathrm{l}}}$ and $s_{{\mathrm{ego}}_{\mathrm{f}}^{\mathrm{r}}}$ being the fictive ego vehicle positions. Here, ${\tau }_{\mathrm{gap},\mathrm{r}}^{\mathrm{th}}$ and ${\tau }_{\mathrm{gap},\mathrm{l}}^{\mathrm{th}}$ represent the threshold values for the time gap. If the time gape values are lower than these thresholds, the possibility of steering will not be considered for the overall assessment. In this case, TTR is set to be TTB.

4. Simulation Results

For demonstrating the introduced assessment method, two exemplary test scenarios shown in Figure 5 are taken into account. In the first scenario, the ego vehicle is approaching the target vehicle ${\mathrm{T}}_{2,2}$ in lane 2, while vehicles ${\mathrm{T}}_{1,2}$ and ${\mathrm{T}}_{3,2}$ are placed adjacent to ${\mathrm{T}}_{2,2}$. In the second scenario, the trailing ${\mathrm{T}}_{1,1}$ is added. Threshold for the time gap (${\tau }_{\mathrm{gap},\mathrm{l}}^{\mathrm{th}}$) between ${\mathrm{T}}_{1,1}$ and ${\mathrm{EGO}}_{\mathrm{f}}^{\mathrm{l}}$ is set to $3$ s. This means that, if ${\tau }_{\mathrm{gap},\mathrm{l}}^{\mathrm{th}}\le 3$, then the ego cannot consider steering in lane 1. Scenarios are created and simulated within the simulation software CarMaker from IPG Automotive, and the initial positions and velocities of vehicles are given in Table 3.

Figure 6 shows the traveled distance and velocities of vehicles; while the ego vehicle keeps its speed constant, the target vehicles reduce their speed after $10$ s to the defined velocities. Finally, scenarios will lead to the collision of the ego with ${\mathrm{T}}_{2,2}$, as the ego vehicle does not brake and has no assistance functions. Collision is not mitigated on purpose, in order to demonstrate how the criticality of the scenario changes as the ego approaches ${\mathrm{T}}_{2,2}$.

Velocity reductions are different for each lane which cause different criticality values, and this behavior is shown in Figure 7. It can be seen that the criticality of the ego lane has the longest criticality phase with a criticality value of 4 (high criticality). This is because of the aggressive speed-reduction maneuver of the front vehicle, ${\mathrm{T}}_{2,2}$. The right lane has a slightly lower criticality, and the left one has the lowest criticality over time. Because the left lane has a criticality of 2 (low criticality) until 7.72 s, it represents a potentially avoidable situation for the ego vehicle in this time interval. Therefore, the overall criticality $c_{\mathrm{s}}$ for the ego vehicle is one level lower between 5.92 s and 7.62 s, as shown in Figure 7. This is only valid up to 7.62 s, where the ego vehicle comes closer to the vehicle ${\mathrm{T}}_{2,2}$ and the potential evasive maneuver space becomes smaller. The last point where the ego vehicle could make evasive maneuver is at the point where the TTR time intersects the ${\tau }_{\mathrm{th}}^{\mathrm{H}}$ at 11.6 s. After that, a collision cannot be avoided anymore.

The whole scenario behaves slightly differently when a trailing vehicle is considered. The small time gap between the ${\mathrm{T}}_{1,1}$ and ${\mathrm{EGO}}_{\mathrm{f}}^{\mathrm{l}}$ eliminates the possibility of ego to steer in the left lane so the total criticality rises to the maximum criticality for the time interval 5–8 s, see Figure 8.

In summary, these two examples demonstrate the functionality of the criticality assessment method for the assumptions made to calculate the TTR and the chosen VCT. In addition, these particular examples were chosen to show the potential of the method by considering and analyzing the influences of the ego vehicles surrounding for the criticality calculation.

5. Conclusions

In the current paper, we introduced a criticality assessment method for ADS. The novelty of the method lies in the introduction of fictive ego vehicles from the surrounding of the ego. In order to determine the overall criticality, the collision time reserve as TTR metric was introduced and evaluated using four criticality levels. The criticality levels are determined using variable threshold values, which are converted into the time domain from minimal safety distance metrics and acceleration values. Considered acceleration values present a linkage between objective investigation and threshold values, while the derived VCT fills the research gap by taking into account the kinematic relations between vehicles, and thus variable nature of the criticality. To derive feasible and intelligible method, certain assumption has been made. The method is derived for straight highway roads, while formulation assumes constant velocity of a target vehicle.

Finally, the method is demonstrated on two highway scenarios that defer in the presence of a trailing vehicle. The results show higher criticality of scenarios in which trailing vehicles obstruct the adjacent lane of the ego. This narrowed down the alternative options for the ego and increased the overall criticality. The potential and advantage of the proposed criticality assessment method reside in simplified consideration of the surrounding ego vehicles and the evaluation of possible evasive actions; in addition to the criticality assessment, it could be used for decision-making applications. For future works, the proposed criticality metric and threshold calculation will be applied in a wide range of scenarios, while the mathematical formulation will be extend to cover considered assumptions and to make realistic lane change models for steering maneuvers.

Author Contributions

Conceptualization, D.N., T.M. and M.S.; methodology, D.N. and T.M.; software, D.N. and F.O.; validation, D.N., F.O., T.M. and M.S.; formal analysis, F.O., A.E., C.L. and W.S.; investigation, D.N. and T.M.; resources, D.N. and T.M.; data curation, D.N. and F.O.; writing—original draft preparation, D.N., T.M., F.O., M.S., C.L. and W.S.; writing—review and editing, D.N., T.M., F.O., M.S., C.L., A.E. and W.S.; visualization, D.N.; supervision, A.E.; project administration, A.E.; funding acquisition, A.E. and W.S. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

Supported by TU Graz Open Access Publishing Fund.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

ADS	automated driving systems
SCS	safety-critical scenario
TTC	time to collision
TTR	time to react
TTB	time to brake
TTS	time to steer
TTK	time to kickdown
TTx	time to x
LPTB	last point to brake
LPTS	last point to steer
VCT	variable criticality threshold

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 3. Graphical representation of the two-lane and three-lane scenarios.

Figure 4. Illustration of the criticality levels.

Figure 5. Graphical representation of test scenarios.

Figure 6. Distances traveled and velocities in the tested two scenarios.

Figure 7. Criticality assessment results for the two introduced scenarios.

Figure 8. Criticality assessment results for the time frame between [Forumla omitted. See PDF.] s and [Forumla omitted. See PDF.] s.

Table 2

Acceleration levels for lateral and longitudinal movement.

Acceleration Levels	${\mathit{a}}_{\mathbf{req},\mathbf{x}}$	${\mathit{a}}_{\mathbf{req},\mathbf{y}}$
Comfort	$-2{\mathrm{m}/\mathrm{s}}^2$	$0.2{\mathrm{m}/\mathrm{s}}^2$
Intermediate 1	$-3{\mathrm{m}/\mathrm{s}}^2$	$0.5{\mathrm{m}/\mathrm{s}}^2$
Intermediate 2	$-5{\mathrm{m}/\mathrm{s}}^2$	$1.9{\mathrm{m}/\mathrm{s}}^2$
Emergency	$-\mu g$	$7{\mathrm{m}/\mathrm{s}}^2$

Table 3

Initial states and parameters for the test scenarios.

Scenario Description
	Start position on the lane in m	Start velocity in km/h	Speed reduction time in s	Desired speed reduction in km/h
Scenario 1	Trailing vehicle is not considered
EGO	50	130	None	None
${\mathrm{T}}_{1,2}$	250	105	15	90
${\mathrm{T}}_{2,2}$	250	100	15	50
${\mathrm{T}}_{2,3}$	250	105	15	70
Scenario 2	Trailing vehicle is considered.
Trailing vehicle	0	130	15	100

References

1. Feng, S.; Feng, Y.; Sun, H.; Zhang, Y.; Liu, H.X. Testing Scenario Library Generation for Connected and Automated Vehicles: An Adaptive Framework. IEEE Trans. Intell. Transp. Syst.; 2020; 23, pp. 1213-1222. [DOI: https://dx.doi.org/10.1109/TITS.2020.3023668]

2. Neurohr, C.; Westhofen, L.; Henning, T.; de Graaff, T.; Möhlmann, E.; Böde, E. Fundamental Considerations around Scenario-Based Testing for Automated Driving. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV); Las Vegas, NV, USA, 19 October–13 November 2020; pp. 121-127. [DOI: https://dx.doi.org/10.1109/IV47402.2020.9304823]

3. Nalic, D.; Mihalj, T.; Bäumler, M.; Lehmann, M.; Eichberger, A.; Bernsteiner, S. Scenario Based Testing of Automated Driving Systems: A Literature Survey. Proceedings of the FISITA Web Congress 2020; Virtual Event, 24 November 2020.

4. Wen, M.; Park, J.; Cho, K. A scenario generation pipeline for autonomous vehicle simulators. Hum.-Centric Comput. Inf. Sci.; 2020; 10, 24. [DOI: https://dx.doi.org/10.1186/s13673-020-00231-z]

5. Andreotti, E.; Boyraz, P. Selpi. Mathematical Definitions of Scene and Scenario for Analysis of Automated Driving Systems in Mixed-Traffic Simulations. IEEE Trans. Intell. Veh.; 2020; 6, pp. 366-375. [DOI: https://dx.doi.org/10.1109/TIV.2020.3031981]

6. Leitner, A. ENABLE-S3: Project Introduction. Validation and Verification of Automated Systems; Springer: Cham, Switzerland, 2020; pp. 13-23.

7. Rodemerk, C.; Habenicht, S.; Weitzel, A.; Winner, H.; Schmitt, T. Development of a general criticality criterion for the risk estimation of driving situations and its application to a maneuver-based lane change assistance system. Proceedings of the 2012 IEEE Intelligent Vehicles Symposium; Madrid, Spain, 3–7 June 2012; pp. 264-269. [DOI: https://dx.doi.org/10.1109/IVS.2012.6232129]

8. Stumper, D.; Knapp, A.; Pohl, M.; Dietmayer, K.C.J. Towards Characterization of Driving Situations via Episode-Generating Polynomials. Advanced Microsystems for Automotive Applications 2016; Springer: Cham, Switzerland, 2016; pp. 165-173.

9. Riedmaier, S.; Ponn, T.; Ludwig, D.; Schick, B.; Diermeyer, F. Survey on Scenario-Based Safety Assessment of Automated Vehicles. IEEE Access; 2020; 8, pp. 87456-87477. [DOI: https://dx.doi.org/10.1109/ACCESS.2020.2993730]

10. Broadhurst, A.; Baker, S.; Kanade, T. Monte Carlo road safety reasoning. Proceedings of the IEEE Proceedings. Intelligent Vehicles Symposium; Las Vegas, NV, USA, 6–8 June 2005; pp. 319-324.

11. Mahmud, S.S.; Ferreira, L.; Hoque, M.S.; Tavassoli, A. Application of proximal surrogate indicators for safety evaluation: A review of recent developments and research needs. IATSS Res.; 2017; 41, pp. 153-163. [DOI: https://dx.doi.org/10.1016/j.iatssr.2017.02.001]

12. Li, Y.; Zheng, Y.; Morys, B.; Pan, S.; Wang, J.; Li, K. Threat Assessment Techniques in Intelligent Vehicles: A Comparative Survey. IEEE Intell. Transp. Syst. Mag.; 2020; 13, pp. 71-91. [DOI: https://dx.doi.org/10.1109/MITS.2019.2907633]

13. Ozbay, K.; Yang, H.; Bartin, B.; Mudigonda, S. Derivation and validation of new simulation-based surrogate safety measure. Transp. Res. Rec.; 2008; 2083, pp. 105-113. [DOI: https://dx.doi.org/10.3141/2083-12]

14. Wachenfeld, W.; Junietz, P.; Wenzel, R.; Winner, H. The worst-time-to-collision metric for situation identification. Proceedings of the 2016 IEEE Intelligent Vehicles Symposium (IV); Gothenburg, Sweden, 19–22 June 2016; pp. 729-734. [DOI: https://dx.doi.org/10.1109/IVS.2016.7535468]

15. Minderhoud, M.M.; Bovy, P.H. Extended time-to-collision measures for road traffic safety assessment. Accid. Anal. Prev.; 2001; 33, pp. 89-97. [DOI: https://dx.doi.org/10.1016/S0001-4575(00)00019-1]

16. Hillenbrand, J. Fahrerassistenz zur Kollisionsvermeidung. Ph.D. Thesis; Universität Karlsruhe (TH): Karlsruhe, Germany, 2007; [DOI: https://dx.doi.org/10.5445/IR/1000007728]

17. Junietz, P.M. Microscopic and Macroscopic Risk Metrics for the Safety Validation of Automated Driving. Ph.D. Thesis; Technische Universität: Darmstadt, Germany, 2019.

18. Van Der Horst, R.; Hogema, J. Time-to-collision and collision avoidance systems. Proceedings of the 6th ICTCT WorkshopSafety Evaluation of Traffic Systems: Traffic Conflicts and Other Measures; Salzburg, Austria, 27–29 October 1993; pp. 109-121.

19. Chen, R.; Sherony, R.; Gabler, H.C. Comparison of Time to Collision and Enhanced Time to Collision at Brake Application during Normal Driving. Proceedings of the SAE 2016 World Congress and Exhibition. SAE International; Detroit, MI, USA, 12–14 April 2016; [DOI: https://dx.doi.org/10.4271/2016-01-1448]

20. Hillenbrand, J.; Kroschel, K.; Schmid, V. Situation assessment algorithm for a collision prevention assistant. Proceedings of the IEEE Proceedings. Intelligent Vehicles Symposium 2005; Las Vegas, NV, USA, 6–8 June 2005; pp. 459-465. [DOI: https://dx.doi.org/10.1109/IVS.2005.1505146]

21. Eckert, A.; Hartmann, B.; Sevenich, M.; Rieth, P. Emergency steer & brake assist: A systematic approach for system integration of two complementary driver assistance systems. Proceedings of the 22nd International Technical Conference on the Enhanced Safety of Vehicles (ESV); Washington, DC, USA, 13–16 June 2011; pp. 13-16.

22. Winner, H.; Lemmer, K.; Form, T.; Mazzega, J. Pegasus—First steps for the safe introduction of automated driving. Road Vehicle Automation 5; Springer: Cham, Switzerland, 2019; pp. 185-195.

23. Weber, H.; Bock, J.; Klimke, J.; Roesener, C.; Hiller, J.; Krajewski, R.; Zlocki, A.; Eckstein, L. A framework for definition of logical scenarios for safety assurance of automated driving. Traffic Inj. Prev.; 2019; 20, pp. S65-S70. [DOI: https://dx.doi.org/10.1080/15389588.2019.1630827] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31381437]

24. Shalev-Shwartz, S.; Shammah, S.; Shashua, A. On a formal model of safe and scalable self-driving cars. arXiv; 2017; arXiv: 1708.06374

25. Nguyen, T.; NguyenDinh, N.; Lechner, B.; Wong, Y.D. Insight into the lateral ride discomfort thresholds of young-adult bus passengers at multiple postures: Case of Singapore. Case Stud. Transp. Policy; 2019; 7, pp. 617-627. [DOI: https://dx.doi.org/10.1016/j.cstp.2019.07.002]

26. Hoberock, L.L. A survey of longitudinal acceleration comfort studies in ground transportation vehicles. J. Dyn. Syst. Meas. Control; 1977; 99, pp. 76-84. [DOI: https://dx.doi.org/10.1115/1.3427093]

27. Maurer, M.; Eskandarian, A. Forward collision warning and avoidance. Handbook of Intelligent Vehicles; Springer: London, UK, 2012; pp. 657-687.

28. Rogic, B.; Nalic, D.; Eichberger, A.; Bernsteiner, S. A novel approach to integrate human-in-the-loop testing in the development chain of automated driving: The example of automated lane change. IFAC-PapersOnLine; 2020; 53, pp. 10188-10195. [DOI: https://dx.doi.org/10.1016/j.ifacol.2020.12.2747]

29. Yang, M.; Wang, X.; Quddus, M. Examining lane change gap acceptance, duration and impact using naturalistic driving data. Transp. Res. Part C Emerg. Technol.; 2019; 104, pp. 317-331. [DOI: https://dx.doi.org/10.1016/j.trc.2019.05.024]