Adaptive Fault Detection and Emergency Control of Autonomous Vehicles for Fail-Safe Systems Using a Sliding Mode Approach

This paper presents a sliding mode-based adaptive fault detection and emergency control algorithm for implementation in fail-safe systems of autonomous vehicles. The overall algorithm is comprised of a fault detection part and a fail-safe control part. For the former, sliding mode observer-based fault detection algorithms were developed for environment and chassis sensors, including LiDAR, Radar, and acceleration sensors. Unidentified fault signals from the sensors are reconstructed through the adaptive sliding mode observer. The reconstruction is based on the MIT rule through the use of an estimated sensitivity parameter. For the latter, a sliding mode control (SMC)-based emergency control method designed to respond to fault occurrences has been proposed to ensure the functional safety of autonomous vehicles. An adaptive gain parameter was designed, taking convergence time into consideration, to secure consistent and rapid responses from the controller. When the detection algorithm detects a fault, the appropriate control input is computed by a lower controller for the vehicle. This control input is calculated based on the last scene information obtained from an upper controller. The performance of the proposed fault detection and control algorithms has been evaluated through simulations and actual vehicle tests of various scenarios.

SECTION I. Introduction

There is no contest regarding the safety of autonomous vehicles being of the utmost importance within the industry. The inherent nature of autonomous vehicles brings forth the issue that even a minor vehicle fault may lead to serious, possibly fatal, accidents. Hence, studies on fail-safe systems (also known as fallback systems) are particularly important for autonomous vehicle research. Autonomous vehicles are equipped with various equipment such as sensors, computers, and actuators. While the various parts are communicating with each other, monitoring the condition of each part is essential; Fail-safe systems form decisions based on these conditions. For autonomous vehicle sensors, abnormal conditions can be identified with relative ease as manufacturers provide essential information regarding the sensor conditions. However, a general solution to the autonomous vehicle fail-safe system still proves to be challenging as the hardware and software configurations differ depending on the vehicle [1]. Each vehicle system requires a corresponding and independent fail-safe structure. Because numerous companies and research institutes independently develop their own autonomous vehicles, the challenge of developing a universal fail-safe system persists.

Prior studies have shown that several vehicle companies have developed a dual-system and fallback system for safety. For example, a fallback system can be found in Google Waymo’s autonomous vehicles. Waymo’s fallback system is composed of 4 backup systems: Backup computing, backup braking, backup steering, and backup power [2]. GM CRUISE, on the other hand, has a backup computer, backup actuators, and redundant signal communication [3]. Baidu’s Apollo autonomous vehicle has a guardian module and a fail-safe system. The open-source nature of the Apollo software structure was applied to all control commands through the guardian module [4]. The studies mentioned so far introduce the various independent fail-safe methodologies applied to different autonomous vehicles.

Other areas of studies apart from autonomous vehicles have also shown progress in fail-safe systems through the use of fault diagnosis and countermeasure methods for system malfunction or performance degradation. Of such, fault detection and diagnosis methods have been used in fail-safe structures to develop the stability and reliability of their fail-safe systems. Fault diagnosis usually utilizes the concept of hardware and software redundancy. The hardware redundancy concept relies on the usage of 2 pieces of the same hardware with the same input. The output signals of both pieces of hardware are compared to detect faults. However, the downside to this method is cost and limitations on mounting locations; Environment sensors such as LiDARs, RADARs, and Cameras cannot be installed on the exact same locations. Consequently, the progression of fault diagnosis has led to software redundancy (analytical redundancy) methods becoming the mainstream of fault detection and diagnosis study since the 1980s [5]. Some examples of software redundancy implementation are given below. Fault reconstruction and detection were achieved through the use of a sliding mode observer with parameters obtained from the LMI technique [6]. Fault diagnosis on applications for satellite formation flight was done through a robust unknown input observer. In this case, the isolation of faulty actuators was done while taking model uncertainty, input, and output environmental disturbance into account [7]. Fault estimation and fault-tolerant control problems of a class of switched stochastic systems were characterized using an independently designed observer; sensor faults were estimated and used for fault diagnosis. The dimensions of the designed observer were reduced when compared to existing results. [8] The advantages and practical implications of artificial intelligence (AI) were reviewed in rotating machinery. [9] Motor bearing condition monitoring and fault diagnosis via a vibrational signal were achieved using a wireless sensor network’s amplitude and frequency data [10]. Real-time monitoring and fault detection for motor arrays through vibration, current, and temperature data was achieved via a wireless sensor network, in consideration of time delays [11]. A fault diagnosis system for autonomous vehicles based on neural networks using the internet-of-things (IoT) was developed to collect various types of information [12]. Fail-safe motion planning and verification were conducted to generate vehicle trajectories that could allow vehicles to maintain desired maneuvers with enhanced safety [41]–​[43]. A fail-safe priority-based intersection approach was developed to manage signal-less intersections autonomously [44]. A fault detection, isolation, and identification architecture for multi-faults in multi-sensor systems has been introduced in [39]. Algorithms designed to diagnose faults in functional parts of autonomous vehicles were developed for fail-safe systems [38], [45]–​[47]. The following studies have placed greater focus on designing and analyzing environmental sensor faults. For example, vehicle chassis sensor faults [47], [48] handled faults related to vehicle chassis sensors, whereas faults in environment sensors such as LiDARs and Radars were tackled in [38], [45]–​[47], [49]–​[52]. [53]–​[55], on the other hand, focused on actuator faults. All these studies have performed validations through simulations and vehicle tests. Various methodologies regarding fault diagnosis and fault-tolerant control were proposed [45], [46], [48], [52]–​[54].

A thorough review of the studies mentioned above has shown that aspects of fail-safe systems such as fault detection and reconstruction have been studied through several methods, including observer-approaches, statistical methods, sensor monitoring, and artificial networks. However, most methods rely on the observer or estimator assumption that fault is bounded by parameter design.

This paper proposes two fault detection algorithms for longitudinal sensor fault detection. Other studies have shown the occasional use of the sliding mode observer method to detect faults. However, the conventional sliding mode observer requires an output error boundary to design the injection term parameters. Designing this observer still proves to be challenging due to the time-variant nature of the output error boundary. Therefore, in this paper, an adaptive sliding mode observer was developed to update the injection term parameters in accordance with output errors affected by acceleration faults. Methodologies with the acceleration fault can be reconstructed using the Lyapunov stability concept, also applicable in detecting acceleration sensor faults through a threshold approach. Moreover, a linear model-prediction algorithm has been designed for faults in environmental sensors as well.

In their current state, environmental sensors utilize clearance data and relative velocity to make predictions based on the discretized system model. Two predicted states can be computed from the minimum and maximum acceleration usage. The feasible boundaries are then configured from each predicted state. Together with past accumulated states, comparisons are made with the current state. The fault index can be calculated by counting values sitting outside the configured boundaries. Finally, an environmental sensor fault is determined based on a defined threshold after averaging the count. Three representative fault types are classified as follows: Power off, Holding Signal, Offset Signal. Of the type of faults mentioned so far, this paper will only consider and demonstrate the offset fault. This is because the detection of offset faults results in easy identification of the other types.

This paper also contains an emergency control portion essential for an autonomous vehicle to be in accordance with level 4 autonomous driving. This research utilizes an SMC method that takes adaptive convergence time into account. Because the convergence time of the control error was determined mathematically, the corresponding control values were designed to change this predetermined convergence time. The method incorporating convergence time was applied to the test vehicle under the fail-safe control portion. Later sections describe the method’s proof of stability in greater detail. Shown below are some of the relevant literature surveys regarding SMCs and the usage of such a method.

Jun Hu et al. proposed a survey on SMC for networked control systems [56]. This survey was able to provide a greater understanding of the current application trends of the SMC. Jun Hu et al. also investigated the uncertain nonlinear systems of actuator faults [57]. In this research, the team designed various uncertain actuator faults and validated a newly proposed controller, the ISMC scheme, to ensure stability. Validations were carried out with the rocket fairing structure model. SMC for Networked control systems (NCSs) shows great promise in fail-safe research in that methodologies used here can also be effectively utilized in fail-safe systems as well. Moving on, the SMC was effectively utilized in reducing the effects of actuator attack failures [58]. The proposed approach was able to guarantee a type-2 fuzzy system and the input-to-state stabilization of sliding motion. Among the studies showing various SMC methodologies, their approach to time delay could be categorized into 4 types: constant time-delay, time-varying delay, distributed delay, and probabilistic interval time delay [56]. The constant time-delay type could be further divided into present-based SMCs and time-shift SMCs. A related study proposed a predictive SMC for a networked control system with time delay and packet dropout [59]. A chattering-free law was also designed and utilized in the predictive SMC. Within the field of time-varying delay, recent trends have leaned towards delay-fraction SMC research.

D. Ao et al. proposed a super-twisting sliding mode control algorithm (STA), based on the Lyapunov theory, to enhance the robustness and attenuation of the chattering issue [60]. S. Wang et al. suggested a new sliding mode control strategy based on an RBF (Radial Basis Function) neural network to solve tracking errors and substantial chattering phenomena [61]. A mathematical nonlinear longitudinal model was proposed, and their algorithm was validated through a simulation study under typical driving conditions. B. Peng et al. proposed an improved sliding mode control strategy for vehicle platoons, capable of forming a platoon in shorter time periods in simulated environments with better stability [62]. The proposed strategy was able to reach a stable and controllable area rapidly. P.Wang et al. proposed a robust automatic control strategy based on the back-stepping sliding mode control theory [63]. The study conducted a co-simulation with CarSim/Simulink under various scenarios. Y. Han et al. proposed a sliding mode control method based on the improved reaching law [63]. The method was developed to control the attitude of an automated guided vehicle, where an outer loop controller outputs the desired attitude. Validations were conducted through simulations.

So far, a thorough review of the numerous related studies has shown current trends and achievements made with the SMO and SMC. However, some issues are yet to be addressed. Firstly, methodologies proposed by most studies are developed under the assumption that faults are bounded by parameter design. Secondly, there is still a lack of research on the usage of the SMO and SMC in emergency controls. Additionally, a “PC (Upper controller) Shutdown” situation, where algorithms may only work with the last available values, is a genuine possibility that cannot be ignored. This paper aims to address all these issues with the algorithms proposed by our research team. The main contributions of this work can be summarized as follows:

1. A newly designed framework for fail-safe modules in Autonomous Vehicles has been proposed and implemented in an actual autonomous vehicle.

2. An MIT rule-based adaptive SMO methodology has been proposed for the detection of longitudinal faults in chassis and environment sensors (lidar, radar) and for the reconstruction of unknown faults in real driving scenarios.

3. The proposed adaptive SMC (In consideration of convergence time) has been successfully implemented and tested in fault scenarios involving actual vehicle experiments. Possible fault types of AVs have been defined through these experiments as well.

The remainder of this paper is divided into six sections and organized as follows. Section II provides an overview of the proposed fail-safe system and a brief introduction to the controller. Section III is further divided into 5 subsections that provide detailed explanations of the proposed observer. Section IV describes the vehicle actuator system model. Section V is further divided into 5 subsections that provide a description of the fail-safe algorithm and the evaluation results of the emergency controller. Finally, Section VI finalizes the paper with a conclusion and planned future works.

SECTION II. Overview of a Fail-Safe System in Autonomous Vehicles

This section introduces the fail-safe system applied to the autonomous vehicle used in this paper. While the system has only been applied and tested on autonomous vehicles used in university laboratories, its uses are not limited to select test vehicles. The methods proposed in this paper are still extensively applicable in other areas. The overall controller is composed of an upper controller and a lower controller: A commercial, industrial PC (IPC) as the upper controller, and a MicroAutobox (dSPACE) as a lower controller. It is to be noted that the lower controller is more robust than the upper controller. The test vehicle’s sensor configuration is depicted in Fig. 1. In this research, the focus is primarily placed on developing a fail-safe module for an automated driving system. Fig. 2 depicts a diagram of the module that contains Fault Detection and Fail-Safe Control.

FIGURE 1.

Autonomous vehicle hardware configuration.

FIGURE 2.

Schematic of FAILSAFE module algorithm.

The Fault Detection portion is made up of a Hardware Fault Detection part and an Algorithm-Based Detection part. Fault detection in this paper will mainly refer to algorithm-based environmental sensor fault detection and chassis sensor reconstruction methodologies. Section III will introduce the algorithm-based fault detection methodology in greater detail.

The next part introduces the adaptive sliding mode observer-based fault reconstruction and detection. On the other hand, fail-safe control is comprised of Tolerance Control and Emergency Control parts. Due to differences in the longitudinal controller, Tolerant Control could not be effectively applied to the test vehicle. Hence it was only tested via simulations. This is further described in [13]. Within the Emergency Control portion, a reference target building block and a control block, one that follows the reference target model, can be found.

This paper also utilizes a longitudinal control algorithm encompassing the fault detection algorithms. The following three consecutive parts make up the control algorithm. First, a decision part determines the vehicle’s current state through the CAN bus signal. Next, confirmation regarding a fault where the CAN communication gets stuck occurs. Indexes based on the best reference target model are determined. Finally, the lower-level controller determines the desired control input: The steering angle is calculated through a dead reckoning method, whereas the longitudinal acceleration control input is determined based on a sliding mode control method that tracks the reference target model. Table 1 describes the overall algorithm used in this paper for the fail-safe system in autonomous vehicles. In Table 1, the fault detection method corresponding to each hardware module can be found. An appropriate maneuver strategy for each module is also proposed.

TABLE 1 Hardware-Based Fail-Safe Module Description

In Table 1, a single asterisk indicates that the method was applied to an actual autonomous test vehicle. A double asterisk indicates that the method was only verified through simulations.

SECTION III. Adaptive Sliding Mode Observer and Linear Prediction Based Sensor Fault Detection

The following driving condition was proposed to test for various fault scenarios. The test vehicle was to follow another leading vehicle located in its longitudinal direction. The environmental sensors were used to measure and obtain vehicle clearance and relative velocity information. The acceleration of the leading vehicle was measured through a wireless communication sensor. The observer’s kinematic model defined for the longitudinal direction is given in the following form: \begin{align*} \dot {x}=&Ax+Bu\tag{1a}\\ y=&Cx\tag{1b}\\ A=&\left [{ {{\begin{array}{cc} 0 &\quad 1 \\ 0 &\quad 0 \\ \end{array}}} }\right]\!,\quad B=\left [{ {{\begin{array}{c} 0 \\ 1 \\ \end{array}}} }\right]\!,~C=\left [{ {{\begin{array}{cc} 1 &\quad 1 \\ \end{array}}} }\right]\tag{1c}\end{align*} $\begin{aligned} x=\left [{ {{\begin{array}{cc} {x_{1}} &\,\,{x_{2}} \\ \end{array}}} }\right]^{T} \end{aligned}$ refers to the clearance and relative velocity of the leading vehicle. $u$ indicates the relative acceleration, the difference in longitudinal accelerations of the leading and the test vehicles, defined by $a_{p} -a_{s} $ . The adaptive sliding mode observer and linear prediction algorithm for fault detection in the longitudinal direction are designed utilizing Equations (1a) – (1c).

A. Adaptive Sliding Mode Observer for Acceleration Sensor Fault Reconstruction and Detection

The adaptive sliding mode observer is designed to reconstruct the acceleration fault: see model schematics in Fig. 3. The description of Fig. 3. is as follows.

FIGURE 3.

Model schematics of the acceleration fault reconstruction and detection.

The environment, communication, and acceleration sensors transmit various sensor values to the observer. The SMO then calculates an estimated error, $e_{y} $ , and sends it to the forgetting factor module. The forgetting factor module then calculates the relationship coefficients $C_{e},C_{p} $ . Upon receiving the coefficients $C_{e},C_{p} $ , the Adaptive MIT rule module updates parameter $\rho $ and transmits it back to the SMO. Throughout the sequence, the fault detection threshold block continuously receives the threshold results and reconstructed fault from SMO, and exports a fault detection signal as an output. Since this process was designed to work in real-time, the period of each step was set to 10ms(0.01Hz). The overall algorithm was then implemented and tested on an actual vehicle.

The kinematic model inclusive of the fault term $f_{a} $ (if it exists) is represented as follows: \begin{align*} \dot {x}=&Ax+Bu+Ff_{a}\tag{2a}\\ F=&\left [{0\quad -1}\right]^{T}\tag{2b}\end{align*} Here, we introduce a coordinate transformation $x_{t} \mapsto Tx$ , while taking the system model into consideration. $T$ is defined as \begin{equation*} T=\left [{Null(C)\quad C }\right]^{T}\tag{2c}\end{equation*} The following assumptions are made when designing the observer:

1. When observer error is zero, acceleration faults can exist after convergence.

2. When convergence time is extremely small, the observer performs appropriately.

The assumptions are further discussed in subsection B. We denote the observer estimated equation as follows:\begin{align*} \dot {{\hat {x}}}_{t}=&A_{t} \hat {x}_{t} +B_{t} u+G_{n} v \tag{2d}\\ y_{t}=&C_{t} \hat {x}_{t} \\ A_{t}=&\left [{ {{\begin{array}{cc} {A_{t11}} & \quad {A_{t12}} \\ {A_{t21}} &\quad {A_{t22}} \\ \end{array}}} }\right]\!,\quad B_{t} =\left [{ {{\begin{array}{c} {B_{t1}} \\ {B_{t2}} \\ \end{array}}} }\right]\!, \\ C_{t}=&\left [{ {{\begin{array}{cc} 0 &\quad I \\ \end{array}}} }\right]\tag{2e}\\ G_{n}=&\left [{ {{\begin{array}{cc} L &\quad {-I} \\ \end{array}}} }\right]\!,\quad v=\rho sign(e_{y})\tag{2f}\end{align*} Here, $\hat {x}_{t}$ represents the observer’s estimated state. $G_{n} $ represents the distribution matrix, inclusive of the design parameter $L$ in equation (2f). The transformed system matrix is $A_{t} =TAT^{-1}$ , $B_{t} =TB$ , and $C_{t} =CT^{-1}$ . $\rho $ represents a coefficient bounded through a stability analysis, where parameters were determined through a vehicle test, shown in Table 3. Subsections B and C provide further explanations of the coefficient $\rho $ . Error dynamics can be derived using system and observer models by defining error as $e=\hat {x}_{t} -x_{t} $ . The error dynamics are given as follows: \begin{align*} \dot {e}_{1}=&A_{t,11} e_{1} +A_{t,12} e_{2} +Lv\tag{3a}\\ \dot {e}_{y}=&A_{t,21} e_{1} +A_{t,22} e_{y} -v\tag{3b}\end{align*} Here, $v$ represents the injection term. A proper design of the injection term $v$ results in $v$ being large. In this case, the errors $e_{y} $ and $\dot {e}_{y} $ in equation (3a) converge to zero within finite time. This is further described in the next section. Equation (3a) can be rewritten as \begin{equation*} v=A_{t,21} e_{1}\tag{3c}\end{equation*} Substituting equation (3c) to equation (2f), a dynamic of error $e_{1}$ can be calculated as follows:\begin{equation*} \dot {e}_{1} =(A_{t,11} +LA_{t,21})\cdot e_{1}\tag{3d}\end{equation*} Error $e_{1} $ and $\dot {e}_{1} $ can also converge to zero when the eigenvalue of $A_{t11} +LA_{t21} $ is negative definite. If a fault occurs after the observer error $e$ converges to zero, only the fault term $f_{a} $ remains in the error dynamic model. It can then be described as follows: \begin{align*} 0=&F_{t,1} f_{a} +Lv\tag{4a}\\ 0=&F_{t,2} f_{a} -v\tag{4b}\end{align*} Here, $F_{t,1} $ and $F_{t,2} $ are variables of transformed fault matrix calculated using $F$ in equation (2b). The designed parameter $L$ and reconstructed acceleration fault $f_{a,rec} $ are described as follows:\begin{align*} L=&-F_{t,1} (F_{t,2})^{-1}\tag{4c}\\ f_{a,rec}=&-(F_{t,1})^{-1}Lv_{eq} =-(F_{t,2})^{-1}v_{eq}\tag{4d}\end{align*} $F_{t,1},F_{t,2} $ are variables of the transformed fault matrices. $v_{eq} $ denotes the equivalent injection term. Here, equation (4c) should simultaneously satisfy the condition derived from equation (3d). The reconstructed fault is used to detect acceleration fault based on a threshold approach, referred to as the fault detection signal in equation (4e). The fault detection index $I_{a} $ is described as follows:\begin{align*} I_{a} =\begin{cases} \displaystyle 1(Fault) & (f_{a,rec} \ge Th_{a}) \\ \displaystyle 0(No\,\,Fault) & (f_{a,rec} < Th_{a}) \\ \displaystyle \end{cases}\tag{4e}\end{align*} $f_{a,rec} $ and $Th_{a} $ represent the reconstructed fault and threshold index, respectively. $Th_{a} $ is introduced in Table 3 of the next section. The following subsection introduces a design parameter for the sliding mode observer to satisfy the stability condition of output error $e_{y} $ .

TABLE 2 Module Description Model Schematics of Adaptation Algorithm Based on Mit Rule

TABLE 3 Model Parameters

B. Stability Analysis of Sliding Mode Observer

The convergence stability of the output error can be ensured through the use of the injection term $v$ , defined in expression (3c). The Lyapunov direct method-based injection term, magnitude $\rho $ , is designed by the finite-time convergence condition. The Lyapunov function and conditions are defined to ensure stability in the design of the injection terms. It is given as follows: [20], [21] \begin{align*} V=&\frac {1}{2}e_{y}^{2}\tag{5a}\\ \dot {V} < &0,\quad \forall e_{y} \ne 0 \\ \mathop {lim}\limits _{\left |{ {e_{y}} }\right |\to \infty }~V=&\infty\tag{5b}\end{align*} To secure the stability within finite time, the following conditions are considered:\begin{equation*} \dot {V}\le -\alpha V^{1/2}\tag{5c}\end{equation*} These are calculated based on a variable separation method and $\alpha $ can be derived with the assumption that the cost function reaches zero within a finite time through the following relation:\begin{equation*} t_{f} \le -\frac {2V^{1/2}(0)}{\alpha }\tag{5d}\end{equation*} From equation (5d), $\alpha $ can now be defined. Based on equations (5a) and (5c), the Lyapunov term can be given as:\begin{equation*} \dot {V}\le -e_{y} \dot {e}_{y}\tag{5e}\end{equation*} Equation (5e) can be rewritten using equation (3b) as follows:\begin{align*} \dot {V}\le&-e_{y} (A_{t21} e_{1} +A_{t22} e_{y})-\rho \left |{ {e_{y}} }\right | \tag{5f}\\ < &-\left |{ {e_{y}} }\right |(\rho -\left |{ {A_{t21} e_{1} +A_{t22} e_{y}} }\right |) \\ \dot {V}\le&-\frac {\alpha }{\sqrt {2}}\left |{ {e_{y}} }\right |\tag{5g}\end{align*} where $\alpha $ is predefined from equation (5d). The parameter $\rho $ is designed from equations (5f) – (5g). The boundary conditions are described as follows:\begin{equation*} \left |{ {A_{t21} e_{1} +A_{t22} e_{y}} }\right |\le L_{b}\tag{5h}\end{equation*} where $\rho $ is designed as follows:\begin{equation*} \rho =L_{b} +\frac {\alpha }{\sqrt {2}}\tag{5i}\end{equation*} Next, boundary condition, $L_{b} $ , needs to be determined in consideration of the fault magnitude, obtained from the experiment. Through parameter $\rho $ , the output error of the sliding mode observer could converge to zero. $L_{b} $ represents the boundary value that includes acceleration fault information from equation (5h). Note that $L_{b} $ is a time-varying value. An MIT rule-based adaptive algorithm to design the magnitude of the injection term parameter is proposed in a later subsection. The adaptation method is utilized to account for unexpected values of acceleration error boundary.

C. Adaptation Algorithm Based on the MIT Rule

This subsection introduces the utilization of an MIT adaptive rule that does not require system model parameters. Here, estimated coefficients and MIT rules were used to update feedback gain. The parameter $\rho $ is designed using the Lyapunov function under the assumption of boundary condition $L_{b} $ . However, considering the nature of the acceleration fault, we cannot forgo the possibility that the error boundary itself could be the unexpected value. This indicates that the conventional sliding mode observer has a fixed parameter and cannot be adjusted for fault reconstruction. The adaption rule methodology is proposed to overcome this limitation of the algorithm.

The equations for the cost function $J(\hat {\theta }(k),k)$ of the recursive least square term, optimal gain $L_{g} $ , covariance $P$ , and forgetting factor $\lambda $ are given as follows: \begin{align*} \min \,J(\hat {\theta }(k),k)=&\frac {1}{2}\sum \limits _{i=1}^{k} {\lambda ^{k-i}(y(i)-\phi (i)\hat {\theta }(k))^{2}}\tag{6a}\\ \hat {\theta }(k)=&\hat {\theta }(k-1)+L_{g} (k)(y(k)-\phi (k)\hat {\theta }(k-1)) \\ \tag{6b}\\ L_{g} (k)=&P(k-1)\phi (k)(\lambda +\phi ^{T}(k)P(k-1)\phi (k))^{-1} \\ \tag{6c}\\ P(k)=&(I-L_{g} (k)\phi ^{T}(k))P(k-1)/\lambda\tag{6d}\end{align*} Here, $\hat {\theta }(k),k,y(i),\phi (i)$ , $\phi ^{T}(i)$ , and $\lambda $ are estimated terms at the k-th step, time step number, output for RLS, regressor, transpose of regressor, and forgetting factor, respectively. The MIT rule-based cost function $J_{e} $ and $J_{p} $ , designed for the adaptation algorithm, are given as follows: [22], [23] \begin{align*} J_{e}=&\frac {1}{2}e_{y}^{2},\quad J_{p} =\frac {1}{2}e_{p}^{2}\tag{7a}\\ \frac {d\rho }{dt}=&-\gamma _{e} \frac {dJ_{e}}{d\rho }=-\gamma _{e} e_{y} \frac {\partial e_{y}}{\partial \rho }=-\gamma _{e} e_{y} \hat {c}_{e} \\ \frac {d\rho }{dt}=&-\gamma _{p} \frac {dJ_{p}}{d\rho }=-\gamma _{p} e_{p} \frac {\partial e_{p}}{\partial \rho }=-\gamma _{p} e_{p} \hat {c}_{p}\tag{7b}\end{align*} where $\gamma _{e} $ and $\gamma _{p} $ are adaptation gains designed to increase and decrease the $\rho $ term. Here, $\hat {c}_{e} $ and $\hat {c}_{p} $ are coefficients estimated from $J_{e} $ and $J_{p} $ . $e_{p} $ refers to the error of $\rho $ that can be simply described as $\hat {\rho }-\rho $ . The equations (7a) and (7b) can be integrated into one equation using a weighting factor $w$ . The integrated equation determines the magnitude of the output error and is given as follows: \begin{align*} \frac {d\rho }{dt}=&-w\gamma _{e} e_{y} \hat {c}_{e} -(1-w)\gamma _{p} e_{p} \hat {c}_{p}\tag{8a}\\ \rho=&\int _{0}^{t} \left ({{-w\gamma _{e} e_{y} \hat {c}_{e} -(1-w)\gamma _{p} e_{p} \hat {c}_{p}} }\right)dt\tag{8b}\\ w=&\begin{cases} \displaystyle 1, & \left |{ {e_{y}} }\right |\ge \varepsilon \\ \displaystyle 0, & \left |{ {e_{y}} }\right | < \varepsilon \\ \displaystyle \end{cases}\tag{8c}\end{align*} Here, $\varepsilon $ represents a small positive value and a criterion for determining whether an increase or decrease in parameter $\rho $ is necessary, depending on the status of the output error affected by a fault. $w$ represents a switching factor that changes (increase or decrease) the parameter $\rho $ , which is updated through equation (8b). From equation (8b), it is evident that $\rho $ is dependent on output error $e_{y} $ . Therefore, the primary purpose of the design adaptation algorithm is satisfied at this point. The following subsection describes the methodology for fault detection in environmental sensors.

D. Linear Model Prediction-Based Environment Sensor Fault Detection

This subsection introduces the linear model prediction algorithm for environmental sensor fault detection. A predictive and accumulative method is used to derive the feasible boundary of the current measured state. The same longitudinal driving kinematic model in equations (1a) – (1c) has been used. The discretized kinematic model is used for prediction and is given as follows: \begin{align*} x(k+1)=&A_{d} x(k)+B_{d} u(k)\tag{9a}\\ A_{d}=&\left [{ {{\begin{array}{cc} 0 &\quad {\Delta t} \\ 0 &\quad 0 \\ \end{array}}} }\right]\!,\quad B_{d} =\left [{ {{\begin{array}{c} 0 \\ {\Delta t} \\ \end{array}}} }\right]\tag{9b}\end{align*} Here, $\Delta t,A_{d} $ , and $B_{d}$ represent the discrete-time interval, the discretized system matrix, and the discretized input matrix, respectively. The discretized system model-based state could be predicted linearly through the following equation:\begin{equation*} x(k+N)=A_{d}^{N}x(k)+\sum \limits _{i=0}^{N-1} {A_{d}^{i}B_{d} u(k+N-1-i)}\tag{9c}\end{equation*} where the value of $N$ is 20 (indicating a 2 second interval). Vehicle tests have determined the 2 second time frame to be an appropriate estimation time.

In order to consider the predicted input term $\sum \limits _{i=0}^{N-1} u(k+N- 1-i) $ of equation (9c), an acceleration value (maximum or minimum) is applied to a differential acceleration value, defined as $Jerk$ . Equation (9c) can then be divided into two separate equations for prediction, utilizing the maximum or minimum acceleration information instead of the predicted input term. The equations are given as follows:\begin{align*} x_{u} (k+N)=&A_{d}^{N}x(k)+\sum \limits _{i=0}^{N-1} {A_{d}^{i}B_{d} u(k)} \\&+\,\sum \limits _{i=0}^{N-2}{(N-i-1)A_{d}^{i}B_{d} \Delta u_{u}} \tag{9d}\\ x_{l} (k+N)=&A_{d}^{N}x(k)+\sum \limits _{i=0}^{N-1} {A_{d}^{i}B_{d} u(k)} \\&+\,\sum \limits _{i=0}^{N-2} {(N-i-1)A_{d}^{i}B_{d} \Delta u_{l}} \tag{9e}\\ \Delta u_{u}=&u(k)+Jerk\cdot \Delta t, \\ \Delta u_{l}=&u(k)-Jerk\cdot \Delta t\tag{9f}\end{align*} Here, $x_{u} (k+N)$ and $x_{l} (k+N)$ represent the predicted upper and lower limit values. The state is predicted from the first to the N-th step through the equations above. The predictions are then accumulated for precision. Terms $\Delta u_{u} $ and $\Delta u_{l}$ represent the upper and the lower bound of input difference respectively. Fig. 4. visualizes the linear prediction algorithm’s concept; each step’s lower subscripts indicate the past step from which the prediction was made. The present time shows an accumulation of predicted states (red point in Fig. 4). Predicting the states of $x_{u} $ and $x_{l} $ allows the derivation of the $N-1$ feasible boundary between $x_{u} $ and $x_{l} $ . A deviation of the current state from the feasible boundary derived by the accumulation of predicted states (Equation (9f)) is regarded as an environment sensor fault. The final decision methodology for detecting an environment sensor fault after applying the described method is given below. Fig. 4 shows that the current index determines a fault by comparing the value predicted over 20 steps with the current measured value. The clearance fault index $F_{1,t} $ , relative velocity fault index $F_{2,t} $ , and fault sum average index $F_{s} $ are given as follows: \begin{align*} F_{1,i}=&\begin{cases} \displaystyle 1, & x_{1,k-i,u} (k) < x_{1,k} \,\,or\,\,x_{1,k-i,l} (k) > x_{1,k} \\ \displaystyle 0, & x_{1,k-i,u} (k)\ge x_{1,k} \ge x_{1,k-i,l} \end{cases} \\ \tag{10a}\\ F_{2,i}=&\begin{cases} \displaystyle 1, & x_{2,k-i,u} (k) < x_{2,k} \,\,or\,\,x_{2,k-i,l} (k) > x_{2,k} \\ \displaystyle 0, & x_{2,k-i,u} (k)\ge x_{2,k} \ge x_{2,k-i,l} \end{cases} \\ \tag{10b}\\ F_{s}=&\left ({{\sum \limits _{i=1}^{N} {(F_{1,i} +F_{2,i})}} }\right)/N\tag{10c}\end{align*}

FIGURE 4.

Concept of linear model prediction algorithm & Environment sensor fault detection method using predicted state.

The final fault index is derived using the threshold approach as follows:\begin{align*} F_{x} =\begin{cases} \displaystyle 1 & (F_{s} \ge Th_{s}) \\ \displaystyle 0 & (F_{s} < Th_{s}) \\ \displaystyle \end{cases}\tag{10d}\end{align*} $F_{x}$ and $Th_{s}$ represent the final fault index and threshold of the fault sum average index $F_{s} $ . The algorithms presented above have been proposed as a methodology for fault detection of acceleration and environment sensors in the longitudinal direction.

E. Vehicle Test Results of Fault Detection Algorithm

This subsection introduces vehicle test scenarios, the test environment, and test results. Table 2 summarizes the proposed fault detection algorithm for the environment sensor and the vehicle chassis sensor. The designed fault scenario is distinguished as follows: Three scenarios were designed as shown in Table 2. A square wave fault was used with different magnitudes for acceleration sensors. Faults with the environmental sensors are designed as fault for $x_{1} $ , fault for $x_{2} $ , and fault for $x_{1} $ and $x_{2} $ simultaneously. In the last case in which faults of the environment sensor and acceleration sensors occur simultaneously, the sliding mode observer could not be converged using the wrong state information. To get reasonable results, the vehicle test was carried out using the results from the environment sensors (lidar, radar), the RT range, and the RT differential GPS equipment.

A vehicle test situation is designed as shown Fig. 5 to utilize equation (1c) in the kinematic model. The parameters for adaptive sliding mode observer and linear prediction algorithm are described in Table 3.

FIGURE 5.

Vehicle test situation description & test snapshot.

In Table 2, some results are not depicted in this paper. Some results [14], [38], [39] are not in this paper but get similar results as shown here. The omitted graphs are replaced by the resulting graph in the reference paper.

Two small and large fault signals are applied to acceleration information to evaluate the performance of the adaptation algorithm for the sliding mode observer. The results from applying a large fault are shown in Fig. 6 for comparison. The results from applying a small fault are shown in reference papers [14], [39]. Fig. 6 (a) shows comparison of results of applied and reconstructed fault between the timestamps 25 sec ~ 35 sec. From the applied fault plot, it can be observed that output error results are large. A switch in the weighting factor to 1 from 0 can be seen as well. The parameter $\rho $ is updated using the estimated coefficient from output error. Even though output error represents trembling on the region with applied fault in Fig. 6 (b), it maintains stability using the adaptive parameter in the injection term. The switched weighting factors are shown in Fig. 6 (c) and Fig. 6 (d). The estimated coefficients are shown in Fig. 6 (e) and Fig. 6 (f) used for the adaptation algorithm. Based on reconstructed fault, the acceleration fault has been detected as shown in Fig. 6 (g). Fig. 6 shows the scenario applied to a relatively large acceleration fault. The results are similar to the results of the small fault scenario. However, the results of the adaptive parameter in Fig. 6 (c) are more significant than small fault scenarios. This indicates that the method has a large output error from a large acceleration fault, shown in Fig. 6 (b). Estimated coefficients are also shown in the large estimation changing rate (compare Fig. 6 (e) and (f)). Therefore, the applied fault can be reconstructed using a switched weighting factor based on adaptive parameters. Despite the large acceleration fault, the performance of fault

FIGURE 6.

The results of adaptive sliding mode observer: large magnitude of fault.

reconstruction and detection was checked and is secure (Fig. 6 (a) and (g)). A large value is seen because it is the initial convergence of observer output error. A high adaptive gain can be defined to ensure initial convergence performance, but the results are over a relatively large adjustment of parameters and can lead to unreasonable results. Fig. 7 shows the fault detection results for environment sensors based on the linear prediction method. Three test scenarios were considered in Table 2 for reasonable performance evaluation: clearance fault, relative velocity fault, and simultaneous faults in both sensors. The square wave fault was induced at the 20 second mark. The fault-sum-average index $F_{s} $ becomes larger at around the 20 second mark. However, around 0 seconds, an increase in the fault-sum-average index can be observed as well. Such faults at the initial interval are ignored in the vehicle test.

FIGURE 7.

The results of clearance fault detection.

SECTION IV. Vehicle Actuator System Model

The test vehicle performs longitudinal control by applying the desired longitudinal acceleration commands to the vehicle gateway. The gateway connects the lower controller and the vehicle system through a CAN communication platform to achieve vehicle behavior that follows the desired command. As depicted in Fig. 8, the vehicle actuator system can be defined as a dynamic system consisting of a gateway and a vehicle system. The dotted line in Fig. 8 represents a schematic diagram of an actual vehicle actuator system. For the actuator system, the desired acceleration command is the input, and the actual acceleration of the vehicle is the output. Consider the case of an emergency braking scenario. The emergency control corresponds to a command is continuously applied to the gateway. e.g., negative acceleration. The vehicle then decelerates while tracking the braking input command.

FIGURE 8.

Representation of vehicle actuator system of the test vehicle.

From a vehicle control perspective, immediate responses from the actuator system are critical in ensuring that the desired motion planning gives an acceptable performance. An ideal actuator system outputs a vehicle acceleration response instantly when an acceleration input command is given. However, response delays exist in realistic vehicle actuator systems. This is due to the vehicle’s inherent characteristics, such as body weight and powertrain performance. These actuator delays lead to an accumulation of error within the control objective, resulting in an overshoot response to compensate for the error. Therefore, actuator response characteristics need to be accounted for to reduce the disparity between the desired and actual performance.

The system’s response characteristics can be captured in a simple dynamic model. The First Order Plus Dead Time (FOPDT) model can aptly define the vehicle actuator system as follows:\begin{equation*} \dot {a}_{x} (t)=-\frac {1}{\tau }a_{x} (t)+\frac {1}{\tau }a_{x,des} (t-t_{d})\tag{11}\end{equation*} Here, $a_{x} $ represents the actual longitudinal acceleration, $a_{x,des} $ represents the desired longitudinal acceleration command, $\tau $ is a time constant, and $t_{d} $ represents the dead time. By fitting the time series data of the control command and actual sensor measurement against the FOPDT model, model parameters could be obtained. The reference data was obtained from actual vehicle tests in order to evaluate the step response of the vehicle actuator system with different step command scenarios. The model parameters were obtained using the MATLAB system identification tool, which estimates the parameters of a Laplace transfer function from a given time series data of input and output. The Laplace transfer function model corresponding to (14) in the frequency domain is given as follows:\begin{equation*} G(s)=\frac {a_{x} (s)}{a_{x,des} (s)}=\frac {e^{-t_{d} s}}{1+\tau s}\tag{12}\end{equation*}

$a_{x} (s)$ represents the Laplace transform of $a_{x} (t)$ , and $a_{x,des} (s)$ represents the Laplace transform of $a_{x,des} (t)$ . Each braking and acceleration command results in responses with differing characteristics. Hence the representative model parameters need to be estimated individually for each command scenario. From a control perspective, the scenario of interest is of the vehicle braking. Hence, system identification was mainly conducted for a braking scenario.

Fig. 9 shows a comparison of the step input responses between the actual vehicle and a simulation based on the FOPDT model with the estimated parameters. The comparison shows that the FOPDT model has a satisfactory characterization of the vehicle response with the estimated parameters. When a step command input was given, the Root Mean Squared Error (RMSE) value between the actual and the simulated acceleration results was less than 0.12 m/s². Fig. 9 shows that the FOPDT model with the appropriate parameters can accurately describe the actuator response’s overall characteristics. The corresponding parameters for the acceleration and braking scenario were found to be $(\tau,t_{d})=(0.5s,\,0.8s)$ and $(\tau,t_{d})=(0.17s,\,0.25s)$ , respectively. By utilizing the FOPDT model with the estimated parameters, the characteristics of the actuator response could be accounted for within vehicle system model design of the emergency control system.

FIGURE 9.

Acceleration results of vehicle actuator system (Step response).

SECTION V. Fail-Safe Emergency Control – Adaptive Sliding Mode Control Based (Longitudinal, Module in Vehicle)

A. Overall Hardware Structure

The hardware concept schematic of the autonomous vehicle controllers (PC, Autobox), inclusive of fail-safe algorithms, is depicted in Fig. 10. The fail-safe module and the perception, decision and control algorithms in the upper controller, were configured under typical environmental circumstances, in consideration of the autonomous vehicle’s hardware structure. If the fault detection module detects a fault and no driver intervention is given, the last available information is used to predict and control the system. The drive-able path information received from the upper controller is utilized in two ways: in the lateral direction and the longitudinal direction. In the lateral direction, the dead reckoning algorithm utilizes the distance information to calculate the appropriate steering angle. Lateral control only utilizes the vehicle chassis information. This algorithm uses the last information (desired path) available from the upper controller to follow the desired path using the DR method. In the longitudinal direction, reference target building and sliding mode control-based deceleration algorithms are executed. Longitudinal control works by calculating the reference model based on the received drive-able path information. Information is delivered to the lower controller in real-time, and the control constantly operates in the vehicle. The entire module is composed of a fault detection portion that detects fault within the total module. The module is comprised of a fault detection part that classifies the fault and a control model that controls deceleration with limited information.

FIGURE 10.

Failsafe hardware concept diagram.

B. Reference Deceleration Model Rebuilding and Filtering

The typical reference deceleration model is made from general driver deceleration data [24]. The typical model is described in Fig. 10 under the fail-safe control portion. The model considers driver safety and ride comfort. A first-integrated velocity model and a second-integrated station model are used to construct an algorithm for stopping at safe distances. Images and formulas for the longitudinal acceleration, the longitudinal velocity, and the longitudinal distance models are illustrated in Fig. 11. Here, $\mathbf {V}_{\mathbf {0}}$ represents initial velocity, $\mathbf {a}_{\mathbf {m}}$ represents maximum used deceleration, $\boldsymbol {\theta }$ represents time ratio, $\mathbf {t}_{\mathbf {d}}$ represents deceleration time, $\mathbf {m}$ represents a model variable parameter, and $\mathbf {r}$ represents a model parameter. Detailed information and model parameters regarding the reference model have been introduced in various studies [24]. The authors of [24] utilized normal driver deceleration data to build a deceleration model. Driver data-based models such as the one built in [24] may reflect the subject driver’s driving habits. Consequently, it has been shown in a simulation study that some reference models show characteristics that result in an uncomfortable feeling for the driver. The applied reference model was a function of time. This implies that the reference model is calculated in real-time in the event of a fault that requires emergency deceleration control.

FIGURE 11.

Deceleration reference model [24].

This paper has adopted a new method to rebuild the reference model offline, according to the vehicle’s velocity. A reference model for each velocity was first created. Next, the Carsim simulator was used to evaluate the reference model’s three main index types. In this paper, the proposed indexes regarding safety and comfort are described in two parts. Under safety, the indexes utilized are the time to collision inverse (TTC inv) index and longitudinal warning index. Under comfort, the indexes utilized are pitch, pitch rate, and vertical acceleration. More detail regarding the safety indexes can be found in [25], [26]. The index proposed here is shown in Fig. 12. For Comfort indexes, this study used the passenger’s ride comfort data and indexes from a related paper [27]. In [27], jerk value and vertical acceleration are proposed as riding comfort indexes. Other indexes utilized in [27] are shown in Fig. 13. Table 4 summarizes the indexes regarding safety and comfort. For this paper, a new index was developed through a combination of the indexes investigated.

TABLE 4 Error Analysis

FIGURE 12.

AEB & longitudinal safety index [25], [26].

FIGURE 13.

Riding comfort evaluation in paper [27].

C. New Index-Based Reference Model Rebuilding and Filtering

This section introduces a methodology for filtering out improper reference models using the newly proposed indexes. Fig. 11 shows an example of an improper reference model [24]. Reference target models were first generated according to the velocity at the moment of vehicle failure. Filtering of these models using safety and omfort indexes was carried out to generate a final filtered reference model.

Through a Carsim simulation test, the pitch, vertical acceleration, and jerk value could be derived. In the Carsim simulation, each test vehicle was driven with the velocity and station of each reference model. A total of 78 derived cases were carried out. The Safety and Comfort indexes used and referred to are presented in Table 4, and the derived results are shown in Figs. 14 to 16. The guidelines for filtering are as follows: 1. overshot of indexes occurs when its value is not within a reasonable range and the passenger is deemed to be uncomfortable [27]. The filtered referenced model used for the vehicle tests is shown in Fig. 17. The model was constructed for vehicle tests of velocities between 30-50 kph, shown in Fig. 17

FIGURE 14.

Generated reference model filtering: pitch rate, warning index, and TTC inverse.

FIGURE 15.

Generated reference model filtering - pitch, warning index, TTC inverse.

FIGURE 16.

Vertical acceleration & Jerk – model filtering index.

FIGURE 17.

Filtered reference model.

D. Sliding Mode Control - Adaptive Converge Time Gain and Stability

This subsection proposes a control methodology applicable for the fail-safe control portion in an autonomous vehicle. Reasons for the usage of the sliding mode control method and the adaptive convergence time method are detailed in this section. To ensure a strict level of safety with the designed conditions, knowledge regarding response time to a failure is vital. Therefore, the convergence time of the control error was determined mathematically, and the corresponding control values were designed to be changed based on the predetermined convergence time. Thus, this particular method was applied to the test vehicle within the fail-safe control portion. This subsection demonstrates the stability of the longitudinal emergency braking system. The proof of stability is also shown in this subsection. In the fail-safe control part, the longitudinal model is a nonlinear system without any disturbance and uncertainties. The model can be described as follows: \begin{equation*} \dot {x}(t)=f(x,t)+g(x,t)\cdot u(t)\tag{13a}\end{equation*}

From section IV (vehicle actuator module system identification), it was determined that the actuator system followed a first-order delay system [35]–​[37]. We define the longitudinal model and error dynamics as follows:\begin{align*} \dot {x}_{long} (t)&=Ax(t)+Bu_{long} (t)\tag{13b}\\ A&=\left [{ {{\begin{array}{ccc} 0 &\quad 1 &\quad 0 \\ 0 &\quad 0 &\quad 1 \\ 0 &\quad 0 &\quad {-1/\tau _{ax}} \\ \end{array}}} }\right]\!,\quad B=\left [{ {{\begin{array}{c} 0 \\ 0 \\ {1/\tau _{ax}} \\ \end{array}}} }\right]\tag{13c}\\ e_{1} (t)&=x_{1,ref} (t)-x_{1} (t)=s_{x,ref} (t)-s_{x} (t)~ \\ e_{2} (t)&=x_{2,ref} (t)-x_{2} (t)=v_{x,ref} (t)-v_{x} (t)~ \\ e_{3} (t)&=x_{3,ref} (t)-x_{3} (t)=a_{x,ref} (t)-a_{x} (t)~\tag{14}\end{align*} Here, $u_{long} (t),\tau _{ax} $ , and $k$ represent the control input, time constant, and estimated $s_{x,ref},v_{x,ref},a_{x,ref} $ . The estimated model parameters were $(\tau,t_{d})=(0.5s,\,0.8s)$ and $(\tau,t_{d})=(0.17s,\,0.25s)$ in accelerating and braking scenarios, respectively. Considering the system (13b, 13c) and reference model, error states can be defined as shown in (14). The time derivative of the error dynamics are derived as follows:\begin{align*} \dot {e}_{1} (t)=&\dot {x}_{1,ref} (t)-\dot {x}_{1} (t)=e_{2} (t)+\dot {x}_{1,ref} (t)-x_{2,ref} (t)~ \\ \dot {e}_{2} (t)=&\dot {x}_{2,ref} (t)-\dot {x}_{2} (t)=e_{3} (t)+\dot {x}_{2,ref} (t)-x_{3,ref} (t)~ \\ \dot {e}_{3} (t)=&-\frac {e_{3} (t)}{\tau _{ax}}-\frac {k}{\tau _{ax} }u_{long} +\dot {x}_{3,ref} (t)-\left({-\frac {x_{3,ref} (t)}{\tau _{ax}}}\right) \\\tag{15}\end{align*} The sliding surface term and the first time-derivative term were defined as follows:\begin{align*} s(t)=&e_{1} (t)-\lambda _{1} e_{3} (t)~ \\ \dot {s}(t)=&\dot {e}_{1} (t)-\lambda _{1} \dot {e}_{3} (t)\tag{16}\end{align*} The control input term $u_{long} $ is related to $e_{3} $ , and the sliding surface term contains $e_{3} $ . The first-time derivative of the sliding surface is defined as follows:\begin{align*} \dot {s}(t)=&\dot {e}_{1} (t)+\lambda _{1} \dot {e}_{3} (t)~ \\=&\underbrace {e_{2} (t)+\dot {x}_{1,ref} (t)-\dot {x}_{3,ref} (t)}_{\dot {e}_{1} (t)} \\&+\,\lambda _{1} \cdot \underbrace {\left [{ {\begin{array}{l} -\displaystyle \frac {1}{\tau _{ax}}\cdot e_{3} (t)-\displaystyle \frac {k}{\tau _{ax}}\cdot u_{long} (t)~\\ +x_{3,ref} (t)+\displaystyle \frac {1}{\tau _{ax}}\cdot x_{3,ref} (t)~\\ \end{array}} }\right]}_{\dot {e}_{3} (t)}\tag{17}\end{align*} We define the Lyapunov function candidate as follows:\begin{equation*} V(t)=\frac {1}{2}s(t)^{2}\tag{18}\end{equation*} We define first-derivative term $\dot {V}(t)$ as follows. It is to be noted that this term is always negative.\begin{align*} \dot {V}(t)=&s(t)\dot {s}(t)=s(t)\left [{ {\dot {e}_{1} (t)+\lambda _{1} \dot {e}_{3} (t)} }\right] \\=&\left |{ {s(t)} }\right |\cdot \left [{ {\begin{array}{l} \underbrace {\begin{array}{l} e_{2} (t)+\dot {x}_{1,ref} (t)-\dot {x}_{2,ref} (t)-\displaystyle \frac {\lambda _{1} }{\tau _{ax}}\cdot \\ e_{3} (t)+\lambda _{1} x_{3,ref} (t)+\displaystyle \frac {\lambda _{1}}{\tau _{ax}}\cdot x_{3,ref} (t)~\\ \end{array}}_{R_{0} \left [{ {x(t)} }\right]} \\ +\left ({{\underbrace {-\frac {k\lambda _{1}}{\tau _{ax}}}_{Q(\lambda _{1})}\cdot u_{long} (t)} }\right) \\ \end{array}} }\right] \\\tag{19}\end{align*} If the following condition is satisfied, \begin{align*} \underbrace {\left |{ {\begin{array}{l} e_{2} (t)+\dot {x}_{1,ref} (t)-\dot {x}_{2,ref} (t)-\displaystyle \frac {\lambda _{1} }{\tau _{ax}}\cdot \\ e_{3} (t)+\lambda _{1} x_{3,ref} (t)+\displaystyle \frac {\lambda _{1}}{\tau _{ax}}\cdot x_{3,ref} (t)~\\ \end{array}} }\right |}_{R_{0} [x(t)] }\le R\left [{ {x(t)} }\right]\tag{20}\end{align*} $R_{0} [x(t)]$ is bounded by $R[x(t)]$ . The $\dot {V}(t)$ equation then satisfies the following inequality equation.\begin{align*} \dot {V}(t)\le&\left |{ {s(t)} }\right |\cdot R\left [{ {x(t)} }\right]-\left |{ {s(t)} }\right |\cdot \left ({{-\frac {k\lambda _{1}}{\tau _{ax}}\cdot u_{long} (t)} }\right) \\=&\left |{ {s(t)} }\right |\cdot Q(\lambda _{1})\cdot \left [{ {\frac {R\left [{ {x(t)} }\right]}{Q(\lambda _{1})}} }\right]+Q(\lambda _{1})\cdot s(t)\cdot u_{long} (t) \\\tag{21}\end{align*}

From equation (19) and (20), one can obtain a longitudinal control input and an equivalent control input as follows:\begin{align*} u_{long} (t)=&-\rho _{m} sgn(s(t)) \\ u_{eq} (t)=&-\rho _{m} sgn(s(t))-K\cdot sgn(s(t)) \\ \rho _{m}=&\left [{ {\frac {R\left [{ {x(t)} }\right]}{Q(\lambda _{1})}+\alpha } }\right]\!,\quad K > 0,~\alpha > 0\tag{22}\end{align*} Here $u_{long} (t)$ , $u_{eq} (t)$ and $\rho _{m} $ represent the control input, the equivalent control input, and the adaptation parameter for control input magnitude, respectively. From equation (21), the equality equation of $\dot {V}(t)$ can be defined as follows:\begin{align*} \dot {V}(t)\le&\left |{ {s(t)} }\right |\cdot R\left [{ {x(t)} }\right]-s(t)[-Q(\lambda _{1})]\cdot u_{eq} (t)~ \\=&\left |{ {s(t)} }\right |\cdot R\left [{ {x(t)} }\right]-s(t)\cdot R\left [{ {x(t)} }\right]\cdot sgn[s(t)] \\&-\,Q(\lambda _{1})\cdot K\cdot \left |{ {s(t)} }\right |-\alpha \cdot K\cdot \left |{ {s(t)} }\right | \\=&-Q(\lambda _{1})K\sqrt {2V(t)} -\alpha K\sqrt {2V(t)}\tag{23}\end{align*} where $Q(\lambda _{1})$ , $K$ , and $\alpha $ are all positive values. The design $W(t)$ is as follows:\begin{equation*} W(t):=\sqrt {2V\{s[e(t)]\}}\tag{24}\end{equation*} The value of $W(t)$ is defined so that the Lyapunov function candidate converges to zero within finite time.\begin{align*} \frac {dW(t)}{dt}=&\frac {2\dot {V}(t)}{2\sqrt {2V(t)}}\le \frac {-2\sqrt {2V(t)} Q(\lambda _{1})\cdot [K+\alpha]}{2\sqrt {2V(t)}} \\=&-Q(\lambda _{1})\cdot [K+\alpha]\tag{25}\end{align*} Using the Comparison Lemma found in [5] and [20], the following equation can be derived.\begin{equation*} \dot {V}(t) < -\alpha V^{1/2}(t),\quad \alpha > 0\tag{26}\end{equation*} Variables separating and integrating time interval $\tau _{int} $ from 0 to $t_{converge} $ is given as follows: \begin{align*} t_{converge}\le&\frac {2V^{1/2}(t_{0})}{\alpha }=t_{bound}\tag{27a}\\ V^{1/2}(t_{0})=&\frac {1}{2}s(t_{0})=\frac {1}{2}[e_{1} (t_{0})+\lambda _{1} e_{3} (t_{0})]\tag{27b}\end{align*} From equation (27a), an inverse relation between $t_{converge}$ and $\alpha $ can be observed. Therefore, from equation (27a), as $\alpha $ becomes larger, $V(t)$ converges to zero faster. \begin{align*} u_{eq} (t)=&\underbrace {\left [{ {R\left [{ {x(t)} }\right]+\frac {\alpha }{\sqrt {2}}} }\right]}_{\rho _{m}}\cdot sgn[s(t)]-K\cdot sgn[s(t)] \\ \tag{28a}\\ R_{0} [x(t)]=&\left ({{\begin{array}{l} e_{2} (t)+\dot {x}_{1,ref} (t)-\dot {x}_{2,ref} (t)-\displaystyle \frac {\lambda _{1} }{\tau _{ax}}\cdot \\ e_{3} (t)+\lambda _{1} x_{3,ref} (t)+\displaystyle \frac {\lambda _{1}}{\tau _{ax}}\cdot x_{3,ref} (t)~\\ \end{array}} }\right)\tag{28b}\end{align*} However, equation (28a) shows that an increased magnitude of $\alpha $ could result in a more substantial chattering phenomenon of the control. The eta-reachability condition in equation (28b) can be easily defined from the initial condition of error and occurs instantaneously. Thus, equation (28c) can be rewritten as follows:\begin{align*} R_{0} [x(t_{0})]=&\left ({{\begin{array}{l} e_{2} (t_{0})+\dot {x}_{1,ref} (t_{0})-\dot {x}_{2,ref} (t_{0})-\displaystyle \frac {\lambda _{1}}{\tau _{ax}}\cdot \\ e_{3} (t_{0})+\lambda _{1} x_{3,ref} (t_{0})+\displaystyle \frac {\lambda _{1}}{\tau _{ax}}\cdot x_{3,ref} (t_{0}) \\ \end{array}} }\right) \\\le&R[x(t)]\tag{28c}\end{align*} If an upper decision controller determines a need for emergency deceleration, then the exact value of $R_{0} [x(t_{0})]$ can be determined at the moment of the error’s occurrence. Equations (27a, 27b) show that the magnitude of $\alpha $ is adaptively defined, and it immediately has a proportional effect on the magnitude of $\rho _{m} $ , evident from equation (28a). However, there could be a case in which the inequality condition becomes an equality, as shown in equation (29).\begin{equation*} t_{converge} =\frac {2V^{1/2}(t_{0})}{\alpha }=t_{bound}\tag{29}\end{equation*} From the eta-reachability condition, \begin{equation*} \rho _{m} \ge R\left [{ {x_{t} (t)} }\right]+\frac {\alpha }{\sqrt {2}}+\eta\tag{30}\end{equation*} $\rho _{m} $ is large enough such that the condition $\eta \in {\textbf {R}}_{+} $ is satisfied. Thus, it can be inferred that $s(t)$ will converge to zero within finite time. The mathematical proof for the above convergence in finite time allows us to derive the main contribution of this algorithm. This is also the main idea and unique concept behind the algorithm. The bounded time can be defined by various vehicle experiments. Due to the mathematical derivation of $t_{bound}$ values, the logic behind the control guarantees that the time error is at always less than $t_{bound} $ when standard control is in operation.

E. Vehicle Validation

The proposed algorithm has been validated through actual vehicle tests on two different testbeds. The first testbed is located in Seoul National University (SNU), Gwanak-gu Seoul-si. A straight road test at the SNU campus beltway testbed was validated. The second testbed is located in the Future Mobility Technology Center (FMTC), Gyeonggi-do Siheung-si Seoul National University. On the FMTC testbed, we validated the system with test scenarios inclusive of curved roads. The testbed environment is depicted in Fig. 18 (Regions highlighted in orange show the main test area).

FIGURE 18.

Testbed overview.

All velocity-specific (30 - 50 kph) test results are shown simultaneously in Fig. 19. The results for station and acceleration errors from designed velocity, ranging from 30 kph to 50 kph, are shown in Fig. 19. Of the two kinds of errors show, most have values within an acceptable range. Detailed error analysis is tabulated in Table 4. However, a high degree of variance in acceleration error can be observed. The right axis of Fig. 19 represents the gear state of the vehicle. A high degree of variance in acceleration error can be observed during a change in gear number. Further detailed and specific changes were obtained through additional experiments, shown in Fig. 20. The three error values exist between the range of −0.1445 and 0.9815. From Fig. 21, it can be observed that these values are within an acceptable range. The value of the lateral axis in Fig. 21 includes numerically meaningful but physically meaningless units. However, the station error has the highest value at around 0.3 m. These values mainly occurred upon switching off the autonomous driving mode. When this fail-safe control methodology was applied to the autonomous vehicle, the control input value changed to a constant negative value when vehicle velocity was close to zero. It is to be noted that the error values beyond 10 seconds in Fig. 20 and Fig. 21 do not hold any meaning. Overall, the vehicle experiments have shown that errors occur within reasonable boundaries.

FIGURE 19.

Vehicle test result with gear value (15 cases, 5 error occurred final vehicle velocity-based case: 30~50 kph case; autonomous driving finished at orange color line).

FIGURE 20.

Acceleration result and gear value.

FIGURE 21.

Vehicle errors result.

The RoA figure shows a locally and asymptotically stable value that could be calculated using the Lyapunov function [5]. Fig. 22 contains two figures. The first background figure represents a contour, indicated mainly in yellow, which is a part of the set and satisfies $\dot {V}_{s=0} < -V_{s=0} $ . The heatmap legend to the right of Fig. 22 shows the derivative value of the Lyapunov function. Non-yellow regions of the contour indicate a part where the differential value of the Lyapunov function is greater than or equal to zero. A darker contour indicates a larger value. The second line figure in red shows a data plot of the actual vehicle experiment. As illustrated in Fig. 22, the test data in the RoA covers a large region on the error plane. Therefore, it can be concluded that the dynamics are locally and asymptotically stable. Five sets of experimental data are simultaneously shown in Fig. 22. Their plot suggests that the data converge to the origin (0,0) at the final point.

FIGURE 22.

Error phase portrait of vehicle result in Lyapunov function value considered background phase.

Throughout the course of this study, our research team faced 2 major challenges with regards to obtaining data. The first challenge was in acquiring real AV fault data. This issue stands with many other previous studies as well. While these studies made assumptions regarding failures and their sizes, our research team overcame this problem through continuous vehicle testing. A large number of tests allowed us to obtain real fault data and determine failure types. The second major challenge was in obtaining vehicle data for SMO research. Obtaining accurate and precise vehicle data for the leading vehicle and the test vehicle proved to be difficult. This problem was mitigated through extensive use of data from the RT-Range.

SECTION VI. Conclusion

In this paper, a newly designed fail-safe structure for autonomous vehicles has been proposed. The main contributions of this paper can be summarized into three parts. First, the framework of the fail-safe module for autonomous vehicles was introduced and implemented to an actual autonomous vehicle. Second, an adaptive sliding mode observer methodology for longitudinal fault detection about chassis sensors and environment sensors (lidar, radar) in autonomous vehicles was presented. An MIT rule-based adaptation rule to determine the magnitude of the observer injection term was proposed for the reconstruction of unknown faults. Third, fault types of autonomous vehicles are defined through vehicle experimentation. The designed logic has been applied to vehicles and operated to manage a fault situation. In the fail-safe control portion, fault was defined as any situation where a shutdown of the upper controller occurs, be it from continuous vibrations or large physical shocks. The lower controller conducts safety control using the last available scene information from the upper controller. The stability of the system was proved, and the proposed algorithm was investigated via actual vehicle tests. The test results have shown that the proposed algorithm was able to perform the desired vehicle actions successfully. Additionally, error variance has shown to converge within acceptable ranges. The vehicle tests have led to the discovery of a fault in autonomous vehicles’ upper controls.

Fault detection can occur in a number of cases, including, but not limited to, sensor performance degradations and adverse weather conditions. Decision tree-based pull-over controls will be the topic of future research in this field for our research team.

For future research, our research them plans to further develop the fail-safe module our automated vehicle. We plan to utilize a LiDAR point-wise de-noising approach to classify and filter out noise in adverse weather conditions. Our team also plans to develop a tolerant control scheme to cover fatal cases of autonomous vehicle faults, excluding actuator faults. Furthermore, an emergency pull-over algorithm for fail-safe systems corresponding to level-4 autonomous driving is currently being investigated, with emphasis on the control method for achieving a safe emergency pull-over maneuver.