Hybrid Particle Filter Trained Neural Network for Prognosis of Lithium-Ion Batteries

Prognostics and Health Management (PHM) plays a key role in Industry 4.0 revolution by providing smart predictive maintenance solutions. Early failure detection and prediction of remaining useful life (RUL) of critical industrial machines/components are the main challenges addressed by PHM methodologies. In literature, model-based and data-driven methods are widely used for RUL estimation. Model-based methods rely on empirical/phenomenological degradation models for RUL prediction using Bayesian formulations. In many cases, the lack of accurate physics-based models emphasizes the need to resort to machine learning based prognostic algorithms. However, data-driven methods require extensive machine failure data incorporating all possible operating conditions along with all possible failure modes pertaining to that particular machine/component, which are seldom available in their entirety. In this work, we propose a three-stage hybrid prognostic algorithm (HyA) combining model-based (Particle Filters-PF) and data-driven (Neural Networks-NN) methods in a unique way. The proposed method aims to overcome the need for accurate degradation modeling or extensive failure data sets. In the first stage, a feedforward neural network is used to formulate lithium-ion battery’s degradation trends and the corresponding NN model parameters are used to define the initial prior distribution of PF algorithm. In the second stage, the PF algorithm optimizes the model parameters and the posterior model parameter distributions are utilized to ‘warm-start’ the neural network used for prognosis and the third/final stages focuses on prognosis and RUL estimation using the trained NN model leveraging on the posterior distributions of the PF fine-tuned weights and biases. The proposed method is demonstrated on CALCE and NASA lithium-ion battery capacity degradation datasets. The efficacy of the proposed hybrid algorithm is evaluated using root mean square error (RMSE) values and alpha-lambda prognostic metrics. Also, the impact of the NN architecture on the prediction accuracy and computational load are analyzed.

SECTION I. Introduction

Electronic devices and systems are subjected to thermal, electrical and mechanical stresses on the field and hence, the reliability of these devices is of utmost concern. Prognostics and health management (PHM) for electronic systems aims to detect, isolate and predict the onset and source of system degradation as well as the time to system failure [1]. In general, prognostic algorithms are categorized as model-based and data-driven methods. Model-based or physics-based methods (as they are often referred to interchangeably) use an accurate degradation model curated for the specific system/component under pre-defined environmental and operating conditions. The prediction accuracy of the degradation model gets compromised due to the variability in the environment/operating conditions or if an individual system/component tends to follow a significantly different degradation trend compared to the rest of the lot due to intrinsic or extrinsic factors that are not fully attributable. Commonly used model-based methods include Kalman filters (KF) [2], particle filters (PF) [3] and adaptations of the aforementioned methods [4]–​[7].

Data-driven methods are alternate approaches to model-based methods as they identify degradation trends in the available degradation (start to end failure) data and use it for prognosis. Typical data-driven methods include support vector machine (SVM) [8], relevance vector machine (RVM) [9], neural networks (NN) [10]–​[12], Gaussian process regression (GPR) [13]–​[15] etc. Though a powerful approach, data-driven methods elevate computational complexity as they require large amount of failure data incorporating all the possible failure modes and operating conditions.

With emerging technologies and advancement in manufacturing processes, new devices/components are being developed to cater to these emerging needs. The major challenges inhibiting reliability studies on such newly developed devices/systems are the lack of sufficient failure data and also, the lack of full-fledged physics-of-failure models. To address the generalization problem, hybrid approaches combining model-based and data-driven methods are being widely used in the recent past. Hybrid prognostic approaches have an upper edge on such new devices as they neither require an accurate degradation model nor a large amount of training data for the purpose of remaining useful life (RUL) estimation. In other words, they make best use of partial knowledge and sparse data available for the new device/component under prognostic investigation.

Hybrid/Fusion Prognostics has become a research hotspot in the recent past. Wang et al. [16] used RVM for sparse representation of the degradation data along with the use of an empirical degradation model for predicting the RUL of the rolling element bearings. Chang and Fang [17] used RVM to determine the measurement noise in lithium-ion battery capacity degradation dataset and apply it for parameter estimation in a PF algorithm. Similarly, Song et al. [18] combined an autoregression (AR) model with PF for RUL estimation on spacecraft lithium-ion battery dataset from the NASA repository. Sun et al. [19] used the extreme learning machine (ELM) approach to construct a degradation model for battery degradation data and used it as the “measurement function” in PF state-space formulation algorithm for RUL estimation.

From the above-mentioned methods, it is evident that PF based approaches are promising for the purpose of RUL estimation. The reason being the ease of its applicability to highly non-linear systems along with non-Gaussian noise present in it. There have also been attempts to use a suitable surrogate data-driven model as the state transition function in PF algorithm over empirical/phenomenological models. The surrogate model formulation can be done using statistical curve fitting methods such as auto regressive integrated moving average (ARIMA). However, ARIMA methods are more suited for linear trends and for short-time horizon predictions. On the other hand, artificial neural networks outperform other statistical approaches due to its versatility to map complex input-output relationships and its ability to identify hidden degradation patterns in the system failure data. One of the first attempts to combine PF and NN for the purpose of RUL estimation was proposed by Baraldi et al. [20] back in 2013. The authors developed an ensemble NN model which creates a large amount of training datasets. These data are then converted into analytical models by data mining techniques and substituted into a PF algorithm as the measurement and state transition functions. The authors succeeded in overcoming the need of using an empirical model in the PF algorithm though the computational complexity and load was enormous compared to conventional methods. Also, the accuracy of the proposed method primarily depended on the amount of training data simulated by the ensemble NN model.

Sbarufatti et al. [21] proposed a self-learning adaptive algorithm to improve the approach proposed in Ref. [20]. The authors proposed a hybrid prognostic algorithm where the radial bias function (RBF) neural network was used as the degradation model for prognosis of lithium-ion batteries. The kernel parameters of the neural network model were estimated through the PF algorithm. Further, the proposed damage model was projected in future to predict the battery capacity and the remaining useful life. The discharge curves of 5 batteries from NASA’s repository were used in this work. The data with the longest curve corresponding to a pristine battery was used for training. Even though the prediction results of the proposed method were promising, the algorithm was tested on a dataset with a slow but progressive aging dynamics. This puts doubt on the adaptability of the algorithm on degradation data due to accelerated aging and unforeseen sudden changes in the degradation behavior.

Since the RBF networks are restricted to one hidden layer and linear output activation functions, the authors further extended their work to adopt a multilayer perceptron (MLP) NN model and tested the approach on NASA and CALCE datasets for estimating the RUL [22]. The authors used an MLP network with 5 hidden neurons to construct the degradation model. The NN degradation model was used in the PF algorithm for predicting capacity degradation. The authors tested the same method on fatigue crack growth data [23] as well. For crack growth data, the algorithm was trained with constant-amplitude degradation curve and tested on changing-amplitude degradation data. The prediction accuracy was poor in the early degradation stages and improved when more than 50% of degradation data was available for prognosis. Wu et al. [24] introduced a bat algorithm for resampling the particles in PF algorithm to improve the performance of MLP + PF hybrid approach. In all of the above-mentioned methods, NN model was used as the damage evolution model in the PF framework and the network parameters were solely dependent on the training dataset. No information from the test dataset was plugged into the algorithm and hence a large amount of data from the test data was required for the algorithm to identify the damage evolution trend.

In this work, we propose an intelligent and adaptive hybrid approach where PF based state estimation is used to warm-start a NN model. The informed stochastic parameter initialization helps to overcome the generalization issues faced by other hybrid approaches in literature. Secondly, the PF optimized NN model is trained with available information from the test dataset using a Levenberg Marquardt (LM) algorithm to find the optimal network parameter values which can be used for prediction of the future states and the RUL as well. The proposed approach was tested on NASA and CALCE lithium-ion batteries capacity degradation datasets and the RMSE values were used as the performance metrics. Our work moving forward is organized as follows. In Section II, the standard NN and PF approach for prognosis are explained and the prediction results using these conventional methods are presented in Section III. In Section IV, the proposed hybrid PF based NN approach is introduced and the prediction results for both NASA and CALCE dataset at different prediction starting points using the proposed hybrid framework are examined in Section V. Also, the impact of the network architecture is discussed as well. Finally, the conclusions of the study and possible scope for future work are summarized in Section VI.

SECTION II. Neural Network and Particle Filter Methodologies

A. Feedforward Neural Networks (FFNN)

In this work, the neural network architecture chosen is a Multi-Layer Perceptron (MLP) with $M$ number of hidden neurons. The MLP model adopted here represents the degradation trend of lithium-ion battery’s discharge capacity with respect to charge/discharge cycles. The number of cycles is fed into the NN model as input. The input node is connected to $M$ neurons in the hidden layer. Each neuron generates an output based on a sigmoidal activation function in the hidden layer as represented below.

$$\begin{equation*} h_{i}=\frac {1}{1+e^{-(w_{i}^{(1)}\ast k+b_{i}^{(1)})}}\tag{1}\end{equation*}$$ where $w_{i}^{(1)}$ and $b_{i}^{(1)}$ are the weight and bias values corresponding to the input node and ${h}$ represents the hidden layer activation function. The charge/discharge cycle index is represented by $k$ and $i = {1,2\ldots,} M$ represents the index for hidden neurons with $M$ being the number of hidden neurons used is the network architecture. The weighted sum of all the hidden neurons gives the predicted battery capacity. A linear activation function is used at the output layer and the overall output of the network can be represented as: $$\begin{align*} g\left(\left ({w,b }\right),k\right)=f\bigg(\sum \limits _{i=1}^{M} {\left(h\left ({\left ({k\ast w_{i}^{\left ({1 }\right)}+b_{i}^{\left ({1 }\right)} }\right)w_{i}^{\left ({2 }\right)} }\right)+b_{i}^{\left ({2 }\right)}\right)} \\\tag{2}\end{align*}$$ where $w_{i}^{(2)}$ and $b_{i}^{(2)}$ are the weight and bias values associated with the hidden layer and $M$ is the number of hidden neurons in the NN network. The network output ${g}(.)$ gives the predicted battery capacity with respect to the cycle index $k$ . Also, ${h}(.)$ is the non-linear sigmoid activation function of the hidden layer and ${f}(.)$ represents the linear output activation function. In a standard FFNN model, the network parameters are optimized for the training dataset by minimizing the mean squared error values using LM algorithm. The trained NN model is used for predicting the battery capacity values on the test dataset.

B. Standard Particle Filter

Particle filters are sequential Monte Carlo (SMC) methods and work on the concept of recursive Bayesian method for state estimation. In this work, the PF algorithm is used for estimating the NN model parameters recursively for a given a set of observations/test data. Ideally, PF algorithm employs empirical or physics-based models for state estimation but in this work, we use the NN degradation model shown in Eqn. (2). Therefore, the state space representation for the system can be expressed as:

$$\begin{align*} x_{k}=&x_{k-1}+\omega _{k-1} \tag{3}\\ z_{k}=&{g(x}_{k},k)+\varepsilon _{k}\tag{4}\end{align*}$$ where $x_{k}$ and $x_{k-1}$ refers to the current and previous state, respectively and $\omega _{k-1}$ is the process noise. The state transition function ${g}(.)$ is the NN degradation model shown in Eqn. (2) and $\varepsilon _{k}$ is the measurement noise. The predicted lithium-ion battery capacity is represented by $z_{k}$ . The particle filtering algorithm consists of two stages $\to$ State Estimation and State Prediction. In the first stage i.e., state estimation, PF recursively estimate the posterior probability distribution of the NN parameters, $x_{k}$ , given a set of test data $z_{1:k}$ where $k$ is the charge/discharge cycle index. At the first time step, $k =1$ , $N_{s}$ number of samples are generated based on the assumed initial prior distribution. For subsequent time steps, posterior distribution of the previous time step (${k}$ -1) is used as the prior distribution for the current time step $(k)$ . Each sample is assigned a weight value $s_{k}$ and the current damage state is transmitted through the state transition function based on a likelihood function to deduce the next damage state. The likelihood function can be expressed as $$\begin{equation*} L\left ({x_{k}\thinspace \vert \thinspace {\Theta }_{k}}\right) =\frac {1}{\sqrt {2\pi } \sigma _{j}}exp\left [{ \frac {-1}{2}\left ({\frac {z_{k}-x_{k}^{j}(\Theta _{k}^{j})}{\sigma _{k}^{j}} }\right)^{2} }\right]\tag{5}\end{equation*}$$ where $z_{k}$ is test data, $x_{k}^{j}$ is the damage state predicted based on the model parameters $\Theta {}_{k}$ . $\Theta {}_{k}$ is the vector of NN hyperparameters. Here, the model parameters are predicted based on the state transition function ${g}(.)$ represented by Eqn. (2). Subsequently, the posterior distribution can then be expressed by: $$\begin{equation*} p\left ({x_{k}\thinspace \vert \thinspace z_{1:k}}\right) =\mathrm { }\sum \limits _{j=1}^{N_{s}} s_{k} \delta (x_{k}-x_{k}^{j})\tag{6}\end{equation*}$$ where $N_{s}$ is the number of samples/particles and each sample is drawn from an initial prior distribution which was obtained based on the user’s knowledge of the system and $\delta (.)$ is the Dirac delta function. It is to be noted that the particle weights, $s_{k}$ , used in Eqn. (6), are different from the NN weight values, $w$ , used in Eqn. (2). The weight of each particle is computed as: $$\begin{equation*} w_{k}=w_{k-1}p(z_{k}\vert x_{k}^{j})\tag{7}\end{equation*}$$ where $p(z_{k}\vert x_{k}^{j})$ is the likelihood of the observation $z_{k}$ . The estimated network parameters are used to project the state transition equation till the end-of-life of the system to predict the future state.

SECTION III. Comparison Between FFNN and Standard PF for RUL Estimation

A. Battery Degradation Datasets

Two different sets of battery degradation data from different laboratory setups are used in this work to evaluate the performance of the proposed hybrid algorithm.

1) CALCE Dataset

Four LiCoO₂ prismatic cells with a rated capacity of 1.1Ah were subjected to degradation. These cells underwent a constant current/constant voltage charging protocol with a constant current rate of 0.5C until the voltage reached 4.2V. The batteries were sustained at 4.2V till the charging current dropped below 0.05A. The failure threshold for these batteries were set to be 0.88Ah. All four batteries showed similar capacity degradation trends; hence we chose CS-36 for training and CS-37 and CS-38 for the purpose of testing the proposed hybrid algorithm (HyA). The capacity degradation curve versus the charge/discharges cycles of the three batteries considered in this work are shown in Fig. 1(a).

FIGURE 1.

The typical battery capacity degradation curves from the (a) CALCE dataset (b) NASA dataset for LiCoO₂ cell technology.

2) NASA Dataset

For the second dataset, we chose to use the degradation data of 18650 LiCoO₂ batteries from the NASA repository. The rated capacity of these batteries is 2.1Ah and unlike the CALCE dataset, these batteries were cycled under random currents rather than constant discharge currents. The battery capacities were measured after every 1500 periods and the failure threshold was set to be 1Ah. The capacity degradation curve versus the charge/discharges cycles is shown in Fig. 1(b). The labels of batteries used as training and test datasets from both CALCE and NASA repository are shown below in Table 1.

TABLE 1 Summary of Battery Labels Used as Training and Test Datasets From CALCE and NASA Repository

B. RUL Estimation Using FFNN or Standard Pf

In this section, the prediction results of the standard FFNN model and standard PF approach are compared. The battery CS-36 of CALCE dataset and RW9 of NASA dataset were used to train the neural network model with 3 hidden neurons. The number of hidden neurons greatly affects the performance of the NN model. However, there are not any definitive methods available in literature for the optimal selection of hidden neurons. The selection methods proposed in literature were either specific to the system/device considered or was specific to the type of NN architecture used for their study. Also, the number of NN weight and bias parameters for 5–6 hidden neurons scales up exponentially which can also result in overfitting of the time series degradation patterns and thereby lead to convergence issues and multiple local optima solutions. Hence, we adopted a trial-and-error method based on Bayesian Information Criterion (BIC) to fix the number of hidden neurons suitable for our study [25], [26].

The trained NN model was used to predict the capacity values of test data sets, RW10 and CS-37. The prediction results are shown in Fig. 2(b) and 2(d), respectively. The trained NN model was executed for 50 repetitions, of which only 3 repetitions were able to trace the actual degradation trend for RW10. A particular repetition was considered to be successful if the prediction trace lies within the $2\sigma$ bounds. If the prediction trace was beyond the $2\sigma$ limits, then it was considered an outlier and eliminated. The CALCE dataset has lesser inflections in the degradation trend compared to the NASA dataset. Despite that, only 5 out of 50 repetitions were successful. Similar predictions results were observed for CS-38 as well. The prediction success rate and accuracy can be improved by opting for more complex NN architecture such as increasing the number of hidden neurons or by using a sigmoid output activation function though both the options comes with additional computation cost. The take away message here is that even a simple NN architecture fails to capture the predict the degradation trend with good prediction accuracy even if it is trained with full run to failure data of one device/system.

FIGURE 2.

The prediction curves using standard PF algorithm for (a) Battery RW10 of NASA dataset and (c) Battery CS-37 of CALCE dataset. The prediction curves based on the standard feedforward NN model for (b) Battery RW10 of NASA dataset and (d) Battery CS-37 of CALCE dataset. The gray lines indicate the prediction traces for 5000 particles (in the case of PF) and 50 repetitions for NN.

To address the limitations of the FFNN, a PF algorithm with NN degradation model as the measurement equation was analyzed. This approach is similar to the hybrid approached used in [21]–​[24], as described earlier in Section I. The curve fitting results for RW9 and CS-36 using NN model were used as the initial parameter guess for the PF algorithm. The prediction traces for RW10 and CS-37 for 5000 particles are shown in Figs. 2(a) and 2(c), respectively. The degradation model with 3 hidden neurons is expected to be capable of capturing the non-linearity in the degradation trend. However, PF algorithm is unable to track the actual degradation trend. The prognostic performance can be improved by using more complex NN architecture or by using meta-heuristic algorithms for resampling weighted particles in the PF algorithm. However, those approaches increase the computational load and complexity tremendously and employing such techniques in real-time would be challenging. This calls for the need of an adaptive hybrid approach for RUL estimation for systems wherein the network architecture is simple yet powerful enough to capture highly non-linear degradation trends.

SECTION IV. Proposed Hybrid Prognostic Framework

The proposed hybrid particle filter trained neural network framework (HyA) is shown in Fig. 3. The proposed method can be split into three stages – (A) Degradation Model Formulation and Parameter Initialization, (B) Bayesian State Estimation and (C) Neural Network Prognosis with Bayesian Posterior Weights and Bias. To begin with in Stage A, one of the batteries run-to-failure data from each dataset was chosen as the training dataset. In this case, CS-36 and RW9 were chosen as the training datasets. The curve fitting results for NN model by varying the number of hidden neurons (2 to 6) were obtained. The Bayesian information criterion (BIC) was evaluated and used as the deciding factor for network model selection. This is because the BIC is a robust metric for model selection which penalizes the use of a model with too many fitting parameters for any given data set. The NN architecture model with the lowest BIC value was chosen for the purpose of this study. The BIC can be expressed as

$$\begin{equation*} BIC=kln\left ({n }\right)-2\mathrm {ln}(\hat {L})\tag{8}\end{equation*}$$ where $\hat {L}$ is the maximized value of the likelihood function of the NN model, $k$ is the number of parameters to be estimated by the model and $n$ is the number of observations in the training dataset. The first term in the equation ($kln\left ({n }\right))$ is the penalty term that prevents the choice of an artificially complex model for any given data set. The BIC values for different model equations are shown in Table 2. The model with the minimum value of BIC was chosen as the best suited model for the purpose of this study. It is evident from Table 2 that three hidden neurons is the ideal choice for this study. The curve fitting parameters for the 3-neuron NN model is fed into the PF algorithm to populate the initial prior distribution of the NN weights and biases for subsequent recursive Bayesian updating. The PF algorithm employed in this framework is solely used for the purpose of state estimation.

TABLE 2 Comparison of BIC Values for Different NN Architectures on CALCE (CS-36) and NASA (RW9) Training Datasets

FIGURE 3.

The schematic of the proposed three-stage hybrid particle filter based neural network algorithm (HyA).

FIGURE 4.

The prediction curves of RW10 for the proposed hybrid algorithm (HyA) with prediction starting point at (a) 30 cycles (b) 60 cycles and (c) 90 cycles. The gray lines represent the prediction traces for 50 repetitions of HyA.

In Stage B, the entire training dataset is fed into the PF algorithm as measurement data. The NN degradation model-based state transition function, shown in Eqn. (3) and Eqn. (4) are used to estimate the network parameters. The first two stages of the framework constitute the training phase. In Stage C, a FFNN model with 3 hidden neurons was configured using the test dataset. As mentioned earlier, the choice of using 3 hidden neurons was based on the BIC analysis. Batteries CS-37, CS-38 and RW10 were the test datasets used for state prediction and RUL estimation. The network parameters estimated by the PF algorithm in Stage B are used to configure the initial weights and biases of the FFNN model. This approach helps to warm start the neural network training for the test dataset. As warm starting the neural network helps to restrict the network parameters closer to the optimal values, the prediction accuracy is expected to be higher. The PF algorithm in the training phase is executed with 5000 particles and the predicted network parameter values with $1\sigma$ limits of the posterior distribution were chosen. A uniform distribution was generated using the $1\sigma$ bounds and 50 random samples were generated from the uniform distribution.

The FFNN model in Stage C was executed for 50 repetitions with 50 different initial configuration of weights/bias values. The FFNN model is now further trained using the available test data (using the training data led PF based posterior weights and biases as the starting values of the NN) for the test dataset with 50 different initial weight/bias values. LM algorithm was opted for training the network parameters. The trained model was used to predict the future battery capacity till the end-of-life and hence estimate the remaining useful life of the batteries.

SECTION V. Results and Discussions

A. RUL Prediction for NASA Dataset Using HyA

The prediction results for battery RW10 at three different prediction starting points i.e., 30, 60 and 90 cycles are shown in Figs. 4(a), 4(b) and 4(c) respectively. At 30 cycles, with just few cycles of data available in the test data, the proposed HyA is able to capture the degradation trend including the inflection at the $120^{\mathrm {th}}$ cycle. Compared to the results obtained using standard FFNN shown in Fig. 2(b), 43 out of the 50 repetitions successfully traced the degradation trend. As mentioned in earlier sections, a particular repetition was considered successful if the degradation trace lies within the $2\sigma$ bounds. At 60 cycles, the prediction accuracy was expected to improve with a greater number of data available for prediction. However, the kink in the degradation data at the $60^{\mathrm {th}}$ cycle caused the HyA to predict an exponentially increasing trend for few of the repetitions. Despite the glitch in the prediction results, the proposed HyA achieved a success rate of about 66%. At 90 cycles, with the availability of more test data along with warm start settings, the HyA approach achieved a success rate of 88%. For the RUL distribution shown in Fig. 5, the repetitions wherein the predictions failed to be within $2\sigma$ bounds were omitted and only the successful repetitions were used to construct the RUL distribution. The predicted RUL almost coincides with the true RUL at 30 cycles but the large width of RUL pdf indicates uncertainty in the predictions. The width of the RUL pdf is narrow at 60 and 90 cycles even though there is an error of 18 cycles between the predicted and true RUL values. In order to evaluate the effectiveness of the proposed hybrid approach, the root mean squared error (RMSE) value was adopted as the prognostic performance metric. The RMSE value can be expressed as

$$\begin{equation*} RMSE=\sqrt {\frac {1}{n}\sum \limits _{k=T}^{n} d_{k}^{2}}\tag{9}\end{equation*}$$ where $k$ is the cycle index, $n$ is the number of predictions, $T$ is the prediction starting point and $d_{k}$ corresponds to the error between predicted capacity and actual capacity value at the $k^{th}$ time instant as shown below: $$\begin{equation*} d_{k}=x_{predicted}-x_{true}\tag{10}\end{equation*}$$

FIGURE 5.

The RUL distribution for Battery RW10 of NASA dataset at three different prediction starting points – $30^{\mathrm {th}}$ , $60^{\mathrm {th}}$ &$90^{\mathrm {th}}$ cycle along with predicted and true RUL values comparison.

Comparison of the RMSE values (mean value of 50 repetitions) between the standard PF algorithm, FFNN and the proposed hybrid algorithm (HyA) are listed in Table 3. The RMSE values clearly show that the proposed hybrid algorithm performs better than the other two conventional methods. Despite the low success rate of 66% at 60 cycles due to the sharp inflection point, the RMSE value is better, clearly indicating the robustness of the proposed hybrid approach.

TABLE 3 Comparison of RMSE Values for Different Prediction Methods at Different Prediction Starting Points for the NASA Test Dataset

B. RUL Prediction for Calce Dataset Using HyA

For the CALCE dataset, battery CS-36 was used for training the network parameters. The failure threshold for these batteries were set at 0. 88Ah. The prediction results at $120^{\mathrm {th}}$ , $185^{\mathrm {th}}$ and $250^{\mathrm {th}}$ cycles for both the batteries CS-37 and CS-38 are shown in Fig. 6 and Fig. 7, respectively. Unlike the NASA dataset, the CALCE dataset has a simple exponentially decreasing trend. Hence, the prediction success rate is in the range of 80-86% for all three prediction starting points for both the batteries. The probability density function of the RUL for CS-37 and CS-38 batteries are shown in Figs. 8(a) and (b), respectively.

FIGURE 6.

The prediction curves of CS-37 for the proposed hybrid algorithm (HyA) with prediction starting point at (a) 120 cycles (b) 185 cycles and (c) 250 cycles. The gray lines represent the prediction traces for 50 repetitions of HyA.

FIGURE 7.

The prediction curves of CS-38 for the proposed hybrid algorithm (HyA) with prediction starting point at (a) 120 cycles (b) 185 cycles and (c) 250 cycles. The gray lines represent the prediction traces for 50 repetitions of HyA.

FIGURE 8.

The RUL distribution for battery CS-37 & CS-38 of CALCE dataset at three different prediction starting points – $120^{\mathrm {th}}$ , $185^{\mathrm {th}}$ & $250^{\mathrm {th}}$ cycle along with predicted and true RUL values comparison.

For CS-37, the width of RUL pdf for all three prediction starting points are about 240 cycles indicating very good performance of the proposed approach. At 250 cycles, 42 out of the 50 predicted traces did not cross the failure threshold as shown in Fig. 6(c). However, for the remaining 8 successful repetitions, the RUL error between the predicted RUL and true RUL was found to be less than 1%. On the other hand, the width of the RUL pdf for CS-38 at the $120^{\mathrm {th}}$ cycle spanned over the entire lifetime of the battery. The reason for this being that the degradation curve for CS-38 has lot of battery regenerative signature compared to the other two batteries.

Similar results were observed for the fourth battery dataset as well (CS-39) but for the sake of brevity, the results are not discussed here. With the availability of a greater amount of test data, the prediction accuracy improves and the RUL pdf width at the $250^{\mathrm {th}}$ cycle is substantially narrow with minimum RUL error of about 50 cycles between the predicted and true values. Also, the RMSE values for both the batteries at all three prediction starting points are listed in Table 4. The error values (highlighted) clearly indicate that the proposed hybrid algorithm (HyA) is efficient and accurate compared to standard PF and NN approaches.

TABLE 4 Comparison of RMSE Values for Different Prediction Methods at Different Prediction Starting Points for the CALCE Test Datasets

C. Choice of Network Architecture

A good choice of network architecture is essential for enhancing the prediction capabilities of the NN model. An optimal number of hidden neurons is essential for improved mapping of the complex input/output relations. In this work, the choice of hidden neurons was done by a trial-and-error procedure using BIC values. A lower number of neurons would inhibit the NN model to capture the nuances in the degradation trend and therefore, the analysis for a single hidden neuron was omitted as it would oversimplify the trend. The BIC results for different number of hidden neurons are shown in Table 2. The model with the minimum BIC value was considered as the best suited model for the purpose of this study. Choosing a very high number of hidden neurons may result in overfitting issues. The prediction results for 4, 5 and 6 hidden neurons are shown in Fig. 9(a), Fig. 9(b) and Fig. 9(c) respectively. It can be seen from Fig. 9(b) that the HyA tries to mimic the kink in the true data at 60 cycles and eventually distorts the prediction trend. Similar behavior was observed for 6 hidden neurons as well. The RMSE values as well as computational time for predictions against number of hidden neurons are listed in Table 5. From Table 5, it is evident that 3 hidden neurons (with minimum BIC value) is the optimal count for the handling the complexity in the considered battery degradation datasets in this study. Since 3 hidden neurons would require optimizing 9 parameters over 13 parameters for 4 hidden neurons, the computational load was found to be comparatively less. Thus, the choice of 3 hidden neurons was eventually found to be a fair compromise between low error values and less computational time as well. Moreover, if we went with the option of a higher neuronal count, then we would end up risking the chances of particle degeneracy or impoverishment in the PF as well.

TABLE 5 Comparison of RMSE Values and Computational Time for Different Number of Hidden Neurons for the NASA (RW 10) and CALCE (CS-37) Test Datasets

FIGURE 9.

The prediction curves of RW10 for the proposed hybrid algorithm (HyA) with (a) 4 hidden neurons (b) 5 hidden neurons and (c) 6 hidden neurons. The gray lines represent the prediction traces for 10 repetitions of HyA.

D. Computational Time

One of the major challenges with developing hybrid approaches is to make sure that the computational time is not compromised for better prediction accuracy. Hence, we chose to analyze the execution time for the proposed algorithm along with PF and FFNN approaches. The standard PF approach with 5000 particles took about 8.12 seconds for execution and a standard FFNN model took about 48.36 seconds for executing 50 repetitions. For the proposed hybrid algorithm (HyA), the execution time was found to be 53.14 seconds for 50 repetitions. The computational time for HyA is the cumulative time taken for executing Stage B and C shown in Fig. 3. Therefore, it can be concluded that the proposed hybrid approach is not only effective in terms of prediction accuracy but also in terms of computational load. It is to be noted that the simulations were executed in a standard DELL^® desktop workstation (Model-Inspiron 14 – 5459) with 16GB RAM and Intel Core i5 processor.

E. Comparison of Prediction Results With Existing PF + NN Hybrid Prognostic Framework

There are few hybrid prognostic methodologies proposed in literature combining PF and NN. However, most of those research works were tested on datasets different from the battery degradation datasets considered in this work such as battery voltage discharge curves, crack propagation datasets etc. However, Ref. [24] shows the prediction results for CALCE battery CS-37 and NASA RW11 battery. In Ref. [24], Wu et al. have used the NN model equation as the state transition function in the PF framework and subsequently PF algorithm was used for the purpose of RUL estimation. The authors have performed curve fitting on each of the battery dataset considered for their study and the corresponding curve fitting parameters were fed into the PF framework as the initial parameter guess values. The prediction results comparing Ref. [24] and our proposed HyA are shown in Fig. 10(a) and Fig. 10(b) for CS-37 and RW11 respectively. It is to be noted that the blue curve representing HyA in Fig. 10 is the mean value of predictions for 50 repetitions. The extracted RMSE values for the prediction results of Ref. [24] was found to be 0.3686 and 0.2698 for RW11 and CS-37 respectively. This is much higher than the RMSE values for the same data set extracted using our framework as shown in Table 6 below. It is evident from the results that our proposed HyA framework outperforms the prediction results of Wu et al. [24] despite their impractical assumption that the entire test dataset information is available apriori for curve fitting. It can thus be concluded that our proposed method is robust, more realistic and highly adaptable.

TABLE 6 Comparison of RMSE Values for Data Sets RW11 and CS-37 Using our Hybrid Framework in Comparison to the Other Hybrid Framework Reported in Ref. [24]

FIGURE 10.

The degradation comparison plots between HyA and ref. [24] for (a) CALCE (CS-37) dataset (b) NASA (RW11) dataset for LiCoO₂ cell technology. The corresponding RMSE values for the two data sets using the two different hybrid prognostics frameworks is shown in table 5.

SECTION VI. Conclusion

In this work, a hybrid particle filter based neural network model is proposed for RUL prediction of lithium-ion batteries. A neural network model with one hidden layer containing 3 hidden neurons and a sigmoid activation function at the input side and linear activation function in the output side was used as the damage evolution model in the particle filter framework. The novelty of the proposed method lies in using PF algorithm for estimating the posterior distributions of the network weight and bias parameters corresponding to the training dataset and utilizing those parameters to warm start an MLP network model for the test dataset. The MLP model is further trained to optimize the network parameters for the test dataset. The trained MLP model is used to predict the future battery capacity values and remaining useful life. The proposed method was tested on NASA and CALCE datasets. The prediction results were compared with standard PF and FFNN methods. It was evident that the proposed approach was versatile enough to use the same NN architecture for both NASA and CALCE even though their degradation patterns are vastly different. The versatility of the proposed method makes it an appropriate choice for reliability studies on newer systems. Also, the proposed methods have very good prediction accuracy with very low RMSE values and is efficient in terms of computational time as well compared to the two conventional methods.

For future work, we intend to test the algorithm on LED luminosity degradation datasets where there are three distinct degradation phases and would be challenging for any prognostic algorithm to model and predict such highly non-monotonic degradation phenomena [27]. Also, the impact of different resampling strategies used in PF algorithm on RMSE values and computational time would be explored in future. The proposed approach can also be modified to include physics-informed surrogate models over FFNN models which would have the capability to encode underlying physical laws in a given dataset.