Air Quality Prediction Using Improved PSO-BP Neural Network

Predicting urban air quality is a significant aspect of preventing urban air pollution and improving the living environment of urban residents. The air quality index (AQI) is a dimensionless tool for quantitatively describing air quality. In this paper, a method for optimizing back propagation (BP) neural network based on an improved particle swarm optimization (PSO) algorithm is proposed to predict AQI. The improved PSO algorithm optimizes the variation strategy of the inertia weight as well as the learning factor, guaranteeing its global search ability during the early stage and later enabling its fast convergence to the optimal solution. We introduce an adaptive mutation algorithm during the search process to avoid the particles from falling into the local optimum. Through an analysis and comparison of the experimental results, BP neural network optimized using the improved PSO algorithm achieves a more accurate prediction of AQI.

SECTION I. Introduction

Population concentration, climate change, industrial production and other factors have lowered the air quality in many parts of China. At present, there are still numerous ongoing problems regarding this issue; for example, the awareness of environmental protection among the citizenry needs to be strengthened, and the level of air quality monitoring and control is low, decreasing the achievements in the regulation of air pollution [1]–​[3]. However, with improvements in living standards and economic development, the air quality in China has become an increasing concern among the public, and greater accuracy in air quality prediction has become an urgent need.

Some statistical methods, such as autoregressive integrated moving average (ARIMA), have been widely used in air quality prediction. However, these linear models may not obtain a reliable prediction if the sequence is nonlinear or irregular. In recent years, support vector regression (SVR) [4] has been applied in nonlinear regression forecasting. However, SVR with implicit kernel mapping such as RBF kernel [5] may not achieve an air quality forecasting model with a good performance because the data used by air quality predictors is massive and complex, which may result in an overfitting. Compared with the above methods, neural network is characterized by large-scale parallel processing, a high learning ability and a high non-linearity, and is more suitable for air quality prediction.

Although back propagation (BP) neural network has numerous advantages, its disadvantages are also obvious, namely, it is apt to fall into the local minimum, requires long-term learning, and achieves a low convergence speed [6]–​[10]. Experts from around the world have noticed these problems, and have put forward numerous suggestions for an improved performance. Tang et al. proposed introducing the adaptive learning rate into BP neural network, reducing the learning time [11]. In addition, Lba et al. introduced genetic algorithm into feedforward neural network, which improves the adaptability of network training [12], and Yao et al. applied an adaptive increase and decrease algorithm to select the structure of the network, stabilizing the network training more effectively [13]. The application of standard particle swarm optimization (PSO) in BP neural network can reduce the learning time and increase the calculation accuracy [14]–​[18]. However, the limited convergence speed of standard PSO is slow, and there are problems regarding local optimization and premature maturity, resulting in an inaccurate weight selection [19]–​[22].

We improved the PSO algorithm accordingly, optimized the overall prediction performance of BP neural network, adjusted the change strategy of the inertia weight as well as the learning factor, and ensured the diversity of particles during the early stage and the fast convergence to the global optimal solution. An adaptive mutation algorithm is also introduced during the search process to avoid particles from being trapped in the local optimum.

SECTION II. Particle Swarm Optimization and its Improvement

A. Standard Particle Swarm Optimization Algorithm

PSO is used to simulate the social state of a biological population. Each bird is regarded as a particle swarm, and through iterations can share information, combine its own experience, continuously improve its own behavior, and improve the flight experience through both individual and group information. PSO initializes the particles first, which will be continually updated during the iterations, based on the individual extremum pbest and global extremum gbest [23]. The best solution found by the particle itself is pbest, and gbest is the best solution for all particles. Supposing a population of m particles in the d-dimensional target search space [24], vector $X_{i} =\left ({{x_{i1},x_{i2},\ldots,x_{id},\ldots,x_{iD} } }\right)$ is the position of a particle, and vector $V_{i} =\left ({{v_{i1},v_{i2},\ldots,v_{id},\ldots,v_{iD}} }\right)$ is the velocity. Each particle updates its velocity and position according to the following formula [25]–​[28]:

$$\begin{align*} v_{id}^{k+1}=&w\left ({k }\right)v_{id}^{k} +c_{1} r_{1}^{k} \left ({{p_{id} -x_{id}^{k}} }\right)+c_{2} r_{2}^{k} \left ({{p_{gd} -x_{id}^{k}} }\right) \\ \tag{1}\\ x_{id}^{k+1}=&x_{id}^{k} +v_{id}^{k+1}\tag{2}\end{align*}$$ where $k$ is the number of current iterations; $v_{id}^{k}$ and $v_{id}^{k+1}$ are the velocities of the d-dimensional components of particle $i$ at $k$ and $k+1$ iterations; and $x_{id}^{k}$ and $x_{id}^{k+1}$ are the positions of the d-dimensional components of particle $i$ at $k$ and $k+1$ iterations, respectively. In addition, $p_{gd}$ is the global extremum of all particles in dimension d; $c_{1}$ and $c_{2}$ are the learning factors for non-negative constants; $r_{1}$ and $r_{2}$ are random numbers of [0, 1]; w(k) is the inertia weight coefficient; and $v_{id} \in [-v_{\max },v_{\max }]$ , where $v_{\max }$ is a constant that prevents particles from escaping from the solution space.

B. Improvement of Particle Swarm Optimization Algorithm

1) Improvement in the Inertia Weight

The weight function can adjust both the overall and local search ability of the algorithm. In the standard PSO algorithm, the inertia weight decreases along a line, which makes it possess a strong global exploration ability in the initial stage of iteration as well as a strong local search ability during the later stage, but tends to be “premature.” We adopted a method to reduce the inertia weight coefficient. A variation diagram of the inertia weight is shown below.

During the initial search phase, the inertia weight coefficient decreases nonlinearly, which enables the algorithm to achieve a stronger capability of conducting an overall search during this stage, and enter the local search as soon as possible. After k iterations, the inertia weight coefficient starts to decrease in line, which allows the algorithm to stably find the optimal solution. The algorithm adjustment is as follows:

$$\begin{equation*} w(t)=\begin{cases} \displaystyle w_{\min } +(w_{\max } -w_{\min })\times l_{1} (t),&t < k\\ \displaystyle 2w_{\min } +2(d-w_{\min })\times l_{2} (t),&t\ge k \end{cases}\tag{3}\end{equation*}$$ where $t$ is the number of iterations; $w_{\max }$ and $w_{\min }$ are the maximum and minimum values of the inertia weight coefficient, respectively; $l_{1} (t)$ is a nonlinear function; $l_{2} (t)$ is a linear function; and $d$ is the initial inertia weight after the initial search. The values of $l_{1} (t)$ and $l_{2} (t)$ are derived as follows: $$\begin{align*} l_{1} (t)=&e^{-30\times \left ({{t/t_{\max }} }\right)^{15}} \tag{4}\\ l_{2} (t)=&-\frac {t}{t_{\max }}\tag{5}\end{align*}$$ where $t_{\max }$ is the maximum number of iterations.

2) Improvement in the Learning Factors

To obtain the diversity of the particles during the initial search phase and converge to the global optimal solution as soon as possible during the later stage, by analyzing the influence of the change in the learning factor, the parameters $c_{1}$ and $c_{2}$ are dynamically adjusted using the tangent function to better balance the global and local searches. The tangent function is expressed as follows:

$$\begin{align*} c_{1} \left ({t }\right)=&\left ({{c_{1\_{}start} -c_{1\_{}end}} }\right) \\&\times \tan \left({0.875\times \left ({{1-\left ({{\frac {t}{N}} }\right)^{0.6}} }\right)}\right)+c_{1\_{}end} \tag{6}\\ c_{2} \left ({t }\right)=&\left ({{c_{2\_{}start} -c_{2\_{}end}} }\right) \\&\times arc\tan \left({2.8\times \left ({{1-\left ({{\frac {t}{N}} }\right)^{0.4}} }\right)}\right)+c_{_{2\_{}end}}\tag{7}\end{align*}$$

The curves of parameters $c_{1}$ and $c_{2}$ are shown in Fig. 2.

FIGURE 1.

Changing curve of inertia weight.

FIGURE 2.

Changing curve of c1 and c2.

As we can see from the figure, during the initial stage of the search, $c_{1}$ is larger than $c_{2}$ and each the particle pays attention to its own historical information to ensure diversity. However, during the later stage, $c_{1}$ decreases, whereas $c_{2}$ increases, making the particles pay more attention to the social information of the group to maintain a fast convergence.

3) Adaptive Mutation Particle Swarm Optimization

During the iterative process, the standard PSO algorithm easily falls into the local extremum, and the population loses the ability of the overall search during this time. By referring to the “mutation” operation of the genetic algorithm, we can mutate one dimension of the particle, adjust its position with a certain probability and enter other regions to continue the search. By doing so, we can effectively expand the search range, and obtain the global optimal solution of the algorithm. This is the basic idea behind the adaptive mutation PSO, the formula of which is as follows:

$$\begin{equation*} p\left ({{i,k} }\right)=5\times rand,\quad x>0.95\tag{8}\end{equation*}$$ where $p\left ({{i,k} }\right)$ represents the k-dimensional mutation operation of the $i$ particle of the population. A variation occurs when a random number $x$ of within 0 to 1 is greater than 0.95, and rand is a random value of within 0 to 1.

SECTION III. Improved Particle Swarm Algorithm to Optimize BP Neural Network

A. Determining Structure of Neural Network

First, we decided that BP neural network has a three-layer structure, and the input layer neuron $n_{1}$ and output layer neuron $n_{3}$ were determined according to the number of inputs and outputs. Second, the number of neurons in the hidden layer $n_{2}$ was determined based on the empirical formula, obtaining the minimum error of Formula (9). Here, $\eta _{k}$ is the threshold for the output layer, and $\theta _{j}$ is the threshold of the hidden layer. The connection weight between the input layer and the hidden layer is defined as $W_{ij}$ . The connection weight between the hidden layer and the output layer is defined as $V_{jk}$ , $f_{0}$ is the Sigmoid excitation function of the hidden layer, and $f_{1}$ is the linear excitation function of the output layer. A schematic diagram of the BP applied is shown in Fig. 3 [29].

$$\begin{align*} E=&\frac {1}{2}\sum \limits _{k=1}^{n_{3}} {\left ({{y_{k} -f_{1} \left ({{\sum \limits _{j=1}^{n_{2}} {V_{jk}} f_{0} \left ({{\sum \limits _{i=1}^{n_{1} } {a_{i} w_{ij}} -\theta _{j}} }\right)-\eta _{k}} }\right)} }\right)^{2}} \\ \tag{9}\\ f_{0}=&\frac {\left ({{e^{x}-e^{-x}} }\right)}{\left ({{e^{x}+e^{-x}} }\right)}\tag{10}\end{align*}$$

FIGURE 3.

Topological structure of three-layer BP neural network model.

B. Optimization of BP Neural Network Using Improved Particle Swarm Optimization Algorithm

The mean square error (MSE) generated by each network training set is regarded as the approximate fitness function used to calculate the fitness value, i.e., Eq. (11), and the minimum error value $E_{\min }$ is calculated based on the fitness function $f\left ({x }\right)=E\left ({x }\right)$ .

$$\begin{equation*} MSE\left ({{{\hat {y}_{i}}} }\right)=\frac {1}{n}\sum \limits _{i=1}^{n} {\left ({{y_{i} -{\hat {y}_{i} }} }\right)^{2}}\tag{11}\end{equation*}$$

In Eq. (11), $y_{i}$ and ${\hat {y}_{i}}$ are the target and predicted values, respectively. The smaller the MSE is, the more accurate the model. We update $V_{i}$ of the particles under different components until the training error is less than $E_{\min }$ or the number of iterations reaches $t_{\max }$ . If the error after training does not meet $E_{\min }$ , we can adjust the weight and threshold to meet this condition.

Fig. 4 shows the improved PSO algorithm process of the BP neural network.

FIGURE 4.

Flow chart of improved PSO-BP algorithm.

The main steps are described as follows:

• Step 1:

  The BP neural network topology is determined from the training sample data.

• Step 2:

  The particle velocity, position, individual extremum and global extremum value are initialized.

• Step 3:

  The appropriate fitness function is selected to evaluate the adaptive value of each particle.

• Step 4:

  Each particle fitness value is evaluated. If this value is better than the individual optimal solution, then the individual extremum pbest is updated with the current value. If the individual is better than the global best, the same approach replaces the global extremum gbest.

• Step 5:

  The velocity and position of the particles are calculated according to formulae (1), (2), (3), (6) and (7), and a mutation operation is carried out according to formula (8).

• Step 6:

  If the number of iterations is less than the set maximum value, or if the error parameter is less than the set error value, then return to step 3.

• Step 7:

  Using the improved PSO algorithm, the optimal weights and thresholds obtained are assigned to the BP neural network for training and learning.

SECTION IV. Prediction of Air Quality Index Model of Improved Particle Swarm Algorithm to Optimize the BP Neural Network

A. Data Selection

The air quality data used in this paper are from the China air quality monitoring and analysis platform, and include the average daily fine particulate matter ($PM_{2.5}$ ), inhalable particulate matter ($PM_{10}$ ), ozone ($O_{3}),NO_{2}$ , $CO$ , $SO_{2}$ [30]–​[34], and the Chongqing air quality index (AQI) for all of 2018-2019. After deleting invalid and missing data, a total of 10,272 pieces of data were collected. The 9,844 data from January 1, 2018 to November 30, 2019 were used as training samples. A total of 428 data from November 1 to 30, 2019 are used as testing samples. A portion of the data are shown in Table 1.

TABLE 1 Partial Sample Data

B. AQI Predictive Simulation and Results Analysis

1) Prediction of AQI Values Based on BP Neural Network

The MATLAB2018a platform was used for the simulation experiments. Python programming and a MySql database were used. The trainlm function was selected as the training function of the BP neural network, the sigmoid function was selected as the transfer function of the hidden layer, and the purelin function as the transfer function was selected as the output layer. The maximum training number is 4,000, the learning rate is 0.01, and the target error is 10⁻⁷. The errors of the hidden layer are minimized when the number of nodes is 9. The mean absolute percentage error (MAPE) was used to evaluate the performance of the prediction model [35].

Using standardized training samples and test samples, the BP neural network was trained and tested with the network parameters set, with an accuracy of 92.84%.

As we can see from the Fig. 5 and Fig. 6, the BP neural network prediction model has a good prediction ability for the AQI index, whereas the error of the single-point prediction is large; hence, the model still needs further improvement.

FIGURE 5.

BP network prediction value.

FIGURE 6.

BP network prediction error value.

2) Prediction of the AQI Index Based on PSO Neural Network and Improved PSO Neural Network

In this paper, multiple linear regression, PSO-BP neural network and improved PSO-BP neural network are used to compare the prediction accuracy. The results of multiple regression experiments are shown in the Fig. 7 and Fig. 8. The prediction accuracy is 97.14%, and the prediction effect is much better than that of the BP network. However, the accuracy still needs to be improved.

FIGURE 7.

MLRM prediction value.

FIGURE 8.

MLRM prediction error value.

The parameters of the PSO algorithm are set as follows: $c_{1} =c_{2} =1.49445$ , where the number of particles is 30, the number of population iterations is 50, the range of particle velocity is $\left [{ {-1,1} }\right]$ , the range of particle position is $\left [{ {-5,5} }\right]$ , and the BP neural network is optimized using PSO. As shown in the Fig. 9, the prediction accuracy is 98.04%.

FIGURE 9.

PSO-BP network prediction value.

The improved PSO-BP neural network was trained, and the parameters were set as follows: $c_{1\_{}end} =0.5$ , $c_{1\_{}start} =2.5$ , $c_{2\_{}start} =1$ , $c_{2\_{}end} =3$ , $w_{\min } =0.5$ and $w_{\max } =1$ . The experimental results of the improved PSO-BP neural network are shown in Fig. 11 and Fig. 12, and the prediction accuracy is 99.03%.

FIGURE 10.

PSO-BP network prediction error value.

FIGURE 11.

Improved PSO-BP network prediction value.

FIGURE 12.

Improved PSO-BP network prediction error value.

From Fig. 13, we can see that the improved PSO algorithm in this paper is better than the standard PSO algorithm in terms of the convergence accuracy, speed and optimization results. After optimization, the average optimal fitness value of the improved PSO algorithm is lower than that of the standard PSO algorithm.

FIGURE 13.

Fitness curve.

3) Results Analysis

The prediction error statistics of the BP neural network, multiple linear regression, PSO-BP neural network and improved PSO-BP neural network are shown in Table 2.

TABLE 2 AQI Value Prediction Statistics

The total error value of the improved PSO-BP neural network is only 25.91, which is far lower than that of the other models. At the same time, the prediction accuracy of the improved PSO-BP neural network is 99.03%, which is 6.19, 1.89 and 0.99 higher than that of the BP neural network, multiple linear regression and PSO-BP neural network, respectively, indicating that the model achieves a better prediction performance. In addition, the simulation results from Fig. 5, Fig. 7, Fig. 9 and Fig. 11 show that the predicted value of the improved PSO-BP neural network is closer to the real value.

The simulation results show that the BP neural network, which is optimized by the improved PSO-BP algorithm, has an excellent learning ability.

SECTION V. Conclusion

In this study, an improved PSO algorithm was used to optimize the BP neural network and predict the AQI. Based on the PSO algorithm, the mechanism combined with the BP neural network is introduced. The PSO algorithm was improved, and the improved BP neural network was established to optimize the prediction model. Compared with a traditional BP neural network, it is not easy to fall into the local minimum and achieve a better search ability. The simulation results show that the network model achieve a high accuracy in predicting the AQI and is a promising application.