Multiple Linear Regression Prediction and Wavelet Neural Network Based Intelligent Online Tuning Control Method

Manual-based parameters tuning methods of control system are widely used in many industrial fields. However, better control performances such as faster control speed, more stable, less workload and safer working conditions are expected. Therefore, artificial intelligent based improved classical control (AI-CC) is highly valued in control fields. Main works are as follows: Firstly, wavelet neural network based PID (WNN-PID) method is proposed, BPNN of BPNN based PID method (BPNN-PID) is replaced by WNN. Secondly, multiple linear regression-based wavelet neural network PID (MR-WNN-PID) is proposed, the control parameters are adjusted according to the predicted output of control system. Thirdly, simulations are implemented to compare the performance of the proposed methods and the existing methods. Finally, the stability of the proposed methods is analyzed by theory. Effects are as follows: Firstly, the proposed AI-CC methods have better performances such as faster control speed, smaller overshoot, better ability of anti-interference. Secondly, theoretical analysis of stability of proposed methods is proven.

SECTION I. Introduction

Manual-based parameters tuning control method is a common and widely used method [1], [2]. In the field of industrial control, classical control (CC) methods and systems [3], such as PID control, have been applied for a long time. PID control method, as the earliest controller in practical industry, possesses a history of nearly one hundred years, because the theories of PID method are complete and mature. Recently, improved artificial intelligence based classical control methods (AI-CC) are highly valued in automatic control fields [4], [5]. AI-CC methods have many advantages such as self-adaptation, self-learning, and sufficient theoretical support. Therefore, AI-CC methods are more and more widely applied in practical projects.

Problems of existing control parameters online tuning methods are as follows: (1) The control performances are unsatisfied, because more and more practical applications need faster control speed, lower steady-state error and better stability and anti-interference ability. (2) The control system cannot maintain the best performance continuously, because the parameters of the control system cannot be tuned online by the control system itself. (3) Long time working in harmful industrial environments is bad for engineers’ health, such as chemically pollution, radiant [6], [7], etc.

There are 3 existing advanced control methods [8]:

1. Model predictive control (MPC) is built by multi-step testing, rolling optimization, etc., which is more suitable for multivariable systems, it focuses on the optimization of control systems [9]. MPC is developed from the dynamic matrix control (DMC) in 1980s. MPC can present a dynamic response quickly, and it allows constraints to be incorporated into the control law. Moreover, MPC has good control performance and strong robustness, it can overcome the uncertainty of control process. However, MPC has some disadvantages such as large computational burden, monotonic feedback correction method and less rolling optimizing strategy. Furthermore, for practical application of non-linear system, MPC still remains some questions such as reliability and efficiency of the on-line computation scheme [10].

2. Adaptive control (ADC) can continuously identify control parameters of models and tune them online to reach a better performance of systems [11]. Adaptive control method is widely applied to uncertain systems, especially for controlling strict-feedback nonlinear systems [12]. Recently, some new adaptive control strategies are proposed for non-strict-feedback nonlinear systems [13]. However, the general solution of nonlinear random systems is difficult to obtain in theoretical level, the estimations of the uncertain parameters and convergence cannot be guaranteed, the parameters in adaptive control systems are much more difficult to be adjusted than in classical control systems [14], [15].

3. Intelligent control methods (not based on PID) are a class of control techniques that use artificial intelligence computing approaches such as expert control methods, fuzzy control methods, and neural network control methods [16]. However, intelligent control algorithms are nonlinear and complex, which make the intelligent control system difficult to be theoretically analyzed.

4. Artificial intelligence based improved classical control method (AI-CC methods) is based on the classical control theories, thus the classical control theories can be used for analyzing of AI-CC system such as non-liner, robustness and stability. AI-CC methods have both advantages of intelligent methods and classical control methods. Classical control method is widely used in many applications, so that research of AI-CC methods is more practical and expected.

Compare with the above three methods, AI-CC have its own advantages: (1) It is valuable to improve the existing classical control systems, because there are various practical applications are based on classical control. (2) The AI-CC methods can be theoretically analyzed easily on the basis of a large number of mature and existing classical control theories. (3) Variety of AI-CC methods can be implemented more easily because the output of AI is the input of CC, the dependence and coupling between AI and CC are very low [17]. Therefore, this study focuses on AI-CC methods and its improvements.

A. Related Work

The existing AI-CC methods are Expert-PID (E-PID), Fuzzy-PID (F-PID), Neural Network-PID (NN-PID), etc [18].

1. E-PID and F-PID are rule-based reasoning method. Control parameters of E-PID and F-PID are online tuned by expert rules tables (ERT) and fuzzy rules calculation (FRC) [19], [20]. ERT and FRC need to be offline set, the expert experiences are directly transformed into ERT and FRC. The advantage of E-PID and F-PID is that the control speed is faster because both E-PID and F-PID have fixed ERT and FRC.

2. NN-PID is neural network optimization-based method. In each adjustment cycle, the control parameters are tuned online based on the training and calculation results of neural network (NN). The inputs of NN include the system error and the differential of error, and the output is the new tuning control parameters. The control effects of NN-PID are determined by the structure of NN, training methods and sample data. The advantage of NN-PID is that the control system is smarter, which means that less manual configuration work is required, and better performances might be obtained.

The problems of E-PID and F-PID are as follows: (1) E-PID uses simple rules to get faster control speed but causes overshoot and instability. (2) The configuration work of F-PID is more complex for engineers, so the improvements of control performances are hard to achieve. (3) The reasoning process is based on fixed rules and strict formulas, ERT and FRC must be offline configured accurately before the system starting up. And the ERT and FRC cannot self-optimized during the control process [21], [22].

The NN-PID method can solve the above problems of E-PID and F-PID: (1) NN-PID has better online learning and optimizing ability, the rules of online tuning can be adjusted by NN-PID itself. (2) the configuration work of NN-PID is simpler, because the weights of NN do not need to be configured precisely before the system is running [23].

The existing NN-PID methods include Back Propagation Neural Network based PID (BPNN-PID) and Radical Basis Function Neural Network based PID (RBFNN-PID) [24], [25], etc. The problems of the existing BPNN-PID and RBFNN-PID are as follows: (1) Some training processes of RBFNN-PID are unstable, which means that the system output cannot converge in the end of the control process. (2) Some processes of BPNN-PID and RBFNN-PID systems appear overshoot which cannot be acceptable in some applications. (3) Performances such as control speed, steady-state error and ability of anti-interference are expected to be improved. Therefore, this study focuses on the improvements of NN-PID [26], [27].

B. Contribution

Main contributions to this study are as follows: (1) WNN-PID method is proposed to improve the performances. The training methods of existing BPNN-PID and RBFNN-PID are BPNN and RBFNN, while WNN is the training method of WNN-PID. Compared with BPNN and RBFNN, WNN has better learning ability which is expected to improve the performances of NN-PID systems. (2) MR-WNN-PID is further proposed to improve the performances. Multiple Linear Regression (MR) predictor is adopted to predict the system output, while WNN-PID has no predictor. MR predictor improves the accuracy of control processes. (3) Comparative simulations of all the above AI-CC methods are implemented to verify the effects of improvements. (4) The stability of all the above AI-CC methods is proven theoretically.

The rest of this article is organized as follows: In section II, the existing CC and AI-CC models are introduced. In section III, WNN-PID method is proposed. In section IV, MR-WNN-PID is proposed, which combines MR predictor with WNN-PID method. In section V, comparative simulations of are implemented with the results analysis of all AI-CC methods. In Section VI, theoretical stability analysis is discussed. In Section VII, conclusions are summarized including all the comparative methods.

SECTION II. Classical PID and NN-PID (Existing Model)

Section II is organized as follows: In part A, the model of existing PID is described. In part B, the models of BPNN-PID and RBFNN-PID methods are described. Structure, formula and pseudocode of the above existing methods are described in detail.

A. Classical PID Model (Existing Method)

1) Structure of Model

The structure of classical PID control system can be drawn as Fig.II-A.2. In PID control, $K_{p}$ , $K_{i}$ , $K_{d}$ are the control parameters which have been set as a fixed value before the system starts up. After the system starts up, in each adjustment cycle, the system error ($e$ and $de/dt$ ) can be calculated based on system input and system output at time $t$ . Then, the PID controller calculates the output of controller $u$ as the input of controlled object according to system error. Finally, Under the influence of $u$ , the output of controlled object is updated to be the system output at time $t+1$ .

Each simulation process has many adjustment cycles, which can be defined as follows:

Definition 1:

adjustment cycle (AC). 1 AC represents 1 control process at 1 fixed simulation time (e.g. $t=100$ ). In each AC, the system error $error(t)$ , $d_{error}(t)$ , the adjusted control parameters $k_{p}(t)$ , $k_{i}(t)$ , $k_{d}(t)$ , the control variable $u(t)$ (the output of controller) and the output of control system $yout\left ({t }\right)$ are updated once.

Definition 2:

simulation process (SP). 1 SP represents 1 independent control process which is from start time $t=1$ to the end time $max\_{}t$ (e.g. max simulation time $max\_{}t=10000$ ). Before the end time, the control system should be steady and stable.

The relationship between SP and AC is that 1 SP is composed of $n$ AC.

2) Calculation of Model

The model of classical PID Controller can be defined as Eq. (1): $$\begin{equation*} u(t)=K_{p}\cdot \left({e\left ({t }\right)+\frac {1}{K_{i}}\int {e\left ({t }\right)} \textrm {d}t+K_{d}\frac {\textrm {d}e\left ({t }\right)}{\textrm {d}t}}\right)\tag{1}\end{equation*}$$

In PID control, the control parameters $K_{p}$ , $K_{i}$ and $K_{d}$ are fixed values. While in AI-CC, the control parameters $K_{p}(t)$ , $K_{i}\left ({t }\right)$ and $K_{d}\left ({t }\right)$ are online tuned in each AC. Therefore, the model of AI-CC controller can be defined as Eq. (2): $$\begin{equation*} u(t)\!=\!K_{p}(t)\!\cdot \!\left({e\left ({t }\right)\!+\!\frac {1}{K_{i}\left ({t }\right)}\int {e\left ({t }\right)} dt\!+\!K_{d}(t)\frac {de\left ({t }\right)}{dt}}\right)\tag{2}\end{equation*}$$

The output of controller is $u\left ({t }\right)=u\left ({t-1 }\right)+\Delta u(t)$ . The updated $u\left ({t }\right)$ is calculated according to the incremental PID method, which can be expressed as Eq. (3): $$\begin{align*}&\hspace {-2.3pc}\Delta u(t)=k_{p}(t)\left ({e\left ({t }\right)-e\left ({t-1 }\right) }\right)+k_{i}(t)e\left ({t }\right) \\&\qquad \qquad +\,k_{d}(t)\left ({e\left ({t }\right)-2e\left ({t-1 }\right)+e\left ({t-2 }\right) }\right)\tag{3}\end{align*}$$

The output of the control system is $yout\left ({t }\right)$ , which can be calculated by the digital PID method of discrete system. The transfer function of the controlled object is discretized as Eq. (4). $$\begin{align*} yout\left ({t }\right)=&-a\left ({2 }\right)y\left ({t-1 }\right)-a\left ({3 }\right)y\left ({t-2 }\right)-a\left ({4 }\right)y\left ({t\!-\!3 }\right) \\&\quad +\,b\left ({2 }\right)u\left ({t\!-\!1 }\right)\!+\!b\left ({3 }\right)u\left ({t\!-\!2 }\right)\!+\!b\left ({4 }\right)u\left ({t\!-\!3 }\right) \\\tag{4}\end{align*}$$

3) Implement of Model

The pseudocode of PID control system is presented as Method 1:

Method 1 PID

1.

Begin

2.

Initialize PID control system;

3.

While $(t\le max\_{}t)$ do:

4.

Simulation time updated: $\text {t}=\text {t} +1$ ;

5.

System input $rin\left ({t }\right)$ is updated;

6.

Calculate $\Delta u\left ({t }\right)$ and $u\left ({t }\right)$ according to Eq. (3);

7.

Calculate $yout\left ({t }\right)$ according to Eq. (4);

8.

Calculate $e\left ({t }\right)=yout\left ({t }\right)-rin(t)$ ;

9.

End while

10.

End

B. BPNN-PID Model (Existing Method)

1) Structure of Model

The structure of neural network (NN) based AI-CC system is drawn as Fig. 2. Existing NN based AI-CC methods are BPNN-PID, RBFNN-PID etc. The control parameters of NN-PID are adjusted in each AC by a neural network.

FIGURE 1.

Structure of classical PID control systems.

FIGURE 2.

Structure of NN based intelligent control system.

2) Input and Output of Model

The inputs of NN are $rin(t)$ , $yout(t-1)$ , $error(t)$ , which are expressed as Eq. (5). $$\begin{align*} \begin{cases} \displaystyle {net}_{1}^{\left ({1 }\right)}(t)=rin(t) \\ \displaystyle {net}_{2}^{\left ({1 }\right)}(t)=yout(t-1) \\ \displaystyle {net}_{3}^{\left ({1 }\right)}(t)=error(t) \\ \displaystyle \end{cases}\tag{5}\end{align*}$$

The outputs of NN are $K_{p}\left ({t }\right)$ , $K_{i}\left ({t }\right)$ , $K_{d}\left ({t }\right)$ , which are expressed as Eq. (6). $$\begin{align*} \begin{cases} \displaystyle K_{p}(t)=O_{1}^{\left ({3 }\right)}(t) \\ \displaystyle K_{i}(t)=O_{2}^{\left ({3 }\right)}(t) \\ \displaystyle K_{d}(t)=O_{3}^{\left ({3 }\right)}(t) \\ \displaystyle \end{cases}\tag{6}\end{align*}$$

3) Forward Calculation

The outputs of NN are calculated according to Eq. (7) to Eq. (12). $$\begin{align*} {net}_{j}^{\left ({2 }\right)}(t)=&\sum \nolimits _{j=1}^{Q} {w_{ji}^{(2) }(t)\cdot O_{t}^{\left ({1 }\right)}(t)}\tag{7}\\ O_{j}^{\left ({2 }\right)}\left ({t }\right)=&f({net}_{j}^{\left ({2 }\right)}(t)),\quad (j=1,2,\ldots,Q) \tag{8}\\ {net}_{k}^{\left ({3 }\right)}\left ({t }\right)=&\sum \nolimits _{j=1}^{Q} {w_{kj}^{\left ({3 }\right)}(t) O_{j}^{\left ({2 }\right)}\left ({t }\right)} \tag{9}\\ O_{k}^{\left ({3 }\right)}\left ({t }\right)=&g\left ({{net}_{k}^{\left ({3 }\right)}\left ({t }\right) }\right)\!,\quad \left ({k=1,2,3 }\right)\tag{10}\\ g\left ({x }\right)=&\frac {e^{x}}{e^{x}+e^{-x}}\tag{11}\\ f\left ({x }\right)=&tanh \left ({x }\right)=\frac {e^{x}-e^{-x}}{e^{x}+e^{-x}}\tag{12}\end{align*}$$

$f(x)$ represents the scaling transformation function of the hidden layers of BPNN. $O_{j}^{\left ({2 }\right)}\left ({t }\right)$ represents the output of the hidden layer, and $O_{k}^{\left ({3 }\right)}\left ({t }\right)$ represents the output of the output layer. $w_{jk}^{(3) }$ and $w_{ij}^{(2) }$ are weights of the neural network.

4) Object Function

The target of adjusting process is to minimize the mean square error of control system. The object function of online tuner of NN-PID can be expressed as Eq. (13). $$\begin{equation*} \min {E\left ({t }\right)=\frac {1}{2}{(rin\left ({t }\right)-yout(t))}^{2}}\tag{13}\end{equation*}$$

The object function can also be expressed as Eq. (14). $$\begin{equation*} \min {E\left ({t }\right)=\left |{ rin\left ({t }\right)-yout(t) }\right |}\tag{14}\end{equation*}$$

Although the object function is different from Eq. (13) to Eq. (14), the effects are similar. In this study, all of the NNs will apply the same object function for comparison.

5) Training Calculation

According to the above object function, gradient descent method is adopted as the training algorithm in this study, because the gradient descent method is simpler and faster than other training methods. $a$ and $\mu$ are inertia coefficient and learning rate respectively. The adjustment formulas of $w_{jk}^{(3) }$ and $w_{ij}^{(2) }$ are provides as Eq. (15) to Eq. (19): $$\begin{align*} \Delta w_{kj}^{(3) }\left ({t }\right)=&\alpha \cdot \Delta w_{kj}^{\left ({3 }\right)}\left ({t-1 }\right)-\eta \cdot \delta _{k}^{(3) }(t)\cdot O_{j}^{\left ({2 }\right)}(t) \qquad \tag{15}\\ \Delta w_{ji}^{(2) }\left ({t }\right)=&\alpha \cdot \Delta w_{ji}^{\left ({2 }\right)}\left ({t-1 }\right)-\eta \cdot \delta _{j}^{(2) }(t)\cdot O_{i}^{\left ({2 }\right)}(t)\tag{16}\\ \delta _{k}^{\left ({3 }\right)}=&error\left ({t }\right)\cdot sgn\left ({\frac {\partial y\left ({t }\right)}{\partial \Delta u\left ({t }\right)} }\right)\cdot \frac {\partial \Delta u\left ({t }\right)}{\partial O_{k}^{\left ({3 }\right)}\left ({t }\right)} \\&\cdot \,g^{\prime }\left ({{net}_{k}^{\left ({3 }\right)}\left ({t }\right) }\right)\tag{17}\\ \delta _{j}^{(2) }=&f^{\prime }({net}_{j}^{\left ({2 }\right)}(t))\sum \nolimits _{k=1}^{3} {\delta _{k}^{\left ({3 }\right)}\cdot w_{kj}^{\left ({3 }\right)}\left ({t }\right)} \tag{18}\\ g^{\prime }\left ({x }\right)=&g(x)(1\!-\!g(x)),\quad f^{\prime }\left ({x }\right)=(1\!-\!f^{2}(x))/2\tag{19}\end{align*}$$

6) Implement of Model

The pseudocode of NN-PID can be designed as Method 2:

Method 2 NN-PID

1.

Begin

2.

Initialize NN-PID control system model;

3.

Initialize NN weights $W_{mn}\left ({t }\right)$ and offsets $b_{l}\left ({t }\right)$ ;

4.

While $(t\le max\_{}t)$ do:

5.

Update the simulation time: $\text {t}=\text {t}+1$ ;

6.

System input $rin\left ({t }\right)$ is updated;

7.

Calculate $error(t)$ and $d_{error}(t)$ ;

8.

Adjust $W_{mn}\left ({t }\right)$ and $b_{l}\left ({t }\right)$ according to (15) to (29);

9.

Adjust $K_{p}\left ({t }\right)$ , $K_{i}\left ({t }\right)$ , $K_{d}\left ({t }\right)$ according to (7) to (14);

10.

Calculate $\Delta u\left ({t }\right)$ and $u\left ({t }\right)$ according to Eq. (3);

11.

Calculate $yout\left ({t }\right)$ according to Eq. (4);

12.

End while

13.

End

SECTION III. WNN-PID Model (Improved Model)

Section III is organized as follows: In part A, the structure of WNN-PID is described. In part B, the input and output of WNN-PID is expressed. In part C, the objective function of WNN-PID is described. In part D, the training algorithm of WNN-PID is provided. In part E, the pseudocode of WNN-PID is shown.

A. Structure of Model

The structure of WNN-PID is similar to NN-PID, which is shown in Fig. 2. The difference between WNN-PID and BPNN-PID is that they have different structures of neural network and different training algorithms. The training method of WNN-PID is the gradient descent algorithm which is applied to WNN. Meanwhile, the training method of BPNN-PID is the gradient descent algorithm which is applied to BPNN.

B. Input and Output of Model

$K_{p}\left ({t }\right)$ , $K_{i}\left ({t }\right)$ , $K_{d}\left ({t }\right)$ are the control parameters which are the result of forward propagation of BPNN according to Eq. (20) to Eq. (27). $$\begin{align*}&\hspace {-4pc}\begin{cases} \displaystyle {net}_{1}^{\left ({1 }\right)}(t)=rin(t) \\ \displaystyle {net}_{2}^{\left ({1 }\right)}(t)=yout(t-1) \\ \displaystyle {net}_{3}^{\left ({1 }\right)}(t)=error(t) \\ \displaystyle \end{cases}\tag{20}\\&\hspace {-4pc}\begin{cases} \displaystyle K_{p}(t)=O_{1}^{\left ({3 }\right)}(t) \\ \displaystyle K_{i}(t)=O_{2}^{\left ({3 }\right)}(t) \\ \displaystyle K_{d}(t)=O_{3}^{\left ({3 }\right)}(t) \\ \displaystyle \end{cases}\tag{21}\\ O_{j}^{\left ({2 }\right)}\left ({t }\right)=&\Psi _{\textrm {ab}}\left ({\frac {net_{j}^{\left ({2 }\right)}\left ({t }\right)-b_{j}\left ({t }\right)}{a_{j}\left ({t }\right)} }\right) \tag{22}\\ {net}_{k}^{\left ({3 }\right)}(t)=&\sum \nolimits _{j=1}^{Q} {w_{jk}^{(3) }(t)\cdot O_{j}^{\left ({2 }\right)}(t)} \tag{23}\\ O_{k}^{\left ({3 }\right)}\left ({t }\right)=&\mathrm {g}\left ({{net}_{k}^{\left ({3 }\right)}\left ({t }\right) }\right) \tag{24}\\ \mathrm {g}\left ({x }\right)==&\frac {e^{x}}{e^{x}+e^{-x}}\tag{25}\\ \Psi _{\textrm {ab}}\left ({\textrm {t} }\right)=&\cos (1.75t)\cdot \textrm {e}^{-\frac {\textrm {t}^{2}}{2}}\tag{26}\\ \Psi _{a,b}^{\prime }\left ({t }\right)=&-1.75\!\cdot \!sin \left ({1.75t }\right)\!\cdot \!e^{-\frac {t^{2}}{2}}\!-\!t\!\cdot \!cos(1.75t)\cdot e^{-\frac {t^{2}}{2}} \\{}\tag{27}\end{align*}$$

$\Psi _{\textrm {ab}}(t)$ represents the scaling transformation function of the hidden layers of WNN. The selection of $\Psi _{\textrm {ab}}(t)$ must satisfy the framework condition. $O_{j}^{\left ({2 }\right)}\left ({t }\right)$ represents the output of the hidden layer, and $O_{k}^{\left ({3 }\right)}\left ({t }\right)$ represents the output of the output layer. $w_{jk}^{(3) }$ and $w_{ij}^{(2) }$ are weights of the neural network. $\Delta a_{j}(t)$ and $\Delta b_{j}(t)$ are scale parameters of the activation function in the hidden layer.

C. Object Function

WNN-PID method is proposed based on BPNN-PID. Therefore, the object function of WNN is same as BPNN mentioned above, which is expressed as Eq. (13).

D. Training Algorithm of Model

$w_{jk}^{(3) }$ , $w_{ij}^{(2) }$ , ${\Delta a}_{j}\left ({t }\right)$ and $\Delta b_{j}(t)$ are adjusted in each AC (adjustment cycle) as Eq. (28) to Eq. (37). $$\begin{align*} \Delta w_{jk}^{(3) }(t)=&\alpha \cdot \Delta w_{jk}^{(3) }\left ({t-1 }\right)-\eta \cdot \frac {\partial E(t)}{\partial w_{jk}^{(3) }(t)}\tag{28}\\ \Delta w_{ij}^{(2) }(t)=&\alpha \cdot \Delta w_{ij}^{(2) }\left ({t-1 }\right)-\eta \cdot \frac {\partial E(t)}{\partial w_{ij}^{(2) }(t)}\tag{29}\\ \Delta a_{j}(t)=&\alpha \cdot \Delta a_{j}(t-1)-\eta \cdot \frac {\partial E(t)}{\partial a_{j}(t)}\tag{30}\\ \Delta b_{j}(t)=&\alpha \cdot \Delta b_{j}(t-1)-\eta \cdot \frac {\partial E(t)}{\partial b_{j}(t)}\tag{31}\\ \frac {\partial E(t)}{\partial w_{jk}^{(3) }(t)}=&error(t)\cdot sgn\left({\frac {\partial y(t)}{\partial \Delta u(t)}}\right)\cdot \frac {\partial \Delta u(t)}{\partial o_{k}^{(3) }(t)} \\&\,\cdot g^{\prime }(net_{k}^{(3) }(t))\cdot o_{j}^{(2) }(t)\tag{32}\\ \frac {\partial E(t)}{\partial w_{ij}^{(2) }(t)}=&\sum \nolimits _{k=1}^{L} \delta _{k}^{(3) } (t)\cdot w_{jk}^{(3) }(t)\cdot \psi _{a,b}^{\prime }\left ({\!\frac {net_{j}^{(2) }(t)\!-\!b_{j}(t)}{a_{j}(t)}\!}\right) \\&\cdot \,\frac {1}{a_{j}(t)}\cdot o_{i}^{(1) }(t)\tag{33}\\ \frac {\partial E(t)}{\partial a_{j}(t)}=&\sum \nolimits _{k=1}^{L} \delta _{k}^{(3) } (t)\cdot w_{jk}^{(3) }(t)\cdot \psi _{a,b}^{\prime }\left ({\!\frac {net_{j}^{(2) }(t)\!-\!b_{j}(t)}{a_{j}(t)}\!}\right) \\&\cdot \left ({-\frac {net_{j}^{(2) }(t)-b_{j}(t)}{a_{j}^{2}(t)} }\right)\tag{34}\\ \frac {\partial E\left ({t }\right)}{\partial b_{j}(t)}=&\sum \nolimits _{k=1}^{L} \delta _{k}^{(3) }(t)\cdot \textrm {w}_{jk}^{(3) }(t)\cdot \Psi _{\textrm {ab}}^{\prime }\left({\frac {net_{j}^{(2) }(t)-b_{j}(t)}{a_{j}(t)}}\right) \\&\times \,\left({-\frac {1}{a_{j}(t)}}\right)\tag{35}\\ \delta _{k}^{(3) }(t)=&error(t)\cdot sgn\left ({\frac {\partial y(t)}{\partial u(t)} }\right)\cdot \frac {\partial u(t)}{\partial o_{k}^{(3) }(t)}\cdot g^{\prime }\left ({net_{k}^{(3) }(t) }\right) \tag{36}\\ \\ g(x)=&\frac {e^{x}}{e^{x}+e^{-x}},\quad g^{\prime }(x)=\frac {2}{(e^{x}+e^{-x})^{2}}\tag{37}\end{align*}$$

E. Implement of Model

The pseudocode of WNN-PID can be designed as Method 3:

Method 3 WNN-PID

1.

Begin

2.

Initialize WNN-PID control system model;

3.

Initialize WNN weights ${\boldsymbol {W}}_{{{\boldsymbol {mn}}}}\left ({{\boldsymbol {t}} }\right)$ and offsets ${\boldsymbol {b}}_{{\boldsymbol {l}}}\left ({{\boldsymbol {t}} }\right)$ ;

4.

While (${\boldsymbol {t}} \le {{\boldsymbol {max}}}\_{}t$ ) do:

5.

Update the simulation time: $\text {t}=\text {t}+1$ ;

6.

System input ${{\boldsymbol {rin}}}\left ({{\boldsymbol {t}} }\right)$ is updated;

6.

Calculate ${{\boldsymbol {error}}}({\boldsymbol {t}})$ and ${\boldsymbol {d}}_{{{\boldsymbol {error}}}}({\boldsymbol {t}})$ ;

7.

Adjust ${\boldsymbol {W}}_{{{\boldsymbol {mn}}}}\left ({{\boldsymbol {t}} }\right)$ and ${\boldsymbol {b}}_{{\boldsymbol {l}}}\left ({{\boldsymbol {t}} }\right)$ according to (28) to (37);

8.

Adjust ${\boldsymbol {K}}_{{\boldsymbol {p}}}\left ({{\boldsymbol {t}} }\right)$ , ${\boldsymbol {K}}_{{\boldsymbol {i}}}\left ({{\boldsymbol {t}} }\right)$ , ${\boldsymbol {K}}_{{\boldsymbol {d}}}\left ({{\boldsymbol {t}} }\right)$ according to (20) to (27);

9.

Calculate $\Delta {\boldsymbol {u}}\left ({{\boldsymbol {t}} }\right)$ and ${\boldsymbol {u}}\left ({{\boldsymbol {t}} }\right)$ according to Eq. (3);

10.

Calculate ${{\boldsymbol {yout}}}\left ({{\boldsymbol {t}} }\right)$ according to Eq. (4);

11.

End while

12.

End

SECTION IV. Mr-WNN-PID (Proposed Model)

Section IV is organized as follows: In part A, the multiple regression-based (MR) predictor is proposed. In part B, MR-WNN-PID is proposed based on MR predictor and WNN-PID. Structure, formula and pseudocode of MR-WNN-PID are described in detail.

A. Design of MRP

MR predictor is proposed in order to improve the performance of control system such as control speed, steady-state error and anti-interference ability, etc. Multiple regression is a classic regression method with multiple inputs and single output. In this study, MR predictor is proposed to solve the problems of vector time series prediction.

The control parameters will be calculated according to the predicted output of MR predictor, which means the current system output $yout(t)$ at time $t$ is replaced by the predicted output $yout(t+p)$ at time $t+p$ .

1) Model of MR Predictor

The input series of MR predictor at time $t$ are expanded to series at time $t$ , $t-3$ and $t-6$ . Therefore, more time series of control system with more historic information are participated in the prediction.

The input of the MR predictor at time $t$ , $t-3$ , $t-6$ are as follows:

1. Control target: $rin\left ({t }\right)$ , $rin\left ({t-3 }\right)$ , $rin\left ({t-6 }\right)$ ;

2. Control parameter: $K_{p}\left ({t }\right)$ , $K_{i}\left ({t }\right)$ , $K_{d}\left ({t }\right)$ , $K_{p}\left ({t-3 }\right)$ , $K_{i}\left ({t-3 }\right),K_{d}\left ({t-3 }\right)$ , $K_{p}\left ({t-6 }\right)$ , $K_{i}\left ({t-6 }\right)$ , $K_{d}(t-6)$ ;

3. Control variable, which is the output of controller: $u\left ({t }\right)$ , $u\left ({t-3 }\right)$ , $u(t-6)$ , and the output of the control system: $yout\left ({t }\right)$ , $yout\left ({t-3 }\right)$ , $yout(t-6)$

The vector time series can be expressed as Eq. (38): $$\begin{align*} X_{n}=&\{rin\left ({t }\right),K_{p}\left ({t }\right),K_{i}\left ({t }\right),K_{d}\left ({t }\right),u\left ({t }\right),yout\left ({t }\right)\!, \\&{rin\left ({t-3 }\right)\!,K}_{p}\left ({t-3 }\right)\!,K_{i}\left ({t-3 }\right),K_{d}\left ({t-3 }\right)\!, \\&u\left ({t-3 }\right),yout\left ({t-3 }\right)\!,{rin\left ({t-6 }\right),K}_{p}\left ({t-6 }\right)\!, \\&K_{i}\left ({t-6 }\right)\!,K_{d}\left ({t-6 }\right)\!,u\left ({t-6 }\right)\!,yout\left ({t-6 }\right)\}\tag{38}\end{align*}$$

The predicted output $yout(t+p)$ of $p$ steps predictive control system can be predicted according to multiple regression prediction. For example, the output $yout(t+10)$ of MRP at future time $t+10$ can be expressed as Eq. (39): $$\begin{align*}&\hspace {-2pc}yout\left ({t+10 }\right) \\=&{\beta _{0}+\beta }_{1}rin\left ({t }\right)+{\beta _{2}K}_{p}\left ({t }\right)+\beta _{3}K_{i}\left ({t }\right)+beta_{4}K_{d}\left ({t }\right) \\&+\,\beta _{5}u\left ({t }\right)+\beta _{6}yout\left ({t }\right)+\beta _{7}rin\left ({t-3 }\right) \\&+\,{\beta _{8}K}_{p}\left ({t-3 }\right)+\beta _{9}K_{i}\left ({t-3 }\right){+\beta }_{10}K_{d}\left ({t-3 }\right) \\&+\,\beta _{11}u\left ({t-3 }\right)+\beta _{12}yout\left ({t-3 }\right) \\&+\,\beta _{13}rin\left ({t-6 }\right)+{\beta _{14}K}_{p}\left ({t-6 }\right) \\&+\,\beta _{15}K_{i}\left ({t-6 }\right){+\beta }_{16}K_{d}\left ({t-6 }\right)+\beta _{17}u\left ({t-6 }\right) \\&+\,\beta _{18}yout\left ({t-6 }\right)\tag{39}\end{align*}$$

$\beta _{0}$ is a constant, and $\beta _{1}$ , $\beta _{2},\cdots,\beta _{18}$ are the regression coefficients.

2) Model Fitting of MR Predictor

The above model of MRP is solved by Ordinary least squares (OLS) method. The target of solving process is to find the minimal residual sum of square (RSS), which can be expressed as Eq. (40): $$\begin{align*} \sum \limits _{i=1}^{n} \varepsilon _{i}^{2}=&\sum \nolimits _{i=1}^{n} (y_{out}\left ({t+10 }\right)-\beta _{0} -\beta _{1}rin\left ({t }\right)-{\beta _{2}K}_{p}\left ({t }\right) \\&-\,\beta _{3}K_{i}\left ({t }\right){-\beta }_{4}K_{d}\left ({t }\right)-\beta _{5}u\left ({t }\right)-\beta _{6}yout\left ({t }\right) \\&-\,\beta _{7}rin\left ({t-3 }\right)-{\beta _{8}K}_{p}\left ({t-3 }\right)-\beta _{9}K_{i}\left ({t-3 }\right) \\&-\,\beta _{10}K_{d}\left ({t-3 }\right)-\beta _{11}u\left ({t-3 }\right)-\beta _{12}yout\left ({t-3 }\right) \\&-\,\beta _{13}rin\left ({t-6 }\right)-{\beta _{14}K}_{p}\left ({t-6 }\right)-\beta _{15}K_{i}\left ({t-6 }\right) \\&-\,\beta _{16}K_{d}\left ({t-6 }\right)-\beta _{17}u\left ({t\!-\!6 }\right)\!-\!\beta _{18}yout\left ({t\!-\!6 }\right))^{2}\tag{40}\end{align*}$$

In order to find out the minimal RSS, one-order partial derivative conditions are calculated, and the normal equations of MR model are as follows: $$\begin{align*} \begin{cases} \displaystyle \sum \nolimits _{N=1}^{N} \left ({yout\left ({t+10 }\right)-{\beta _{0}-\beta }_{1}rin\left ({t }\right)-\cdots }\right.\\ \displaystyle \qquad -\left.{\vphantom {yout\left ({t+10 }\right)-{\beta _{0}-\beta }_{1}rin\left ({t }\right)-\cdots }\beta _{18}yout\left ({t-6 }\right) }\right)=0 \\ \displaystyle \sum \nolimits _{N=1}^{N} \left ({yout\left ({t+10 }\right)-{\beta _{0}-\beta }_{1}rin\left ({t }\right)-\cdots }\right.\\ \displaystyle \qquad -\left.{\vphantom {yout\left ({t+10 }\right)-{\beta _{0}-\beta }_{1}rin\left ({t }\right)-\cdots }\beta _{18}yout\left ({t-6 }\right) }\right)rin\left ({t }\right)=0 \\ \displaystyle \vdots \\ \displaystyle \sum \nolimits _{N=1}^{N} \left ({yout\left ({t+10 }\right)-{\beta _{0}-\beta }_{1}rin\left ({t }\right)-\cdots }\right. \\ \displaystyle \qquad -\left.{\vphantom {yout\left ({t+10 }\right)-{\beta _{0}-\beta }_{1}rin\left ({t }\right)-\cdots }\beta _{18}yout\left ({t-6 }\right) }\right)yout\left ({t-6 }\right)=0 \end{cases}\tag{41}\end{align*}$$

Therefore, the best parameters matrix $\beta$ of MR model are also the OLS estimators which satisfy the normal equations. After matrix calculation, $\beta$ can be expressed as Eq.(42): $$\begin{align*} \boldsymbol {\beta }=&{(\beta _{0},\beta _{1},\beta _{2},\cdots,\beta _{18})}^{\prime } \\=&\left ({X_{n}^{\prime }X_{n} }\right)^{-1}X_{n}^{\prime }yout(t+10)\tag{42}\end{align*}$$

B. Design of MR-WNN-PID

1) Structure of Model

The structure of MR-WNN-PID control system can be drawn as Fig. 3. MR-WNN-PID is designed by combining MR predictor and WNN-PID algorithm. The difference between MR-WNN-PID and WNN-PID is that: the system output of WNN-PID at time $t$ is replaced by the predicted system output of MR predictor at time $t+p$ in MR-WNN-PID.

FIGURE 3.

Structure of MR-WNN-PID control system.

2) Implement of Model

The pseudocode of the MR-WNN-PID is presented as Method 4:

Method 4 MR-WNN-PID

1.

Begin

2.

Initialize the control system of MR-WNN-PID;

3.

Initialize weights $W_{mn}\left ({t }\right)$ and offsets $b_{l}\left ({t }\right)$ of WNN;

4.

While $(t\le max\_{}t)$ do:

5.

Update the simulation time: $\text {t}=\text {t}+1$ ;

6.

System input $rin\left ({t }\right)$ is updated;

7.

If predict begin==1 do:

8.

Solve parameters of MRP according to (39);

9.

Calculate $yout\_{}predict\left ({t }\right)$ according to MRP;

10.

Calculate $error\left ({t }\right)$ according to $yout\_{}predict(t)$

11.

Else

12.

Calculate $error\left ({t }\right)$ according to $yout\left ({t }\right)$ ;

13.

End If

14.

Adjust $W_{mn}\left ({t }\right)$ and $b_{l}\left ({t }\right)$ according to (28) to (37);

15.

Adjust $K_{p}\left ({t }\right)$ , $K_{i}\left ({t }\right)$ , $K_{d}\left ({t }\right)$ according to (20) to (27);

16.

Calculate $\Delta u\left ({t }\right)$ and $u\left ({t }\right)$ according to Eq. (3);

17.

Calculate $yout\left ({t }\right)$ according to Eq. (4);

18.

End while

19.

End

SECTION V. Simulation and Comparative Analysis

Section V is organized as follows: In part A, general design of simulation is provided.In part B, standard (unsaturated) simulation is provided. In part C, anti-interference (saturated) simulation is provided.

A. General Design of Simulation

In this study, 2 types of simulations are designed:

1. standard (unsaturated) simulation: this simulation aims to verify the feasibility (stability and convergence), performance (dynamic speed and steady-state error) and anti-interference ability of each algorithm. Standard simulation is an unsaturated simulation, which is discussed in the previous section.

2. anti-interference (saturated) simulation: this simulation aims to verify the stability (whether the algorithm can converge after long-term interference). Meanwhile, Saturated simulation proves that the system output is unsaturated in standard simulation.

Same configurations of both above 2 types of simulations are designed as follows:

1. In each AC, parameters are adjusted once;

2. Incremental digital PID algorithm is applied to implement the PID system;

3. The model of controlled object is $sys\left ({s }\right)=\frac {180}{1.5s^{2}+25s+1}+e^{-5s}$ ;

4. Sampling period is set as 1 ($ts=1$ second);

5. The input of control system is set as $rin\left ({t }\right)=1$ , which can make the system output unsaturated;

6. Simulations process is repeated 10 times (10 SPs) to find the statistical characteristics.

B. Standard (Unsaturated) Simulation

1) Simulation Design

Special configuration of standard (unsaturated) simulation is as follows: 1) Maximal simulation time is set as 8000 ($\max \_{}t=8000$ ); 2) The start time of interference is at $d\_{}t=4000$ ; 3) The interference duration is set as $I\_{}t=10$ ; 4) Interference intensity is set as $dst\_{}u(t)=0.5$ , which is added to the output of the PID controller. In summary: from simulation time 4500 to 4510, the control variable (output of PID controller) is changed to $u\left ({t }\right)+dst\_{}u(t))$ .

2) Simulation and Result

Simulation results of 10 SPs of BPNN-PID, WNN-PID, and MR-WNN-PID are drawn in Fig. 4.

FIGURE 4.

10 SPs simulations of 3 algorithms

Curves of system input, output, error, output of controller, parameters of $K_{p}$ , $K_{i}$ , $K_{d}$ , and simulation time are plotted.

The details of 10 SPs (each SP has 8000ACs, $max\_{}t=8000$ ) of BPNN-PID, WNN-PID and MR-WNN-PID are shown in Table 1.

TABLE 1 Detail of 10 SPs Simulations of 3 Algorithms

Table 1 includes 6 indices (Rising Time, Settling Time, Steady-state error, overshoot, max deviation, Recovery time of deviation) of 3 AI-CC control method.

3) Comparative Analysis

According to Fig. 4 and Table 1, results of anti-interference (unsaturated) simulations of BPNN-PID, WNN-PID and MR-WNN-PID are provided separately for the following comparative analyses:

4) Comparative Analysis About Control Speed

MR-WNN-PID method has the fastest speed (smallest rising time and settling time, mean of $Tr=263$ and mean of $Ts=331$ ) than BPNN-PID (mean of $Tr=290$ , mean of $Ts=357$ ), while WNN-PID has minimal settling time (mean of $Ts=356$ ). In addition, minimal rising time and settling time of BPNN-PID (mean of $Tr=290$ , mean of $Ts=357$ ) are reduced by both WNN-PID (minimal of $Tr=162$ , minimal of $Ts=206$ ), and MR-WNN-PID (minimal $Tr=143$ , minimal $Ts=185$ ) algorithm.

5) Comparative Analysis About Overshoot

Several results of WNN-PID and MR-WNN-PID have no overshoot in 10SPs, but all the results of BPNN-PID have overshoots in each SP. Besides, the overshoot of WNN-PID (mean of peek values of overshoot=0.0321) is smaller than BPNN-PID (mean of peek values of overshoot = 0.035)

C. Anti-Interference (Saturated) Simulation

1) Simulation Design

Special configuration of anti-interference (saturated) simulation is as follows: 1) Maximal simulation time is set to $10000\,\,(\max \_{}t=10000)$ , which support enough time to show the effects of interference; 2) The start time of interference is to 3000 ($d\_{}t=3000$ ); 3) The interference duration is set to 2000 ($I\_{}t\mathrm {=2000}$ ) which is a long-term interference; 4) Interference intensity (amplitude or strength of interference) is set to 0.5 ($dst\_{}u(t)=0.5$ ), which is added to the output of the PID controller, and the amplitude of interference is half of the system input. From simulation time 3000 to 5000, the control variable (output of PID controller) is changed into $u\left ({t }\right)+dst\_{}u(t)$ .

2) Simulation and Result

The results of both standard simulation and unsaturated simulation of BPNN-PID, WNN-PID and MR-WNN-PID are drawn in Fig. 5.

FIGURE 5.

Results of saturated and unsaturated simulation.

3) Comparative Analysis

Results of the above anti-interference simulations show that:

1. Comparative analysis about overshoot: WNN-PID and MR-WNN-PID have smaller overshot than BPNN in both standard and unsaturated simulations. This result is shown in Fig. 5(a) and Fig. 5(b).

2. Comparative analysis about anti-interference: The deviation of BPNN-PID after interference is greater than WNN-PID and MR-WNN-PID, and it cannot converge after interference. This result shows the BPNN-PID is most affected by interference. In contrast, WNN-PID and MR-WNN-PID have lower deviation after interference, and they can converge in the end. This result is shown in Fig. 5(a).

3. Comparative analysis about steady-state error: With long-term interference, WNN-PID and MR-WNN-PID have smaller steady-state error after the interference. BPNN-PID has the greatest steady-state error. This result is shown in Fig. 5(a).

SECTION VI. Theoretical Analysis for Stability

Section VI is organized as follows: In part A, the basis of theoretical analysis is described. In part B, theoretical analysis of stability is provided. In part C, the example of stability analysis is provided. In part D, the unstable AI-CC system is improved, which includes the detail of RBFNN-PID method and the improved RBFNN-S-PID method.

A. Basis of Theoretical Analysis

Based on the above discussions, BPNN-PID, RBFNN-PID and WNN-PID are AI-CC method, which combine NN method with classical PID control method. Therefore, all the system analysis methods of classical PID theories are applied to NN based method.

The research scale of controlled objects is limited to the common practical applications of classical PID control. In each AC, the control parameters $(K_{p}\left ({t }\right),K_{i}\left ({t }\right),K_{d}\left ({t }\right))$ are fixed value. Based on maturely recognized PID theories, AI-CC method can be converted to the linear time-invariant and approximate linear time-invariant system [28]. Moreover, stability analysis for high-order systems, delay systems, and inertial systems are discussed, and a specific stability analysis case for second-order delay system is provided as well.

According to the classical PID theories, the theoretical analysis and simulation of AI-CC can also be applied for multi-order systems with delay linear time-invariant and approximate linear time-invariant systems.

B. Theoretical Analysis of Stability

The transfer functions of AI-CC system include open-loop transfer function and closed-loop transfer function, which are expressed as Eq. (43) to (46). $$\begin{align*} G_{Ctr}\left ({s }\right)=&K_{p}+K_{i}\cdot s+K_{d}\cdot \frac {1}{s} \tag{43} \\ G_{Obj}\left ({s }\right)=&\frac {K}{\tau \sigma s^{2}+\left ({\tau +\sigma }\right)s+1}\cdot e^{-\tau s}\tag{44}\\ G_{Open}\left ({s }\right)=&G_{Ctr}\left ({s }\right)\cdot G_{Obj}(s)\tag{45}\\ G_{Close}\left ({s }\right)=&\frac {G_{Open}\left ({s }\right)}{1+G_{Open}\left ({s }\right)}\tag{46}\end{align*}$$

It is assumed that $G_{Obj}(s)$ can be divided into 2 parts: $G_{1}\left ({s }\right)=e^{-\tau s}$ , the delay section, and $G_{2}\left ({s }\right)=\frac {M(s)}{N}$ , the system model. $G_{Close}(s)$ represents closed-loop system, which can be expressed as $G_{Close}(s)=\frac {M(s)\cdot e^{-\tau s}}{N\left ({s }\right)+M(s)\cdot e^{-\tau s}}$ , and the frequency characteristic of open loop system is $G_{k}\left ({j\omega }\right)=\frac {M(j\omega)}{N(j\omega)}\cdot e^{-j\omega t}$ . If $G_{2}(s)$ is calculated as $G_{2}\left ({j\omega }\right)=A(\omega)\cdot e^{-j\theta (\omega)}$ , $G_{Obj}(s)$ can be expressed as $G_{Obj}(s)=A(\omega)\cdot e^{-j[\theta \left ({\omega }\right)-\omega t]}$ .

The equivalent criterions of Nyquist stability criterion are as follows

1. When $L\left ({\omega }\right)=0dB$ , if $\varphi \left ({\omega }\right)>-\pi$ , the system is stable.

2. When $\varphi \left ({\omega }\right)=-\pi$ , if $L\left ({\omega }\right) < 0dB$ , the system is stable.

Therefore, stability of control system can be judged according to the following steps, which is defined as judging method for stability (abbreviated as AI-S-CC method) in this study:

1. In order to ensure the real-time stability of the system, only the stable parameters (which are tuned by AI-CC methods and judged to make the system stable) are accepted;

2. All the unstable parameters are rejected. In this case, the previous stable parameters are retained and used continuously.

C. Example of Stability Analysis

For example, if the controlled object model is $G_{Obj}\left ({s }\right)=\frac {280}{s^{2}+25s+1}+e^{-2s}$ , the controller model is $G_{Ctr}\left ({s }\right)= k_{p}+\frac {k_{i}}{s}+k_{d}\cdot s=\frac {k_{d}\cdot s^{2}+k_{p}\cdot s+k_{i}}{s}$ , and the current time is $t$ . The control parameters at time $t$ are as follows: $K_{p}\left ({t }\right)=0.2, K_{i}\left ({t }\right)=0.2$ and$K_{d}\left ({t }\right)=0.2$ , which are pre-verified to stabilize the control system. Then the open-loop transfer function $G_{Open}\left ({s }\right)$ can be calculated.

According to the open-loop transfer function $G_{Open}\left ({s }\right)$ , the amplitude margin can be calculated as: $Gm=20\cdot {log}_{10}\left ({gm }\right)=-2.98dB < 0$ , and phase margin can be calculated as $Pm=pm=-93.7$ deg $>-\pi$ .

In each AC, amplitude and phase are calculated, the pseudocode is provided as Method 5, which can be implemented by different programming languages:

Method 5 Calculation of Amplitude and Phase

1.

sys=tf(180,[1.5,25,1],’inputdelay’,5);

2.

sys_pid=tf([kd2bc,kp2bc,ki2bc],[1,0]);

3.

sys_open=sys*sys_pid;

4.

[bode_mag,bode_phase,bode_w]=bode(sys_open);

5.

[bode_gm,bode_pm,bode_wcg,bode_wcp]=margin(bode

_mag,bode_phase,bode_w);

The bode plots of the control system with the latest $K_{p}\left ({t }\right),K_{i}\left ({t }\right)$ , $K_{d}\left ({t }\right)$ can be plotted in Fig. 6. According to the above theoretical analysis, the current online tuned parameters $K_{p}\left ({t }\right),K_{i}\left ({t }\right)$ , $K_{d}\left ({t }\right)$ can make the system work steadily, so the current online tuned parameters $K_{p}\left ({t }\right)\mathrm {, }K_{i}\left ({t }\right)$ , $K_{d}\left ({t }\right)$ can be adopted (not abandoned) for PID controller.

FIGURE 6.

Bode plot result.

D. Improvement of Unstable AI-CC System

1) RBFNN-PID Model (Unstable Model)

RBFNN-PID is studied in this section, because RBFNN-PID is an unstable AI-CC system. Assume that $X$ represents the input vector of RBFNN and $y_{m}\left ({k }\right)$ represents the output of RBFNN. $$\begin{align*} X=&\left [{ {net}_{1}^{\left ({1 }\right)}(t),{net}_{2}^{\left ({1 }\right)}(t),{net}_{3}^{\left ({1 }\right)}(t) }\right]^{T}\tag{47}\\ h_{j}=&exp\left ({\frac {\left \|{ X-C_{j} }\right \|^{2}}{2b_{j}^{2}} }\right)\!,\quad (j=1,2,\ldots,m)\qquad \tag{48}\\ y_{m}\left ({k }\right)=&w_{1}h_{1}+w_{2}h_{2}+w_{m}h_{m}\tag{49}\end{align*}$$

The detailed parameters of gradient descent method are as follows: $\Delta k_{p}$ , $\Delta k_{i}$ , $\Delta k_{d}$ are changed as shown in Eq. (50) to (56). $$\begin{align*} \Delta k_{p}=&-\eta \frac {\partial E}{\partial k_{p}}=-\eta \frac {\partial E}{\partial k_{p}}\frac {\partial y}{\partial \Delta u}\frac {\partial \Delta u}{\partial k_{p}} \\=&\eta \cdot error(k)\frac {\partial y}{\partial \Delta u}xc(1) \tag{50}\\ \Delta k_{i}=&-\eta \frac {\partial E}{\partial k_{i}}=-\eta \frac {\partial E}{\partial y}\frac {\partial y}{\partial \Delta u}\frac {\partial \Delta u}{\partial k_{i}} \\=&\eta \cdot error(k)\frac {\partial y}{\partial \Delta u}xc(2) \tag{51}\\ \Delta k_{d}=&-\eta \frac {\partial E}{\partial k_{d}}=-\eta \frac {\partial E}{\partial y}\frac {\partial y}{\partial \Delta u}\frac {\partial \Delta u}{\partial k_{d}} \\=&\eta \cdot error(k)\frac {\partial y}{\partial \Delta u}xc(3) \tag{52}\\ xc(1) =&error(k)-error(k-1)\tag{53}\\ xc(2) =&error(k)\tag{54}\\ xc(3) =&error(k)\!-\!2error(k\!-\!1)\!+\!error(k\!-\!2)\qquad \tag{55}\\ \frac {\partial y(k)}{\partial \Delta u(k)}=&\frac {\partial y_{m}(k)}{\partial \Delta u(k)}=\sum {w_{j}h_{j}} \frac {C_{ji}-x_{1}}{b_{j}^{2}}\tag{56}\end{align*}$$

The simulation result of RBFNN-PID is shown in Fig. 7. Compare with other methods, RBFNN-PID is unstable. RBFNN-PID has serious deviation and it cannot converge in the end.

FIGURE 7.

Simulation Result of RBFNN-PID.t

2) Improved RBFNN-S-PID (Stable Model)

In order to solve the above problem, RBFNN-S-PID method is proposed. In RBFNN-S-PID, the control parameters are judged according to the above AI-S-CC method. In RBFNN-S-PID, the adjusted $K_{p}\left ({t }\right)$ , $K_{i}\left ({t }\right)$ and $K_{d}\left ({t }\right)$ are judged in each AC. The effect of the improved RBFNN-S-PID is shown in Fig. 8.

FIGURE 8.

10SPs Simulation for unstable RBFNN-PID-S systems.

Comparing Fig. 7 with Fig. 8, the RBFNN-S-PID system is much more stable: 1) All the RBFNN-S-PID results are convergent; 2) RBFNN-S-PID has less overshoot.

Throughout the online tuning process, all adjusted parameters are judged. RBFNN-PID is unstable, but RBFNN-S-PID with AI-CC-S method is stable. From the above theoretical analysis and simulation, stability problem is solved.

SECTION VII. Conclusion

In this study, WNN-PID and MR-WNN-PID are proposed, which improve the performance of existing BPNN-PID and RBFNN-PID methods. The stability analyses of the above methods are implemented, and the unstable problem of RBFNN-PID is solved. Comparative simulation results among BPNN-PID, WNN-PID and MR-WNN-PID methods are summarized in Table 2.

TABLE 2 Comparison of Different AI-CC Systems

Comparative indicators are designed as follows: control speed characteristic is measured by $Tr$ and $Ts$ indicator. Capability of output tracking input is measured by steady-state error. Contents of parameter configuration work, workload and operating time are estimated, hence the intelligent level of AI-CC system and the ability requirements for engineers can be evaluated.

According to Table 2, conclusions of this study can be summarized as follows:

1. The improvements of the proposed WNN-PID method are as follows: WNN-PID method is a stable (100% stable) and available control method in both saturated and unsaturated test. The control speed of WNN-PID method (means of $Tr=296$ , means of $Ts=356$ ) is better than BPNN-PID method (means of $Tr=290$ , means of $Ts=357$ ), but it is lower than MR-WNN-PID method (means of $Tr=263$ , means of $Ts=331$ ). The steady-state error of WNN-PID (steady-state error=0.0152) is bigger than both BPNN-PID method (steady-state error=0.0088) and MR-WNN-PID method (steady-state error=0.0134). WNN-PID has the fastest recovery speed (anti-interference), and it can converge in the end (recovery time=384). The effect of overshoot inhibition of WNN-PID (overshoot=0.0321) is the best, and WNN-PID also have several simulation results without overshoot in 10 SPs.

2. The improvements of the proposed MR-WNN-PID method are as follows: MR-WNN-PID method is a stable and feasible method in both saturated and unsaturated test. MR-WNN-PID method has the fastest control speed (means of $Tr=263$ , means of $Ts=331$ ). The steady-state error of MR-WNN-PID method (steady-state error=0.0134) is smaller than WNN-PID, but bigger than BPNN-PID. the recovery speed of MR-WNN-PID method (recovery time=477) is longer than BPNN-PID (recovery time=460) and WNN-PID (recovery time=384), but it can converge in the end. MR-WNN-PID method has several simulation results without overshoot in 10 SPs.

3. The improvement of the proposed RBFNN-S-PID: RBFNN-S-PID is stable after combing RBFNN-S-PID and AI-CC-S method, while RBFNN-S-PID is unstable.

Future Work Are as Follows: More accurate predictors are expected to further improve the performances of AI-CC methods. For example, Pythagorean fuzzy interaction power Bonferroni mean (PBM) operator and weighted Pythagorean fuzzy interaction PBM operator for multiple attribute decision making (MADM) methods [28] can be applied to AI-CC methods. These two new operators considered the relationship between membership and non-membership function of Pythagorean fuzzy numbers (PFNs), which improved the accurate result of the intelligent MADM system. Moreover, better methods of configuration and initialization for the above optimization and neural networks are expected to further improve performance of AI-CC control system.