Fault prediction method for nuclear power machinery based on Bayesian PPCA recurrent neural network model

Jun Ling · Gao-Jun Liu · Jia-Liang Li · Xiao-Cheng Shen · Dong-Dong You

Abstract Early fault warning for nuclear power machinery is conducive to timely troubleshooting and reductions in safety risks and unnecessary costs. This paper presents a novel intelligent fault prediction method, integrated probabilistic principal component analysis (PPCA), multi-resolution wavelet analysis, Bayesian inference, and RNN model for nuclear power machinery that consider data uncertainty and chaotic time series. After denoising the source data, the Bayesian PPCA method is employed for dimensional reduction to obtain a refined data group. A recurrent neural network (RNN) prediction model is constructed, and a Bayesian statistical inference approach is developed to quantitatively assess the prediction reliability of the model.By modeling and analyzing the data collected on the steam turbine and components of a nuclear power plant, the results of the goodness of fit, mean square error distribution, and Bayesian confidence indicate that the proposed RNN model can implement early warning in the fault creep period. The accuracy and reliability of the proposed model are quantitatively verified.

Keywords Fault prediction · Nuclear power machinery ·Steam turbine · Recurrent neural network · Probabilistic principal component analysis · Bayesian confidence

1 Introduction

Using a real-time monitoring system to collect the operations data of mechanical equipment in nuclear power plants (NPPs) for early warning in the early stage of equipment failure allows troubleshooting, which avoids major safety accidents, reduces unplanned shutdown maintenance of units, and reduces costs [1-3]. The establishment of a data-driven prediction model for mechanical equipment fault prediction has become an important means of predictive maintenance, and research on nuclear power machinery has gradually increased because of the recent and rapid development of artificial intelligence algorithms and big data technology [4-6]. Xie et al. [7] designed an online early warning system to track and predict the critical reaction of a nuclear reactor through two independent online simulation systems. Qian et al. [8] presented a hierarchical multi-dimensional method for fault detection of an NPP main pipeline. Min et al. [9] used a pattern recognition early warning system developed with AAKR technology to demonstrate the effectiveness of a real-time monitoring and early warning system for NPPs.Peng et al.[10] utilized the feature selection ability of association analysis and the depth confidence network (DBN) method to detect faults in nuclear power machinery.Yao et al.[11]introduced a fault diagnosis method for NPP full range simulators based on state information imaging. By using machine learning and image processing technology, historical data and synthetic grey image data are analyzed,and the system learns to achieve image feature extraction and classification to perform fault diagnosis.

A variety of data-driven approaches are used in the fault prediction of large mechanical equipment. Qin et al. [12]employed an approach based on time series and Bayesian discriminant analyses to solve the problems of type identification and diagnosis of concurrent faults without characteristic parameters in rotating machinery. Mehrdad et al.[13] presented a nonparametric single spline regression approach to construct the power curve model of the generator set. Aye and Heyns [14] proposed an optimal Gaussian process regression through the combination of simple mean value and variance to predict a low error rate for the remaining service life of low-speed bearings. The judgment methods for regression model accuracy are continuously developing. Jiang and Yin [15] proposed and applied a recursive total principle component regressionbased design and implementation approach for efficient data-driven fault detection for vehicular cyber-physical systems. Gao et al. [16] established a partial least squaresaided data-driven model predictive control approach to improve prediction accuracy. Herp et al. [17] developed a statistical method of online extraction and prediction of turbine state based on Bayesian inference.The residuals of bearing temperature measurement were inferred online,and the prediction probability is calculated by the sample model and the risk function describing the state transition probability to predict the fault state in advance. Li et al.[18] proposed a method for rolling bearing fault identification based on the multifractal and grey system theories,aiming at the non-equilibrium and nonlinear characteristics of bearing vibration signals and the complexity of the distribution of state indication information in the signal.Liu et al. [19] presented the thermal component of a fault prediction method based on the convolutional neural network(CNN)to address the disastrous consequences caused by thermal component faults of gas turbines. This method shows that the CNN is a feasible method to resolve thermal component fault detection.Liu and Karimi[20]established two machine learning models based on an artificial neural network and a high-dimensional model representation to predict the operation characteristics of steam turbines and air compressors and provided a basis for continuous health monitoring and fault diagnosis.

Deep learning can directly reflect the characteristics through the training sample data to reduce the influence of assumptions and simplification on calculation results and has recently been widely used for mechanical fault identification [21-23]. The RNN model shows good performance in capturing temporal correlations in data and can store and transmit the sequence information multiple times. Liu et al. [24] developed an RNN-based fault identification approach that uses a denoising auto-encoder based on a gated recursive unit to predict multiple vibration values of rolling bearings in subsequent time series. Hadi and Shahnazari [25] proposed a fault detection and isolation(FDI)method.Based on the RNN,this method models and inverts the nonlinear system, establishes a factory prediction model, and makes use of the residual generated from the model for fault identification. Wang et al. [26]developed an RNN-based algorithm to effectively handle the multi-classification fault diagnosis for wind power systems. Palau et al. [27] employed Weibull time to an event-RNN algorithm for distributed collaborative prognostics. The industrial gas turbine unit data and c-mapps engine degradation dataset are used in the experiment.Wang et al. [28] analyzed the motor vibration signal and multi-scale stator current signal and presented a multiresolution and multi-sensor fusion network model for motor fault diagnosis based on RNN.

The dimension of the data must be reduced, and more refined information must be used to conduct a comprehensive analysis of the collected data to refine and simplify the research. Principal component analysis (PCA) can effectively reduce the dimensionality of compressed data by retaining the original data feature information and solve the multi-variable correlation problem to reduce the complexity of the problem analysis and is widely applied in big data processing, pattern recognition, and image processing fields.Li et al.[29]established an optimized PCA model to perform fault detection of sensors in NPPs and verified that the model can detect and reconstruct the fault sensors well by simulation. Prusty et al. [30] employed PCA to reduce the dimension of a large number of plant signals transmitted by the Prototype Fast Breeder Reactor (PFBR) in a NPP, improving the decision-making capability of the operator in catastrophic conditions. Wu et al. [31] constructed a fault detection model of a pressurized water reactor in a NPP based on the BN-FDD system framework.PCA, fuzzy theory, and data fusion were used to promote data accuracy, and multiple sensor data were combined into one node data. Sharifi and Langari [32] divided the measurement space into several local linear regions associated with a PPCA model and presented a sensor fault diagnosis method for a nonlinear system by considering the data uncertainty. Xiang et al. [33] utilized the PPCA denoising model for rolling bearing fault prediction.In this model,the subspace of the principal component retains the more useful original information and fault signal, and the noise and related linear information are projected into the remaining subspace.

The above literature introduces the analysis methods of fault prediction for different mechanical equipment. The following problems regarding the application of fault detection and early warning for nuclear power rotating mechanical equipment must be improved: (1) imperfections in the source data and multiple variable redundancies;(2) monitoring and early warning during the creep period of equipment failure;and(3)the quantitative evaluation of the reliability of the prediction model.

This study combines pattern recognition technology and deep learning to present a fault prediction approach for steam turbines,pumps,and other mechanical equipment in NPPs based on Bayesian PPCA RNN to improve the aforementioned issues. After wavelet packet threshold denoising, the signal data are dimensionally reduced by using a Bayesian PPCA method. A fully connected RNN prediction model is established and verified by using the goodness of fit and mean square error. The model reliability is quantified by calculating the Bayesian factor and confidence. Combined with the prior information in the historical dataset, the proposed method calculates the residual between the prediction and hypothetical health values to find the unit failure during the creep period.

2 Data integration analysis

of the first k principal components in q principal components reaches a certain contribution rate (as determined by the actual situation), only the k-dimensional data after the dimension reduction can be used for the subsequent data processing.

3 Fully connected RNN prediction model

3.1 RNN model construction

In this study, an RNN suitable for sequence data modeling is used to predict a time series. The neurons with a cyclic structure retain and apply the state information of the previous moment as memory to current output calculation; thus, the nodes between the same hidden layers are connected. The RNN can transmit information laterally among neurons and partially express correlations within the data.This information transmission mode matches well with the state process of operational nuclear power machinery. The running state at a given moment will have a certain impact on the running state at the next moment,and the collected data also correlate.

In the model, the same weight parameters are used at different times, and the activation function uses the rectified linear unit (ReLu) function uniformly. Units with a certain amount of m are connected, and the last unit provides the output value. The hidden layer state s0of the initial input is a random value, and the prediction value of the next τ time point outputted by the last unit is ^yt+（n+1）τ.For time t+nτ,formula(8)is used to calculate the forward propagation from input to output:

In formula (8), ctand otare used as intermediate variables to participate in the backpropagation calculation; a and b represent the bias terms of the hidden layer neurons and output layer neurons, respectively.

After constructing the model,a backpropagation through time (BPTT) algorithm [35] is employed to train the RNN model. Based on formula (8), the loss function is established, and the minimum value of the loss function is calculated. The negative log likelihood function is used to establish the loss function:

where y（ti）is the ith element of the output yt,is the predicted data, and n is the number of data points in each group. After the loss function is determined, the partial derivative values of each time step, such as Eq. (10), are accumulated by using optimization strategies such as random gradient descent, to update the weights and bias:

3.2 Model reliability verification

The reliability represents the ability of the model to accurately reflect the characteristics of the dataset and to predict the data information of future time nodes.To verify the reliability of the model, three methods are introduced:(1) goodness of fit R2; (2) mean square error (MSE); and(3)Bayesian confidence.The model has high precision and good reliability when R2∈（0 ～1）{R2is closer to 1,MSE smaller}. The Bayesian hypothesis test method [36-38]considers the data uncertainty and intuitively verifies the model reliability.

where λ reflects the confidence degree of the prediction model. When λ →0, the confidence degree of the support model is 0% and the model reliability is low. When λ →∞, the confidence degree of the support model is 100%, and the reliability is high.

3.3 Fault prediction

In this study, part of the data is used as a training set to construct the RNN prediction model. The abnormal signal is identified by setting the threshold in advance. The threshold emaxis the value of the maximum residual in the training set. In the testing set, residuals between the monitoring value of each time point and the predicted value are expressed as follows:

and when Θtest＞emax, an alarm will be given. In the real-time condition monitoring of the steam turbine unit,when the residual exceeds the set threshold for a long time,the unit is considered a failure.

4 Illustration

This study uses rotating speed signal data of a pressure cylinder of a nuclear power turbine in April 2019 to explain the algorithm flow and model building. The rated rotating speed of the sampled turbine is 1500 rpm. The dataset consists of two rotor speeds, one bearing group speed, and 720 time points. The data before April 20 are used as the training set to build the model and train the weights, and the data after April 20 are used as the verification set to verify the model reliability.

Figure 2 shows a flowchart of fault prediction by the RNN model.After the original three-dimensional rotational speed signal is denoised by DWPT and reduced by PPCA,a one-dimensional time series signal is obtained.The signal data with the delay time and embedding dimension optimized by the enumeration method are set as a training dataset. The RNN model after training characteristic parameters is used to predict the value of the next time point, and the residual between the prediction and monitoring values is calculated for early warning.

4.1 Data denoising and dimensional reduction

Three speed signals are denoised by the DWPT. The time series signal is decomposed into three levels by using the db8 wavelet packet, and the wavelet coefficients of each point are obtained. The wavelet coefficients are filtered according to the Bayesian threshold approach,and the signal is reconstructed. Figure 3 shows the noise and denoising data of the bearing group speed signal. The denoising signal is very similar to the original signal, and the trend is consistent. The feature information of the original signal is retained.

The dimensions of each type of PPCA data are reduced to more than 70% of the cumulative variance contribution rate of the retained dimension. Thus, the proportion of information after dimensional reduction is more than 70%.Three speed signals are dimensionally reduced in this case.Table 1 shows the results of dimensional reduction, where wiis the weight value of each component signal to the principal component after Bayesian PPCA of the rotating speed signals. The component signal contributes more to the result analysis for greater absolute values of the weight parameter. The cost of training calculation can be decreased,and one-dimensional data can retain most of the information from the original data without any signal distortion by reducing the three-dimensional rotating speed signal to the one-dimensional signal corresponding to PC1(Table 1).

Table 1 PCA weight and contribution rate of rotating speeds

4.2 Determining the embedding dimension and time delay of the input layer

The time series data after dimensional reduction by PPCA are prone to chaos;therefore,the input layer must be determined by phase space reconstruction for the prediction model.By using the enumeration method with various time delay and embedding dimension combinations, the changes in the R2and MSE parameters of the RNN model training are analyzed. Figures 4 and 5 show the changing trends of R2and MSE with different time delays and embedding dimensions, respectively. R2and MSE slowly decrease and increase, respectively, with an increase in time delay in the training set; in the verification set, R2descends steeply,and MSE increases(Fig. 4).MSE and R2change in waves as the embedding dimension increases(Fig. 5). Additionally, R2reaches a maximum at m=4 in the training and verification sets, and MSE reaches a minimum at m=4，7，9 and m=4，14 in the training and verification sets, respectively. Thereafter, to ensure the maximum value of R2and the minimum value of MSE,the optimal embedding dimension m=4 is chosen. For the optimal time delays, t=1 and t ＞1 indicate single-step and multi-step training, respectively. Larger t values predict data a later times, but a t of 2 or 3 provides optimum accuracy (Fig. 4).

Table 2 Values of R2, MSE,and λ

4.3 RNN model prediction and verification

The fully connected RNN model is constructed according to the optimized time delay and embedding dimension.For the rotating speed principal component,the time delay is the interval time between two adjacent input data, and the embedding dimension is the number of RNN input units. After model training, the R2, MSE, and Bayesian confidence λ indexes are employed for training and verification datasets to validate the accuracy and reliability of the model. When R2is closer to 1, MSE is smaller, λ is higher, and the model has high accuracy and good reliability.Table 2 lists the values of R2,MSE,and λ in the case of time delay τd=2 and embedding dimension m=3，4，...8, respectively. Under the parameters τd=2， m=4, the R2values of the training set and verification set are both above 0.94, MSE values are maintained at the order of 0.001 magnitudes,and the λ values of the verification set are stable at 93% (Table 2, Figs. 4 and 5).

Figure 6 shows the comparison between the monitoring and predictive values of the rotating speed signal under three different conditions. Figure 6a illustrates the results of the proposed model and shows that the two curves coincide well.To analyze the effect of noise reduction,the rotating speed signals without DWPT are trained and predicted (Fig. 6b). The difference between two curves in the figure indicates that noise reduction of the source data is necessary. Furthermore, the traditional artificial neural network(ANN)model is applied for comparative analysis,and the predictive results are described in Fig. 6c. The degree of agreement between the two curves is lower than that of the RNN model in Fig. 6a. The R2values of the training and verification sets of the ANN model are 0.963 and 0.933,respectively,and the MSE values of the training and verification sets of the ANN model are 0.017104 and 0.022082,respectively.The results of these two parameters of the RNN model under m=4， τd=2 in Table 2 are better than those of the ANN model. Thus, the accuracy and reliability of the proposed model are validated.

4.4 Fault prediction of a two-stage impeller

A dataset with fault points is applied to further verify the model reliability and test the early warning function for faults with the model.Figure 7 shows cracks(found in late May 2017 during major maintenance) in a two-stage impeller of a turbine unit in an NPP. The flaw detection results show that five blades have cracks, of which the shortest and longest measure approximately 29 and 45 mm,respectively.

Table 3 RNN model parameters and fault early warning parameters

The rotating speed and vibration signals from February 10 to March 16, 2017, are extracted from the monitoring system as the experimental dataset. Both the speed and vibration datasets consist of 34 days of monitoring data with 24 time points per day and a total of 816 time points.The signal data are divided into training (February 10 to March 2), verification (March 2-4), and testing (after March 5) datasets for modeling and fault prediction. Similar to the previous case,the two datasets are employed for the RNN model prediction after noise reduction and reconstruction of the embedding dimension and time delay.Table 3 lists some model prediction parameters and fault early warning parameters.The results of R2and MSE in the training set indicate good RNN model reliability. The positive threshold,which is the maximum positive residual allowed in the early warning system,is set using 1.2 times the maximum residual in the training set. The negative threshold,which is the maximum negative residual allowed in the early warning system, is set using 1.2 times the minimum residual in the training set. Table 3 shows the values of the positive and negative thresholds. Two warning lines are drawn in orange (Figs. 8 and 9). When the green residual curve exceeds the warning line, the blue monitoring signal curve still fluctuates normally (Fig. 8).The monitoring system, which sets fixed alarm thresholds for specific signal values instead of residuals,was recorded to give an alarm at 17:00 on March 7, whereas the alarm time of the RNN model is 5:00 on March 5 (60 h in advance). Similarly, Fig. 9 shows the early warning effect of another vibration signal. Table 3 lists the corresponding alarm times of the RNN model and monitoring system.The RNN model produces an alarm 44 h in advance. The experimental results indicate that the cracks in the twostage impeller cause the abnormal signals of turbine speed and vibration; the RNN model can predict the anomaly well. The model is further validated as reliable.

5 Conclusion

Because of multiple variable redundancies and turbine data imperfections, the Bayesian PPCA method is used to preprocess the DWPT denoising data and obtain a dataset with a high signal-to-noise ratio and low dimension. The rotating speed signal is reduced from a three- to onedimensional principal component,and the contribution rate is more than 70%.

A fully connected RNN prediction model is established.The goodness of fit of each signal data is calculated to be higher than 0.93,and MSE fluctuates on the order of 0.001,which verifies the model reliability. Furthermore, a Bayesian hypothesis testing method, which considers the data uncertainty and prior information of the training set, is employed to quantify the model confidence. The Bayesian confidence values of the verification set under different embedding dimensions are calculated at more than 90%.In the comparison case study of the two-stage impeller cracking and the monitoring system, the RNN prediction model produces alarms 60 and 44 h in advance for the rotating speed and vibration signals, respectively. The prediction results indicate that the RNN model can effectively identify faults during the creep period.