An effective method for performance evaluation of ACLS based on improved grey analytic hierarchy

LI Yan, YIN Haitao

(Xi’an Aerospace Propulsion Testing Technology Research Institute, Xi’an 710025, China)

Abstract： The performance evaluation of automatic carrier landing system (ACLS) is an important part in the field of carrier aircraft landing control. Combining grey analytic hierarchy theory and data normalization theory, an improved grey analytic hierarchy method is introduced to evaluate the performance of ACLS. A complete performance evaluation indicators system of ACLS is established, and the definition and calculation formula of each indicator are provided. The grey analytic hierarchy model is modified to improve the real-time performance of the algorithm, where traditional expert scoring sampling matrix is substituted by an indicator normalized sample matrix. Taking a certain ACLS as an example, the experimental simulation is carried out, and the simulation results verify the reliability and the accuracy of the improved grey analytic hierarchy method.

Key words： automatic carrier landing system (ACLS)； evaluation indicators system； grey analytic hierarchy model； normalization sample matrix

0 Introduction

The automatic carrier landing system (ACLS) has been widely studied for the past decades. Many methods have been developed based on different research backgrounds and objects to apply to different scenarios. The current issue is how to evaluate the performance of ACLS scientifically and reasonably.

There are few studies on the evaluation of ACLS. The evaluation of air combat performance of the fighter aircraft is studied based on AHP method[1-2]. The safety assessment of civil aviation landing stage is studied based on grey clustering method[3]. An evaluation method is established for the suitability of carrier-based aircraft on landing[4]. The evaluation methods are compared based on landing rate, landing quality grade, landing point error and glide track error[5]. A measurement and evaluation system is proposed based on high-speed camera technology[6]. These references lack research on the evaluation system of landing, and the dimension of performance evaluation is relatively single, which cannot reflect the influence of variables on performance evaluation in the process of landing.

Grey analytic hierarchy[7-8]is widely used and has yielded satisfying results in many fields. However, the sample matrix determined by experts of grey analytic hierarchy cannot meet the requirement of real-time evaluation of ACLS. Non-dimensional normalization method is an objective method for evaluating the target, and can carry out on-line computing. It can replace expert scoring sample, and compensate the disadvantage of the grey analytic hierarchy.

Starting from the process of landing, the evaluation indicators structure is established by analyzing the key factors of each stage in process. Combining grey analytic hierarchy and non-dimensional normalization method, an improved grey analytic hierarchy is proposed based on normalization sample matrix to evaluate the performance of ACLS. Firstly, according to the characteristics of ACLS, the main evaluation indicators are defined and the evaluation indicators system is established. Then, the complete model of improved analytic hierarchy is given. Finally, the simulation is carried out. The results show the improved grey analytic hierarchy is reasonable and effective for the evaluation of ACLS.

1 Establishing evaluation indicators system

1.1 Establishing principle

ACLS is a complex system with maneuverability, which is affected by aft flow and deck motion. Two aspects are considered to establish a reasonable evaluation system.

1.1.1 Modulize evaluation system

To make a reasonable and accurate comprehensive evaluation of ACLS,a reasonable evaluation of each subsystem should first be made. Therefore, the total system are divided into several modules according to its different performance requirements.

1.1.2 Select evaluating indicators for each module

Selecting evaluation indicators is the most important step in performance evaluation. An appropriate evaluation indicator can not only make reasonable and effective evaluation of the module, but can also provide references for further optimization of the system[9-10].

1.2 Performance indicator system of ACLS

According to the above principles, the evaluation indicator system is established based on landing performance. Combined with controller parameters and indicators, the performance indicator system of ACLS is established in Fig.1[11].

Fig.1 Performance indicator system of landing controller

In Fig.1, the performance indicator system of ACLS includes gliding angle deviation, maximum attack angle, maximum overload, horizontal position deviation, vertical height deviation and trajectory tracking accuracy. The indicators are described as 1)-6).

(1)

whereγirepresents theith gliding angle, andγ0=3.5°.

2) Maximum attack angleαmax： the maximum attack angle of aircraft during landing.

3) Maximum overloadnymax： the maximum overload of aircraft during landing.

4) Horizontal position deviation Δx： the horizontal distance between actual landing point and ideal landing point of aircraft.

5) Vertical height deviation Δh： the vertical distance between the actual glide path and the ideal glide path of aircraft.

6) Trajectory tracking accuracye(x,h)： the deviation between the tracking trajectory and the ideal trajectory on landing.

(2)

wherexgandhgare the ideal trajectories in the longitudinal and vertical directions, andxandhare the tracking trajectories.

2 Improved grey analytic hierarchy model

2.1 Basic theory of grey analytic hierarchy

For the system to be evaluated, a typical grey analytic hierarchy modelis established as shown in Fig.2.

Fig.2 Process of grey analytic hierarchy

2.1.1 Building hierarchy of evaluation object

First, target (the evaluation object) is separated into several elements； then these elements are divided into several groups according to different attributes, forming different levels. In this hierarchy, the same level elements dominate the elements in the next level while being controlled by the upper level elements. The root elements are the required evaluation indicator.

2.1.2 Calculating indicator weight

The weight of each indicator is different in the evaluation indicator system. Therefore, the weight value of each indicator needs to be determined first. Generally, the analytic hierarchy process (AHP)[12]is used to calculate the indicator weight. Quantitative comparison of the importance between each pair of indicators on the same level is given according to expert’s judgment, and the results are constructed into a judgement matrix. Then calculating the characteristic vector and the maximum characteristic value of this matrix, and undertaking consistency checking for the matrix, finally the weight value of each indicator is obtained.

2.1.3 Determining evaluation sample matrix

In the indicator evaluation, the grades are divided into several levels according to the actual situation. For example, grades are divided into five levels： very good, good, fair, poor and very poor, giving the corresponding scores of 9, 7, 5, 3, 1.

Assuming thatpexperts participate in the evaluation and thek-th expert gives a score ofdijkfor indicatorVij, the evaluation sample matrix is

(3)

2.1.4 Determining evaluation grey type

Determining the evaluation grey type is to determine the level of grey type, grey degree of grey type and whitening function of grey degree through qualitative analysis. Typical whitening functionfn(x) has three forms[7].

1) Upper function. The grey degree is ⊗∈[0,d1,∞], and the whitening function is shown in Fig.3(a).

(4)

2) Intermediate function. The grey degree is ⊗∈[0,d1,2d1], and the whitening function is shown in Fig.3(b).

(5)

3) Lower function. The grey degree is ⊗∈[0,d1,d2], and the whitening function is shown in Fig.3(c).

(6)

(a) Upper function

(c) Lower function

2.1.5 Calculating grey evaluation factor

For the evaluation indicatorVi, the grey evaluation coefficient of theNth(N=a,b,c,d…) grey type isxi,N, the total grey evaluation coefficient isxi, and the grey evaluation weight ofNth grey type isri,N, so there is

(7)

Then weight vectorriof normalized grey evaluation is

ri=(ri,1,ri,2,…,ri,N).

(8)

After integrating grey evaluation weight vector ofmevaluation indicators to each grey type, the grey evaluation weight matrixRis obtained by

(9)

2.1.6 Results of evaluation

Suppose the vector ofmevaluation indicators isw=(w1,w2,…,wm), and the comprehensive evaluation result is

B=w·R=(b1,b2,…,bN).

(10)

The information provided byBcould determine the degree of grey type of object evaluated by maximum membership principle.

2.2 Improved evaluation sample matrix

In the typical grey analytic hierarchy method, the sample matrix is determined by experts scoring the indicators. When the sample size is large, it would require heavy load of manual work. So the typical grey analytic hierarchy method is inadequate for systems that need to be evaluated during working process and not practical for engineering applications.

Experts in the same field usually have similar recognitions of the same indicator. Therefore, in this paper, reasonable non-dimensional normalization of the indicators is used as the basic recognition of the indicator by different experts. On this basis, perturbation is carried out to simulate the slight differences of different experts on the same indicator, which can replace the expert scoring and can greatly improve the real-time performance of the algorithm.

It takes three steps to determine the evaluation sample matrix.

1) In the evaluation, the indicator grades are divided into several levels according to the actual situation. For example, grades are divided into five levels, namely very good, good, fair, poor and very poor, given the corresponding scores of 9, 7, 5, 3, 1.

2) The corresponding non-dimension normalization is carried out formevaluation indicators. According to the different characteristics of each indicator, different normalization methods are shown as follows[13].

a. Give a reasonable range and compare.Assigned fixed values between [0,1] to the original data according to the interval in sections, then averaged. Such as trajectory tracking accuracye(x,h), etc.

b. Compare original data with the reasonable value.If it is enabled, assigned 1 or else assigned 0, then averaged. Such as maximum attack angleαmax, maximum overloadnymax, etc.

c. Range method.Use the maximum and minimum value of the variables to convert original data to data with specific range boundary. This method can eliminate influence of the dimension and degree level and change weight of variables during analysis to solve the problem of different measurement.

Positive indicator is

(11)

Inverse indicator is

(12)

wherey∈[0,1], the maximum value is 1 and the minimum value is 0. Table 1 shows the normalization method of each indicator in the indicator system.

Table 1 Normalization method of each indicator

So the normalized vectordof the evaluation sample is

(13)

wheredi(i=1,2,…,m) is the normalized value of thei-th evaluation indicator.

(14)

(15)

(16)

whereDis the normalized sample matrix.

The normalized sample matrix simulates the expert scoring, and meets the real-time requirements of the system evaluation. From the above analysis, it can be seen that instead of the traditional expert scoring matrix, the normalized sample matrix improves greatly the real-time performance of the ACLS evaluation.

3 Simulation example

Now the performance of a certain ACLS is evaluated by using the improved method of grey analytic hierarchy. This system indicator refers to Ref.[14]. The indicator hierarchy is shown in Fig.1.

3.1 Calculating synthetic weight of evaluation indicator

Obtain the judgement matrixA1of the system through comparing the importance of each pair of indicators.

(17)

Calculate the feature vector ofAand obtain the synthesis weight of the target corresponding evaluation indicator

w=[0.026 0,0.081 6,0.081 6,0.193 2,0.193 2,0.424 4].

(18)

3.2 Determining evaluation grey type and evaluation sample matrix

Evaluating performance of ACLS, the evaluation grey degree set isk={very good, good, fair, poor}. The control system simulation results in Ref.[14] are sampled and normalized according to section 2.2. The normalized vectordis

Then the evaluation sample matrixDis

(19)

The corresponding grey degree and whitening functions are as follows.

1)N=1, set the grey degree as ⊗1∈[0,9,∞], and whitening weight function asf1, which is shown as Fig.4(a).

2)N=2, set the grey degree as ⊗2∈[0,7,10], and whitening weight function asf2, which is shown as Fig.4(b).

3)N=3, set the grey degree as ⊗3∈[0,5,8], and whitening weight function asf3, which is shown as Fig.4(c).

4)N=4, set the grey degree as ⊗4∈[0,1,5], and whitening weight function asf4, which is shown as Fig.4(d).

(a) Whitening function f1

(c) Whitening function f3

3.3 Calculating grey evaluation weight vector and matrix

Based on the above steps, the ACLS grey evaluation weight matrixRis obtained by

(20)

3.4 Results of evaluation

The evaluation result of performance of the ACLS is

(21)

According to the principle of maximum membership degree,the ACLS performance belongs to the level of “very good”.

3.5 Results comparison

According to the performance indicator of the ACLS, four experts in this field are invited to score the indicator, and the sample matrixD0of expert evaluation is

(22)

According to the example of the section 3.1, the grey evaluation weight matrixR0is obtained by

(23)

The evaluation result of the controller is

(24)

According to the principle of maximum membership degree,the ACLS performance belongs to the level of “very good”. The expert evaluation results are consistent with the evaluation results based on the improved grey analytic hierarchy method, which proves the reliability and accuracy of the method. Compared with the traditional grey analytic hierarchy method, the improved method based on normalized samples has advanced real-time performance.

4 Conclusions

In view of the complex operation environment of ACLS, the improved grey analytic hierarchy method is presented to construct the normalized sample matrix with the theory of normalized data processing instead of the traditional expert scoring matrix. The performance of the system with the improved grey analytic hierarchy method is evaluated. The simulation results show that the improved grey analytic hierarchy method overcomes the limitations of traditional grey analytic hierarchy in evaluating real-time ACLS.