Trace-Norm Regularized Multi-Task Learning for Sea State Bias Estimation

ZHONG Guoqiang, QU Jianzhang, WANG Haizhen, LIU Benxiu, JIAO Wencong, FAN Zhenlin, MIAO Hongli, and HEDJAM Rachid

Trace-Norm Regularized Multi-Task Learning for Sea State Bias Estimation

ZHONG Guoqiang1), *, QU Jianzhang1), WANG Haizhen1), LIU Benxiu1), JIAO Wencong1), FAN Zhenlin1), MIAO Hongli2), and HEDJAM Rachid3)

1),,266100,2),,266100,3),, P. O. B 36,123,,

Sea state bias (SSB) is an important component of errors for the radar altimeter measurements of sea surface height (SSH). However, existing SSB estimation methods are almost all based on single-task learning (STL), where one model is built on the data from only one radar altimeter. In this paper, taking account of the data from multiple radar altimeters available, we introduced a multi-task learning method, called trace-norm regularized multi-task learning (TNR-MTL), for SSB estimation. Corresponding to each individual task, TNR-MLT involves only three parameters. Hence, it is easy to implement. More importantly, the convergence of TNR-MLT is theoretically guaranteed. Compared with the commonly used STL models, TNR-MTL can effectively utilize the shared information between data from multiple altimeters. During the training of TNR-MTL, we used the JASON-2 and JASON-3 cycle data to solve two correlated SSB estimation tasks. Then the optimal model was selected to estimate SSB on the JASON-2 and the HY-2 70-71 cycle intersection data. For the JSAON-2 cycle intersection data, the corrected variance () has been reduced by 0.60cm2compared to the geophysical data records (GDR); while for the HY-2 cycle intersection data,has been reduced by 1.30cm2compared to GDR. Therefore, TNR-MTL is proved to be effective for the SSB estimation tasks.

sea state bias (SSB); radar altimeter; geophysical data records (GDR); trace-norm; multi-task learning

1 Introduction

One of the goals of remote sensing is to measure the sea surface height (SSH) using satellite altimeter techno- logy. The SSH measurement is very important for determining and monitoring ocean currents and eddies (Wunschand Stammer, 1998), climate change, wave height and wind speed, and for studies in geodesy and ocean geophysics (Barrick, 1972). Sea level has continued to rise in recent years, mainly due to the melting of polar glaciers and the thermal expansion of upper seawater, which is induced by climate changes. Studies have showed that global sea level has risen by 10 to 20 centimeters over the past 100 years and will accelerate in the future (Dasgupta., 2009). Sea level rising has a significant impact on the so- cio-economy, natural environment and ecosystems inthe coastal areas. First of all, sea level rising may submerge some low-lying coastal areas. Second, it may increase the intensity of storm surges. Frequent of storm surges may not only endanger the lives and property of the people in the coastal areas, but also increase the level of land salini-zation and seawater intrusion. Without intervention, by 2100 most large coastal cities will face sea levels that are more than three feet higher than they are currently (Moore., 2018). Therefore, accurate measurement of the SSH has become more and more important and has received extensive attentions from experts in the areas related to marine sciences.

The satellite radar altimeter can efficiently measure glo- bal SSH (Wan, 2015), significant wave height (SWH), and wind speed (U). However, since the radius of curvatureof the wave trough is larger than the radius of curvature of the crest, the troughs can reflect more electromagnetic energy, which makes the measured average levels of sea surface generally lower than the true average levels. This effect is named as electromagnetic bias (Ghavidel., 2016). Electromagnetic bias and skewness bias are collectively referred to as the sea state bias (SSB). An illustration of SSB is given in Fig.1. With the development of precise orbit determination technology, SSB has replaced the orbital error and become the first error source for the SSH measurement (Yaplee., 1971; Elfouhaily., 2000; Coleman, 2001). Therefore, the estimation of SSB is very important for the accurate measurement of SSH.

The SSB estimation methods can be divided into theoretical models and empirical models. Theoretical models are generally not practical because it is difficult to obtain the required parameters (Coleman, 2001). The currently applicable empirical models include parametric models (Chelton, 1994; Gaspar., 1994; Zhou., 2012), non-parametric models (Gaspar and Florens, 1998; Gaspar., 2002; Labroue., 2004), and direct estimation methods (Vandemark., 2002). However, for empirical models, certain errors are always involved. Tran. (2010) pro- posed a method to improve the precision of satellite-de- rived sea-level measurements, which used a non-parametric model,., an enhanced three-dimensional (3D) SSB cor- rection model, to improve the accuracy of SSB estimation. The evaluation over the years on the JASON-1 data showed that the sea level height variance was reduced by 1.26±0.2cm2compared to the previous parametric models. In the paper of Miao. (2016), based on the HY-2 altimeter data, a non-parametric estimation method based on local linear regression was used to estimate the SSB on the 70-71 cycle intersections data, and it was proved that the effect was superior to the existing parametric models. However, the SSB estimation accuracies can be further improved.

Fig.1 Sea state bias (SSB) induced by waves troughs shadowing when measuring off-nadir at low elevation angles with high amplitude and frequency of waves.

2 Related Works

Recently, machine learning has developed rapidly, and been successfully applied in many fields. Particularly, many researchers try to use machine learning algorithms to estimate SSB. Based on the JASON-1 altimetry data, Li. (2013) have studied on a parametric model for SSB estimation, using linear regression to estimate the parameters, and obtained the optimal model. Miao. (2015) estimated SSB by fitting 32 ordinary least square (OLS) mo- dels with SWH and U on the JASON-2 radar altimeter data, and selected the optimal one for other SSB estimation tasks. Zhong. (2018) introduced an effective and efficient linear model called LASSO to the SSB estimation. It used the significant wave height and wind speed to fit the LASSO model.

However, all of the above methods were based on single task learning (STL) models and did not make use of the data from multiple altimeters for SSB estimation. In many applications, STL models usually solve the confronted tasks independently, but ignore the connection between the tasks. On the contrary, multi-tasking learning (MTL) models can deal with related tasks at the same time by extracting and utilizing the shared information across different tasks. Zhou. (2011) introduced a variety of MTL algorithms, such as Robust Low-Rank MTL, Clustered MTL, and Mean-Regularized MTL methods. Moreover, Bickel. (2008) utilized MTL methods to predict the effect of human immunodeficiency virus (HIV) therapy based on possible combinations of drugs. Even for drug combinations with few or no training examples, MTL models can still perform predictions (Bickel., 2008). Liu. (2019) proposed an MTL method for Alzheimer’s disease diagnosis, which achieved state-of-the-art performance in both tasks of disease classification and clinical score regression. Chen. (2018) used MTL for dangerous object detection in autonomous driving and achieved better object detection abilities and distance prediction performances compared to STL methods. Moreover, Wang. (2018) proposed a novel MTL approach for improving product title compression with user search log data, which outperformed the compared STL methods in both compression qualities and online business values (Wang., 2018). Although MTL methods have been successfully used in many applications, they have rarely been applied in the SSB estimation.

In the field of sea state bias estimation, data from multiple radar altimeters are available. Taking this into consideration, we introduce an MTL model, which is called trace-norm regularized multi-task learning (TNR-MTL), for the SSB estimation. Compared with traditional STL models, data from multiple radar altimeters are used and multiple correlated tasks are learnt simultaneously by ex- ploiting the shared information across the tasks, which greatly improves the generalization performance of TNR- MTL for SSB estimation.

In this work, we used the JASON-2 and JASON-3 cycle data to train TNR-MTL models and selected the optimal one, which is called Selected_TNR-MTL (SD_TNR- MTL), for new SSB estimation tasks. As we mentioned above, SSB is mainly composed of the electromagnetic bias and the skewness bias. The former usually depends on the SWH and U, while the latter is relatively much smaller. Hence, most works consider the electromagnetic bias as the main component of SSB and only estimate it instead. Following these previous works, we only estimate the electromagnetic bias as well. We use the SWH and U data to fit the TNR-MTL model, and only 3 parameters were set for each task (corresponding to two inputs and one bias, respectively). This improves the efficiency of TNR-MTL to a certain extent. In this paper, we evaluated the SD_TNR-MTL model for the SSB estimation on the JASON-2, JASON-3 and HY-2 cycle data. The results showed that SD_TNR-MTL obtained more accurate SSB estimation values than those of geophysical data records (GDR), which indicates that the SSB estimated by SD_TNR-MTL facilitates the correct measure-ment of SSH. Therefore, SD_TNR-MTL is feasible and effective for SSB estimation.

3 Trace-Norm Regularized Multi-Task Learning (TNR-MTL)

The STL models only learn one task at a time and ignorethe correlation between tasks, while MTL models can learnmultiple correlated tasks simultaneously by extracting and utilizing shared information across tasks. Fig.2 illustrates the difference between STL and MTL models. Fig.2a shows the STL training process, where each task is learnt independently. Fig.2b shows the multi-task learning process, where multiple correlated tasks are learnt simultaneously by sharing important information among the tasks.

Fig.2 Illustration of training process of STL and MTL methods. (a), In the STL method, each task is learnt independently; (b), In MTL models, multiple correlated tasks are learnt simultaneously by extracting and utilizing shared information across tasks.

In this paper, in order to improve the effectiveness of the proposed MTL model, we add the trace-norm regularization on the objective function and call the model TNR-MTL. In the following section, the formulation, op- timization and convergence of TNR-MTL method are in- troduced in detail.

3.1 Problem Formulation

where>0 is the trade-off parameter. We assume that the gradient of(·), denoted asÑ(·), is Lipschitz continuous with constant,.,

3.2 The Optimization Method of TNR-MTL

The extended gradient algorithm is a kind of gradient descent algorithm that can solve the non-smooth problem based on the subgradient algorithm (Ji and Ye, 2009). Be- cause the objective function of TNR-MTL contains the trace-norm term that is non-smooth, in order to find the local minimum solution of the learning problem, we use the extended gradient algorithm to optimize TNR-MTL. Therefore,can be iteratively learned using the following equation:

where

In this case, we can use singular value decomposition (SVD) to solve the minimization problem of Eq. (5) and iteratively optimize the TNR-MTL model.

3.3 Convergence Analysis

Theorem. Let {} be the sequence generated by using the extended gradient algorithm for Problem (3). Then for any≥1, we have

where>1and*=argminF() .

The proof of this theorem can be referred to that for Theorem 3.2 in Ji and Ye, (2009).

4 Numerical Experiments

In the following section, we report the used data sets, measure criterion and the obtained experimental results. As mentioned above, we selected the optimal model from TNR-MTL, which is called SD_TNR-MTL, for SSB estimation on new altimeter data. As a baseline, the SSB value in geophysical data records (GDR) was compared.

4.1 Measure Criteria

To evaluate the performance of SD_TNR-MTL for SSB estimation on the intersection data, we used the corrected variance () to measure the effect of SSB estimation. The corrected variance () is defined as

whereis the size of data, ∆stands for the SSH difference between the ascending and descending orbits, and ∆stands for the difference of SSB between the ascending and descending orbits.

4.2 Datasets

In this experiment, we used the data of three altimeters, including JASON-2, JASON-3 and HY-2. The JASON-2 includes 9 cycles of data and 3 cycles of intersection data. The JASON-3 includes 3 cycles of data, and the HY-2 in- cludes 2 cycles of intersection data,., cycle 70 and 71.

In details, the JASON-2 includes 3594131 data of 9 cy- cles and 4699 intersection data of 3 cycles. JASON-3 in- cludes 1350966 data of cycle 008, 010 and 015. HY-2 in- cludes 2997 intersection data. We used SWH and U as the inputs and SSB as the output to train the TNR-MTL mo- del for SSB estimation. Independent data were used to test the performance of SD_TNR-MTL.

According to the requirements of the GDR data (Li., 2013) on the original altimeter data, we added some error correction terms including instrument error, dry and wet tropospheric delay, ionospheric delay, atmospheric inverse pressure, high frequency oscillation, ocean tide, polar tide, solid earth tide and load tide, and moreover, we removed abnormal data which include SWH<0m or SWH>10m, U<0m or U<10ms−1and SSB>0m or SSB<−0.5m.

4.3 Training and Test of TNR-MTL

We used the JASON-2 and JASON-3 cycle data to train TNR-MTL and selected the optimal model, which is called SD_TNR-MTL, to estimate SSB on the JASON-2 cycle intersection data and the HY-2 70-71 cycle intersection data. In the experiment, we set=10−5. As shown in Fig.3, in TNR-MTL, two tasks were learnt simultaneously by extracting and utilizing appropriate shared information across the tasks. So that TNR-MTL model can be trained for two tasks at the same time, and can take the correlation between the two tasks into account during training, in contrast to the STL models that are trained for each task separately. Therefore, multi-task learning effectively enhances the generalization ability of TNR-MTL.

Fig.3 Schematic diagram of the TNR-MTL method for SSB estimation. JASON-2 and JASON-3 cycle data are input- ted to train the model simultaneously by sharing information between the combined tasks.

We trained TNR-MTL on the JASON-2 and JASON-3 cycle data, and tested it on the JASON-2 cycle intersection data. The probability density of difference value ∆=TNR-MTL−GDRis shown in Fig.4 and most of them are nearly zero. From the diagram of Fig.5, we can see that the estimated SSB by the SD_TNR-MTL model is linearly correlated with the SSB of GDR. Furthermore, the training time of the SD_TNR-MTL is only 37.78s and the test time is only 2.6s. Therefore, TNR-MTL is effective and efficient for the SSB estimation tasks.

Fig.4 The probability density of the difference values between the estimated SSB and the value in the GDR data.

Fig.5 Scatter plot of SSBSD_TNR-MTLvs. SSBGDR. Their relationship is accurately fitted by the linear function.

In order to compare the results of SD_TNR-MTL and GDR further, we calculate the correlation between SWH and SSB, and between U and SSB, recorded in the GDR and estimated by the SD_TNR-MTL model, respectively. The establishment of the model is based on the basic assumption that SSB is related to SWH and U. Therefore, the larger the fit coefficients between SSB and SWH or U are, the smaller the residual is, indicating that the model is more effective. The fitting results of SD_TNR-MTL and GDR with respect to SWH and U are shown in Fig.6.

Fig.6 Scatter plot of SSB vs. SWH and U, respectively. (a), The scatter plot of SSB of GDR vs. SWH; (b), The scatter plot of SSB of GDR vs. U; (c), The scatter plot of SSB of SD_TNR-MTL vs. SWH; (d), The scatter plot of SSB of SD_ TNR-MTL vs. U.

The fitting coefficients of SSB and SWH for the GDR data is 0.8899, while the fitting coefficients of SSB and SWH for SD_TNR-MTL is 0.9939. The fitting coefficients of SSB and U for the GDR data is 0.4325, while the fitting coefficients of SSB and U for SD_TNR-MTL is 0.4539. Hence, it is easy to see that the fitting results for SD_TNR-MTL are better than that for GDR.

4.4 Results on the Cycle Intersection Data

To better demonstrate the effectiveness of the SD_TNR-MTL model, we applied the SD_TNR-MTL to the JASON- 2 cycle intersection data and the HY-2 70-71 cycle intersection data. The experimental results are shown in Table 1. As we can see, for the JASON-2 cycle intersection data, the(noted1) values of SD_TNR-MTL and GDR are 76.64cm2and 77.24cm2respectively. For the HY-2 70-71 cycle intersection data, we can see that the(noted2) value of GDR is 145.27cm2, while that of SD_TNR-MTL is 143.97cm2. Obviously, thevalue of SD_TNR-MTL is smaller than that of GDR. It indicates that the SSB estimated by SD_TNR-MTL is more accurate than that of the GDR data.

Through all the experiments above, it can be seen that thevalue of SD_TNR-MTL is consistently better than that of GDR, which demonstrates the effectiveness of SD_TNR-MTL for SSB estimation. In addition, in these experiments, since our model has only three parameters, the training of the TNR-MTL is much faster. Therefore, TNR-MTL is not only effective but also efficient for SSB estimation tasks.

Table 1 Results on the JASON-2 cycle intersection data and the HY-2 70-71 cycle intersection data

Notes:1 represents the corrected variance of SD_TNR-MTL on the JASON-2 cycle intersection data;2 represents the result of SD_TNR-MTL on the HY-2 70-71 cycle intersection data.

5 Conclusions

In this paper, we introduced an effective and efficient multi-task learning method called TNR-MTL for SSB es- timation. For each individual task, only three parameters (corresponding to the two inputs and one bias, respec- tively) were involved in TNR-MTL. More importantly, the convergence of TNR-MLT is theoretically guaranteed. Compared with the commonly used single task learning models, TNR-MTL can use the correlations between the combined tasks. In this paper, we have trained TNR-MTL with the JASON-2 and JASON-3 cycle data simultaneously and selected the optimal model that is called SD_ TNR-MTL to estimate the SSB on the JASON-2 cycle intersection data and the HY-2 70-71 cycle intersection data. Particularly, when SSB is estimated on the JSAON- 2 cycle intersection data, the corrected variance () has been reduced by 0.60cm2compared to GDR, andhas been reduced by 1.30cm2compared to GDR for the HY-2 cycle intersection data. Therefore, TNR-MTL is proved to be very efficient for SSB estimation and generally delivers more accurate results than GDR.

Acknowledgements

This work was supported by the Major Project for New Generation of AI (No. 2018AAA0100400), the National Natural Science Foundation of China (No. 41706010), the Joint Fund of the Equipments Pre-Research and Ministry of Education of China (No. 6141A020337), and the Fundamental Research Funds for the Central Universities of China.

Barrick, D., 1972. Remote sensing of sea state by radar.–. Newport, USA, 186-192.

Bickel, S., Bogojeska, J., and Lengauer, T., 2008. Multi-tasklearning for HIV therapy screening.. Helsinki, Fin- land, 56-63.

Chelton, D. B., 1994. The sea state bias in altimeter estimates of sea level from collinear analysis of TOPEX data., 99 (C12): 24995-25008.

Chen, Y. R., Zhao, D. B., and Zhang, Q. C., 2018. Multi-task learning for dangerous object detection in autonomous driving., 432: 559-571.

Coleman, R., 2001. Satellite altimetry and earth sciences: A hand- book of techniques and applications., 82 (34): 376-376.

Dasgupta, S., Laplante, B., and Murray, S., 2009. Climate change and the future impacts of storm-surge disasters in developing countries.. Washington, D. C., USA, 1-28.

Elfouhaily, T., Thompson, D., and Chapron, B., 2000. Improved electromagnetic bias theory: Inclusion of hydrodynamic mo- dulations., 105 (C1): 1299-1310.

Gaspar, P., and Florens, J., 1998. Estimation of the sea state bias in radar altimeter mea-surements of sea level: Results from a new nonparametric method., 103 (C8): 15803-15814.

Gaspar, P., Labroue, S., and Ogor, F., 2002. Improving nonpa- rametric estimates of the sea state bias in radar altimeter measurements of sea level., 19 (10): 1690-1707.

Gaspar, P., Ogor, F., and Traon, P., 1994. Estimating the sea state bias of the TOPEX and POSEIDON altimeters from crossover differences., 99 (C12): 24981-24994.

Ghavidel, A., Schiavulli, D., and Camps, A., 2016. Numerical computation of the electromagnetic bias in GNSS-R altimetry., 54 (1): 489-498.

Ji, S., and Ye, J., 2009. An accelerated gradient method for trace norm minimization.. Montreal, Canada, 457-464.

Labroue, S., Gaspar, P., Dorandeu, J., and Zanife, O., 2004. Non- parametric estimates of the sea state bias for the Jason-1 radar altimeter., 27 (3-4): 453-481.

Li, S., Wang, Y., Miao, H., Zhou, X., and Zhang, J., 2013. A pa- rametric model of estimating sea state bias based on Jason-1 altimetry., 37 (2): 181-185.

Liu, M. X., Zhang, J., and Shen, D. G., 2019. Joint classification and regressiondeep multi-task multi-channel learning for Alzheimer’s disease diagnosis., 66: 1195-1206.

Miao, H., Wang, X., Wang, G., Zhang, G., and Zhang, J., 2015. Study on the improved sea state bias parametric estimation model., 45 (12): 119- 124.

Miao, H., Zhang, G., Wang, G., Guo, Y., Jing, Y., and Zhang, J., 2016. Study on nonparametric estimations model of the sea state bias for the HY-2 altimeter., 31 (6): 1031-1036.

Moore, J., Gladstone, R., Zwinger, T., and Wolovick, M., 2018. Geoengineer polar glaciers to slow sea-level rise., 555 (7696): 303-305.

Pong, T. K., Ji, S., and Ye, J., 2010. Trace norm regularization: Reformulations, algorithms, and multi-task learning., 20 (6): 3465-3489.

Srebro, N., and Shraibman, A., 2005. Rank, trace-norm and max- norm.. Bertinoro, Italy, 545-560.

Tran, N., Vandemark, D., Labroue, S., Feng, H., Chapron, B., Tolman, H., Lambin, J., and Picot, N., 2010. Sea state bias in altimeter sea level estimates determined by combining wave model and satellite data., 115 (C3): C03020.

Vandemark, D., Tran, N., Beckley, B., Chapron, B., and Gaspar, P., 2002. Direct estimation of sea state impacts on radar altimeter sea level measurements., 29 (24): 1-1-1-4.

Wan, J., 2015. Study on HY-2 altimeter system delay in-orbit absolute calibration using reconstructive transponder. PhD thesis. National Space Science Center, Chinese Academy of Sciences, Beijing.

Wang, J. A., Tian, J. F., and Li, S., 2018. A multi-task learning approach for improving product title compression with user search log data.. New Orleans, USA, 451-458.

Wunsch, C., and Stammer, D., 1998. Satellite altimetry, the marine geoid, and the oceanic general circulation., 26 (1): 219-253.

Yaplee, B., Shapiro, A., Hammond, D., Au, B., and Uliana, E., 1971. Nanosecond radar observations of the ocean surface from a stable platform., 9 (3): 170-174.

Zhong, G., Liu, B., Guo, Y., and Miao, H., 2018. Sea state bias estimation with least absolute shrinkage and selection operator (LASSO)., 17 (5): 1019-1025.

Zhou, J., Chen, J., and Ye, J., 2011. MALSAR: Multi-task learningstructural regularization. Version 1.1. Arizona State University, Phoenix, USA, 1-50.

Zhou, X., Miao, H., Wang, Y., Fan, C., Cui, Y., and Zhang, J., 2012. Study on the determination of crossovers by piecewise fitting of satellite ground track., 41 (6): 811-815.

. Tel: 0086-532-66781719

E-mail: gqzhong@ouc.edu.cn

June 21, 2019;

January 30, 2020;

March 30, 2020

(Edited by Chen Wenwen)