一类基于L0/1软间隔损失函数的低秩支持张量机

doi:10.15960/j.cnki.issn.1007-6093.2021.03.010

摘要/Abstract

摘要： 支持向量机作为基于向量空间的一种传统的机器学习方法，不能直接处理张量类型的数据，否则不仅破坏数据的空间结构，还会造成维度灾难及小样本问题。作为支持向量机的一种高阶推广，用于处理张量数据分类的支持张量机已经引起众多学者的关注，并应用于遥感成像、视频分析、金融、故障诊断等多个领域。与支持向量机类似，已有的支持张量机模型中采用的损失函数多为L_0/1函数的代理函数。将直接使用L_0/1这一本原函数作为损失函数，并利用张量数据的低秩性，建立针对二分类问题的低秩支持张量机模型。针对这一非凸非连续的张量优化问题，设计交替方向乘子法进行求解，并通过对模拟数据和真实数据进行数值实验，验证模型与算法的有效性。

关键词: 支持张量机, L_0/1软间隔损失, 交替方向乘子法, Tucker秩

Abstract: As a traditional classification method, support vector machine (SVM) has limitations for high order tensorial data, since direct vectorization will lead to the loss of intrinsic spatial structures in tensors, and the small sample size problem as well. As a higher-order extension of SVM, support tensor machine (STM), which targets at tensorial data classification, has attracted more and more attention of many scholars, with wide applications in remote sensing imaging, video processing, finance, fault diagnosis, etc. Analogous to SVM, the involved loss functions in most of the existing STM models are surrogates of the L_0/1 function. In this paper, the original L_0/1 loss is employed, based on which, a low rank STM model is proposed for the binary classification problem, with consideration of the intrinsic low-rankness of tensorial data. The resulting nonconvex discontinuous tensor optimization problem is solved by an alternating direction method of multipliers. Numerical experiments are conducted on synthetic data and real data sets to demonstrate the effectiveness of the proposed approach.

Key words: support tensor machine, L_0/1 soft-margin loss, alternating direction method of multipliers, Tucker rank

中图分类号:

O224

王双月, 罗自炎. 一类基于L_0/1软间隔损失函数的低秩支持张量机[J]. 运筹学学报, 2021, 25(3): 160-172.

WANG Shuangyue, LUO Ziyan. Low rank support tensor machine based on L_0/1 soft-margin loss function[J]. Operations Research Transactions, 2021, 25(3): 160-172.

参考文献

[1] Turk M, Pentland A. Face recognition using eigenfaces[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1991:586-591.
[2] Plataniotis K, Venetsanopoulos A. Color Image Processing and Applications[M]. Berlin:SpringerVerlag, 2000.
[3] Green R, Guan L. Quantifying and recognizing human movement patterns from monocular video images-part II:Applications to biometrics[J]. Transactions on Circuits and Systems for Video Technology, 2004, 14(2):191-198.
[4] Sarkar S, Phillips P, Liu Z, et al. The humanID gait challenge problem:Data sets, performance, and analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(2):162-177.
[5] Renard N, Bourennane S. Dimensionality reduction based on tensor modeling for classification methods[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(4):1123-1131.
[6] Kim M, Jeon J, Kwak J, et al. Moving object segmentation in video sequences by user interaction and automatic object tracking[J]. Image and Vision Computing, 2001, 19(5):245-260.
[7] Yan S, Xu D, Yang Q, et al. Multilinear discriminant analysis for face recognition[J]. IEEE Transactions on Image Processing, 2007, 16(1):212-220.
[8] Lu H P, Plataniotis K N, Venetsanopoulos A N. MPCA:Multilinear principal component analysis of tensor objects[J]. IEEE Transactions on Neural Networks, 2008, 19(1):18-39.
[9] Tao D C, Li X L, Hu W M, et al. Supervised tensor learning[C]//The Fifth IEEE International Conference on Data Mining, 2005:450-457.
[10] Tao D C, Li X L, Wu X D, et al. Supervised tensor learning[J]. Knowledge and Information Systems, 2007, 13:1-42.
[11] Cai D, He X F, Wen J R, et al. Support tensor machines for text categorization[R]. Department of Computer Science Technical Report No.2714, University of Illinois at UrbanaChampaign (UIUCDCS-R-2006-2714), 2006.
[12] Kiers H. Towards a standardized notation and terminology in multiway analysis[J]. Journal of Chemometrics, 2000, 14(3):105-122.
[13] Kotsia I, Guo W W, Patras I. Higher rank support tensor machines for visual recognition[J]. Pattern Recognition, 2012, 45(12):4192-4203.
[14] 杨兵. 基于张量数据的机器学习方法研究与应用[D]. 北京:中国农业大学, 2014.
[15] Lian H. Learning rate for convex support tensor machines[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(8):3755-3760.
[16] Tucker L R. Some mathematical notes on three-mode factor analysis[J]. Psychometrika, 1966, 31:279-311.
[17] Oseledets I V. Tensor-train decomposition[J]. SIAM Journal on Scientific Computing, 2011, 33(5):2295-2317.
[18] Chen C, Batselier K, Yu W J, et al. Kernelized support tensor train machines[J]. arXiv:2001.00360, 2020.
[19] Kour K, Dolgov S, Stoll M, et al. Efficient structure-preserving support tensor train machine[J]. arXiv:2002.05079, 2020.
[20] He L F, Kong X N, Yu P S, et al. Dusk:A dual structure-preserving kernel for supervised tensor learning with applications to neuroimages[C]//Proceedings of the 2014 SIAM International Conference on Data Mining, 2014:127-135.
[21] Hao Z F, He L F, Chen B Q, et al. A linear support higher-order tensor machine for classification[J]. IEEE Transactions on Image Processing, 2013, 22(7):2911-2920.
[22] Xing Y H, Wang M, Yang S Y, et al. Pansharpening with multiscale geometric support tensor machine[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(5):2503-2517.
[23] Zhang L F, Zhang L P, Tao D C, et al. A multifeature tensor for remote-sensing target recognition[J]. IEEE Geoscience and Remote Sensing Letters, 2011, 8(2):374-378.
[24] Guo X, Huang X, Zhang L F, et al. Support tensor machines for classification of hyperspectral remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(6):3248-3264.
[25] Chen H, Ren Q L, Zhang Y. A hierarchical support tensor machine structure for target detection on high-resolution remote sensing images[C]//International Geoscience and Remote Sensing Symposium, 2017:594-597.
[26] Zhou B Y, Song B, Hassan M M, et al. Multilinear rank support tensor machine for crowd density estimation[J]. Engineering Applications of Artificial Intelligence, 2018, 72:382-392.
[27] Zhang Z, Chow T W S. Maximum margin multisurface support tensor machines with application to image classification and segmentation[J]. Expert Systems with Applications, 2012, 39:849-860.
[28] Chen Z Y, Fan Z P, Sun M H. A multi-kernel support tensor machine for classification with multitype multiway data and an application to cross-selling recommendations[J]. European Journal of Operational Research, 2016, 255:110-120.
[29] Calvi G G, Lucic V, Mandic D P. Support tensor machine for financial forecasting[C]//International Conference on Acoustics, Speech, and Signal Processing, 2019.
[30] He Z Y, Shao H D, Cheng J S, et al. Support tensor machine with dynamic penalty factors and its application to the fault diagnosis of rotating machinery with unbalanced data[J]. Mechanical Systems and Signal Processing, 2020, 141:106441.
[31] Hu C F, He S L, Wang Y X. A classification method to detect faults in a rotating machinery based on kernelled support tensor machine and multilinear principal component analysis[J]. Applied Intelligence, 2021, 51:2609-2621.
[32] Yang C, Jia M P. Hierarchical multiscale permutation entropy-based feature extraction and fuzzy support tensor machine with pinball loss for bearing fault identification[J]. Mechanical Systems and Signal Processing, 2021, 149:107182.
[33] Cortes C, Vapnik V. Support vector networks[J]. Machine Learning, 1995, 20:273-297.
[34] 王华军, 修乃华. 支持向量机损失函数分析[J]. 数学进展, 2021.
[35] Wang H J, Shao Y H, Zhou S L, et al. Support vector machine classifier via L_0/1 soft-margin loss[J]. arXiv:1912.07418, 2019.
[36] Li X S, Xu D, Zhou H, et al. Tucker tensor regression and neuroimaging analysis[J]. Statistics in Biosciences, 2013, 10:520-545.
[37] Kolda T G, Bader B W. Tensor decompositions and applications[J]. SIAM Review, 2009, 51(3):455-500.
[38] Qi L Q, Luo Z Y. Tensor Analysis:Spectral Theory and Special Tensors[M]. Society for Industrial & Applied Mathematics Press, 2017.
[39] Yu X T, Luo Z Y, Qi L Q, et al. SLRTA:A sparse and low-rank tensor-based approach to internet traffic anomaly detection[J]. Neurocomputing, 2021, 434:295-314.
[40] Vannieuwenhoven N, Vandebril R, Meerbergen K. A new truncation strategy for the higherorder singular value decomposition[J]. SIAM Journal on Scientific Computing, 2012, 34(2):1027-1052.
[41] Minster R, Saibaba A K, Kilmer M E. Randomized algorithms for low-rank tensor decompositions in the Tucker format[J]. SIAM Journal on Mathematics of Data Science, 2020, 2(1):189-215.
[42] Che M L, Wei Y M, Yan H. The computation of low multilinear rank approximations of tensors via power scheme and random projection[J]. SIAM Journal on Matrix Analysis and Applications, 2020, 41(2):605-636.
[43] Chang C C, Lin C J. LIBSVM:A library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3):1-27.
[44] Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images[R]. University of Toronto, 2009.