中文

Operations Research Transactions >

2021 , Vol. 25 >Issue 3: 160 - 172

DOI: https://doi.org/10.15960/j.cnki.issn.1007-6093.2021.03.010

Low rank support tensor machine based on L0/1 soft-margin loss function

Expand
  • School of Science, Beijing Jiaotong University, Beijing 100044, China

Received date: 2021-03-15

  Online published: 2021-09-26

Fold

Abstract

As a traditional classification method, support vector machine (SVM) has limitations for high order tensorial data, since direct vectorization will lead to the loss of intrinsic spatial structures in tensors, and the small sample size problem as well. As a higher-order extension of SVM, support tensor machine (STM), which targets at tensorial data classification, has attracted more and more attention of many scholars, with wide applications in remote sensing imaging, video processing, finance, fault diagnosis, etc. Analogous to SVM, the involved loss functions in most of the existing STM models are surrogates of the L0/1 function. In this paper, the original L0/1 loss is employed, based on which, a low rank STM model is proposed for the binary classification problem, with consideration of the intrinsic low-rankness of tensorial data. The resulting nonconvex discontinuous tensor optimization problem is solved by an alternating direction method of multipliers. Numerical experiments are conducted on synthetic data and real data sets to demonstrate the effectiveness of the proposed approach.

Key words： support tensor machine; L0/1 soft-margin loss; alternating direction method of multipliers; Tucker rank

Cite this article

WANG Shuangyue, LUO Ziyan . Low rank support tensor machine based on L0/1 soft-margin loss function[J]. Operations Research Transactions, 2021 , 25(3) : 160 -172 . DOI: 10.15960/j.cnki.issn.1007-6093.2021.03.010

References

[1] Turk M, Pentland A. Face recognition using eigenfaces[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1991:586-591.
[2] Plataniotis K, Venetsanopoulos A. Color Image Processing and Applications[M]. Berlin:SpringerVerlag, 2000.
[3] Green R, Guan L. Quantifying and recognizing human movement patterns from monocular video images-part II:Applications to biometrics[J]. Transactions on Circuits and Systems for Video Technology, 2004, 14(2):191-198.
[4] Sarkar S, Phillips P, Liu Z, et al. The humanID gait challenge problem:Data sets, performance, and analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(2):162-177.
[5] Renard N, Bourennane S. Dimensionality reduction based on tensor modeling for classification methods[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(4):1123-1131.
[6] Kim M, Jeon J, Kwak J, et al. Moving object segmentation in video sequences by user interaction and automatic object tracking[J]. Image and Vision Computing, 2001, 19(5):245-260.
[7] Yan S, Xu D, Yang Q, et al. Multilinear discriminant analysis for face recognition[J]. IEEE Transactions on Image Processing, 2007, 16(1):212-220.
[8] Lu H P, Plataniotis K N, Venetsanopoulos A N. MPCA:Multilinear principal component analysis of tensor objects[J]. IEEE Transactions on Neural Networks, 2008, 19(1):18-39.
[9] Tao D C, Li X L, Hu W M, et al. Supervised tensor learning[C]//The Fifth IEEE International Conference on Data Mining, 2005:450-457.
[10] Tao D C, Li X L, Wu X D, et al. Supervised tensor learning[J]. Knowledge and Information Systems, 2007, 13:1-42.
[11] Cai D, He X F, Wen J R, et al. Support tensor machines for text categorization[R]. Department of Computer Science Technical Report No.2714, University of Illinois at UrbanaChampaign (UIUCDCS-R-2006-2714), 2006.
[12] Kiers H. Towards a standardized notation and terminology in multiway analysis[J]. Journal of Chemometrics, 2000, 14(3):105-122.
[13] Kotsia I, Guo W W, Patras I. Higher rank support tensor machines for visual recognition[J]. Pattern Recognition, 2012, 45(12):4192-4203.
[14] 杨兵. 基于张量数据的机器学习方法研究与应用[D]. 北京:中国农业大学, 2014.
[15] Lian H. Learning rate for convex support tensor machines[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(8):3755-3760.
[16] Tucker L R. Some mathematical notes on three-mode factor analysis[J]. Psychometrika, 1966, 31:279-311.
[17] Oseledets I V. Tensor-train decomposition[J]. SIAM Journal on Scientific Computing, 2011, 33(5):2295-2317.
[18] Chen C, Batselier K, Yu W J, et al. Kernelized support tensor train machines[J]. arXiv:2001.00360, 2020.
[19] Kour K, Dolgov S, Stoll M, et al. Efficient structure-preserving support tensor train machine[J]. arXiv:2002.05079, 2020.
[20] He L F, Kong X N, Yu P S, et al. Dusk:A dual structure-preserving kernel for supervised tensor learning with applications to neuroimages[C]//Proceedings of the 2014 SIAM International Conference on Data Mining, 2014:127-135.
[21] Hao Z F, He L F, Chen B Q, et al. A linear support higher-order tensor machine for classification[J]. IEEE Transactions on Image Processing, 2013, 22(7):2911-2920.
[22] Xing Y H, Wang M, Yang S Y, et al. Pansharpening with multiscale geometric support tensor machine[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(5):2503-2517.
[23] Zhang L F, Zhang L P, Tao D C, et al. A multifeature tensor for remote-sensing target recognition[J]. IEEE Geoscience and Remote Sensing Letters, 2011, 8(2):374-378.
[24] Guo X, Huang X, Zhang L F, et al. Support tensor machines for classification of hyperspectral remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(6):3248-3264.
[25] Chen H, Ren Q L, Zhang Y. A hierarchical support tensor machine structure for target detection on high-resolution remote sensing images[C]//International Geoscience and Remote Sensing Symposium, 2017:594-597.
[26] Zhou B Y, Song B, Hassan M M, et al. Multilinear rank support tensor machine for crowd density estimation[J]. Engineering Applications of Artificial Intelligence, 2018, 72:382-392.
[27] Zhang Z, Chow T W S. Maximum margin multisurface support tensor machines with application to image classification and segmentation[J]. Expert Systems with Applications, 2012, 39:849-860.
[28] Chen Z Y, Fan Z P, Sun M H. A multi-kernel support tensor machine for classification with multitype multiway data and an application to cross-selling recommendations[J]. European Journal of Operational Research, 2016, 255:110-120.
[29] Calvi G G, Lucic V, Mandic D P. Support tensor machine for financial forecasting[C]//International Conference on Acoustics, Speech, and Signal Processing, 2019.
[30] He Z Y, Shao H D, Cheng J S, et al. Support tensor machine with dynamic penalty factors and its application to the fault diagnosis of rotating machinery with unbalanced data[J]. Mechanical Systems and Signal Processing, 2020, 141:106441.
[31] Hu C F, He S L, Wang Y X. A classification method to detect faults in a rotating machinery based on kernelled support tensor machine and multilinear principal component analysis[J]. Applied Intelligence, 2021, 51:2609-2621.
[32] Yang C, Jia M P. Hierarchical multiscale permutation entropy-based feature extraction and fuzzy support tensor machine with pinball loss for bearing fault identification[J]. Mechanical Systems and Signal Processing, 2021, 149:107182.
[33] Cortes C, Vapnik V. Support vector networks[J]. Machine Learning, 1995, 20:273-297.
[34] 王华军, 修乃华. 支持向量机损失函数分析[J]. 数学进展, 2021.
[35] Wang H J, Shao Y H, Zhou S L, et al. Support vector machine classifier via L0/1 soft-margin loss[J]. arXiv:1912.07418, 2019.
[36] Li X S, Xu D, Zhou H, et al. Tucker tensor regression and neuroimaging analysis[J]. Statistics in Biosciences, 2013, 10:520-545.
[37] Kolda T G, Bader B W. Tensor decompositions and applications[J]. SIAM Review, 2009, 51(3):455-500.
[38] Qi L Q, Luo Z Y. Tensor Analysis:Spectral Theory and Special Tensors[M]. Society for Industrial & Applied Mathematics Press, 2017.
[39] Yu X T, Luo Z Y, Qi L Q, et al. SLRTA:A sparse and low-rank tensor-based approach to internet traffic anomaly detection[J]. Neurocomputing, 2021, 434:295-314.
[40] Vannieuwenhoven N, Vandebril R, Meerbergen K. A new truncation strategy for the higherorder singular value decomposition[J]. SIAM Journal on Scientific Computing, 2012, 34(2):1027-1052.
[41] Minster R, Saibaba A K, Kilmer M E. Randomized algorithms for low-rank tensor decompositions in the Tucker format[J]. SIAM Journal on Mathematics of Data Science, 2020, 2(1):189-215.
[42] Che M L, Wei Y M, Yan H. The computation of low multilinear rank approximations of tensors via power scheme and random projection[J]. SIAM Journal on Matrix Analysis and Applications, 2020, 41(2):605-636.
[43] Chang C C, Lin C J. LIBSVM:A library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3):1-27.
[44] Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images[R]. University of Toronto, 2009.
Options
Outlines

/

〈 〉