As a feature extraction method, nonnegative tensor factorization has been widely used in image processing and pattern recognition for its advantages of preserving the internal structural features of data and strong interpretability. However, there are two problems in this method: one is that there is unnecessary correlation between the decomposed base images, which leads to more redundant information and takes up a lot of memory; the other is that the coding is not sparse enough, which leads to the expression of the image is not concise enough. These problems will greatly affect the accuracy of face recognition. In order to improve the accuracy of face recognition, a face recognition algorithm based on orthogonal and sparse constrained nonnegative tensor factorization is proposed. Firstly, orthogonal and sparse constraints are added to the traditional nonnegative tensor factorization to reduce the correlation between the base images and obtain sparse coding. Secondly, the original face image and the decomposed base image are used to calculate the low dimensional feature representation of the face. Finally, cosine similarity is used to measure the similarity between low-dimensional features and judge whether two face images represent the same person. Through experiments in AR database and ORL database, it is found that the improved algorithm can achieve better recognition effect.