L1-norm based PCA for unsupervised classification

Principal component analysis (PCA) is a widespread technique for the analysis of multivariate data, which finds applications in the fields of machine learning and artificial intelligence, to name a few, and is commonly used for data compression and denoising [1]. Standard PCA seeks to calculate the subspace that minimizes the Euclidean distance (L2-norm) of the data points to it. Unfortunately, PCA is extremely sensitive to the presence of large outliers in the data. Recently, the L1-norm has been proposed as an alternative criterion to classical L2-norm in PCA, drawing considerable research interest on account of its increased robustness to outliers [2], [3].

The proposed contribution shows that, when combined with a whitening preprocessing step, L1-norm based PCA is endowed with discriminative power and can perform data classification in an unsupervised manner, i.e., sparing the need for labelled data. By minimizing the L1-norm in the feature space, the technique mimics the action of common spatial patterns (CSP), a supervised feature extraction method used in brain computer interfaces. This result is of theoretical interest and opens new interesting research perspectives for L1-PCA. Furthermore, it enables us to perform classification using algorithms for optimizing the L1-norm, which inherit the improved robustness to outliers of the L1-norm criterion. Several numerical experiments will confirm the theoretical findings.

REFERENCES

[1] I. T. Jolliffe. Principal component analysis. Springer, New York, NY, 2002.
[2] N. Kwak. Principal component analysis based on L1-norm maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(9):1672–1680, Sep 2008.
[3] Panos P. Markopoulos, George N. Karystinos, and Dimitris A. Pados. Optimal algorithms for L1-subspace signal processing. IEEE Transactions on Signal Processing, 62(19):5046–5058, Oct 2014.