Enhancing Nonlinear Feature Reduction through Logistic Regression-Guided Hybrid PCA

Abstract

The usage of Principal Component Analysis (PCA) to reduce the number of dimensions is commonly because it is easy to use and works well. But in the real world, a lot of datasets have parts with high variance that don't always give useful information for classification. This reserach solves this problem by suggesting a Hybrid PCA Guided by Logistic Regression (PCA–LR), which chooses components based on how well they work for supervised classification instead of just variance. The proposed method tested on four datasets with different structures: Mental Health, Breast Cancer, Fetal Health, and Pistachio. The result shown PCA–LR achieved better accuracy performance than basic PCA in these datasets. Proposed method also works better than Kernel PCA (KPCA) and Sparse PCA (SPCA). The Mental Health dataset shows the biggest improvement, with PCA–LR getting an accuracy of 98.52%, which is 25.71% better than basic PCA. The method gets 97.37% on Breast Cancer and the same best performance as KPCA on Fetal Health and Pistachio. These results suggest that adding lightweight supervised guidance to PCA can make it much more useful on datasets that aren't strictly linear.