Few-Shot High-Dimensional Feature Selection with Lagrange Programming Neural Networks

Abstract

Few-shot learning and high-dimensional feature selection represent significant challenges in machine learning. In this work, we introduce LPNN-FS, a novel feature selection (FS) technique based on a Lagrange Programming Neural Network (Pk-LPNN) originally developed in the context of compressive sampling. LPNN-FS is a continuous-time recurrent neural network whose dynamics inherently converges to an optimal sparse solution of the LASSO, with its equilibrium point acting as a feature selector. Evaluated in combination with specific downstream classifiers, our model is tested on both synthetic and real-world benchmark datasets characterized by high dimensionality and a limited number of observations. The results demonstrate that LPNN-FS competes with or outperforms state-of-the-art methods such as LASSO, k-best, and sparse PCA in this few-shot, high-dimensional setting. Overall, this study opens new avenues for leveraging compressive sampling-based techniques in challenging feature selection problems.