On Sensitivity of Metalearning: An Illustrative Study for Robust Regression

Abstrakt

Metalearning is becoming an increasingly important methodology for extracting knowledge from a data base of available training datasets to a new (independent) dataset. While the concept of metalearning is becoming popular in statistical learning and readily available also for the analysis of economic datasets, not much attention has been paid to its limitations and disadvantages. To the best of our knowledge, the current paper represents a first illustration of metalearning sensitivity to data contamination by noise or outliers. For this purpose, we use various linear regression estimators (including highly robust ones) over a set of 24 datasets with economic background and perform a metalearning study over them as well as over the same datasets after an artificial contamination. The results reveal the whole process to remain rather sensitive to data contamination and some of the standard classifiers turn out to yield unreliable results. Nevertheless, using a robust classification method does not bring a desirable improvement. Thus, we conclude that the task of robustification of the whole metalearning methodology is more complex and deserves a systematic future research.