A brief review of the ChaLearn AutoML challenge: any-time any-dataset learning without human intervention

2016

Abstract

The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully automatic, black-box learning machines for feature-based classification and regression problems. The test bed was composed of 30 data sets from a wide variety of application domains and ranged across different types of complexity. Over six rounds, participants succeeded in delivering AutoML software capable of being trained and tested without human intervention. Although improvements can still be made to close the gap between human-tweaked and AutoML models, this competition contributes to the development of fully automated environments by challenging practitioners to solve problems under specific constraints and sharing their approaches; the platform will remain available for post-challenge submissions at http://codalab. org/AutoML.

Authors

Isabelle Guyon
Imad Chaabane
Sergio Escalera
Michele Sebag
Damir Jajetic
Hugo Jair Escalante
Núria Macià
Bisakha Ray
James Robert Lloyd
Alexander Statnikov
Isabelle Guyon
Sébastien Treguer
Evelyne Viegas