Benchmarking Automatic Machine Learning Frameworks

Code

• github.com/EpistasisLab/tpot
  In papernone★ 10,049
• github.com/automl/auto-sklearn
  In papernone★ 8,075
• github.com/h2oai/h2o-3
  tf★ 7,516
• github.com/ClimbsRocks/auto_ml
  In papertf★ 1,654

Abstract

AutoML serves as the bridge between varying levels of expertise when designing machine learning systems and expedites the data science process. A wide range of techniques is taken to address this, however there does not exist an objective comparison of these techniques. We present a benchmark of current open source AutoML solutions using open source datasets. We test auto-sklearn, TPOT, auto_ml, and H2O's AutoML solution against a compiled set of regression and classification datasets sourced from OpenML and find that auto-sklearn performs the best across classification datasets and TPOT performs the best across regression datasets.

Tasks

Reproductions