Position: More Rigorous Software Engineering Would Improve Reproducibility in Machine Learning Research

Code

• github.com/BonnBytes/position_we_need_more_tests_in_ml
  OfficialIn paperjax★ 3

Abstract

Experimental verification and falsification of scholarly work are part of the scientific method's core. To improve the Machine Learning (ML)-communities' ability to verify results from prior work, we argue for more robust software engineering. We estimate the adoption of common engineering best practices by examining repository links from all recently accepted International Conference on Machine Learning (ICML), International Conference on Learning Representations (ICLR) and Neural Information Processing Systems (NeurIPS) papers as well as ICML papers over time. Based on the results, we recommend how we, as a community, can improve reproducibility in ML-research.

Tasks

Reproductions