Machine learning competitions as a tool for reproducible research

Datum

14.03.2018

Zeit

16:30 - 17:45

Sprecher

Heikki Huttunen

Zugehörigkeit

Tampere University of Technology, Finland

Sprache

en

Hauptthema

Physik

Andere Themen

Physik

Beschreibung

Machine learning and deep learning have revolutionized machine learning research in this decade. The interest has exploded and the amount of research papers and researchers has grown almost exponentially---some top conferences are even sold out in a matter of weeks. Growing interest in research has also raised concerns on how to validate the results and make them comparable among scientists. The obvious approach is to generate public benchmark sets of some important problems available to all researchers. However, even large collections of data are still prone to overfitting to these datasets. For example, many well performing algorithms have a random component, and it may be tempting to select the random seed in a favorable manner. To address these concerns, many conferences are organizing machine learning competitions as part of their program. The competitions are usually organized on platforms specifically designed for this task, including kaggle.com or crowdai.org. The benefit of a competition is that the results are always comparable between the teams. Most teams also publish all their code, which makes the results directly reproducible and would immediately reveal any fraud in the results. We will discuss the anatomy of a machine learning competition with a few examples, also highlighting some of the current trends in machine learning today: - ICANN 2011 Mind Reading competition (speaker on the 1st place team): http://www.cis.hut.fi/icann2011/mindreading.php - DecMeg 2014 competition (speaker on the 2nd place team): https://www.kaggle.com/c/decoding-the-human-brain - DREAM6---FlowCAP2 Molecular Classification of Acute Myeloid Leukemia Challenge (speaker was a "best performer") - TUT acoustic scenes competition; a student competition organized by the speaker: https://www.kaggle.com/c/acoustic-scene-2018 At the end of the talk, we will also discuss some recent research at TUT in the domain of artificial intelligence for working machines.

Letztmalig verändert: 14.03.2018, 08:56:59