Abstract

Spindle event detection is a key component in analyzing human sleep. However, detection of these oscillatory patterns by experts is time consuming and costly. Automated detection algorithms are cost efficient and reproducible but require robust datasets to be trained and validated. Using the MODA (Massive Online Data Annotation) platform, we used crowdsourcing to produce a large open-source dataset of high quality, human-scored sleep spindles (5342 spindles, from 180 subjects). We evaluated the performance of three subtype scorers: “experts, researchers and non-experts”, as well as 7 previously published spindle detection algorithms. Our findings show that only two algorithms had performance scores similar to human experts. Furthermore, the human scorers agreed on the average spindle characteristics (density, duration and amplitude), but there were significant age and sex differences (also observed in the set of detected spindles). This study demonstrates how the MODA platform can be used to generate a highly valid open source standardized dataset for researchers to train, validate and compare automated detectors of biological signals such as the EEG.

Details

Title
Massive online data annotation, crowdsourcing to generate high quality sleep spindle annotations from EEG data
Author
Lacourse Karine 1   VIAFID ORCID Logo  ; Yetton Ben 2 ; Mednick, Sara 2   VIAFID ORCID Logo  ; Warby Simon C 3   VIAFID ORCID Logo 

 Centre d’études avancées en médecine du sommeil, Montréal, Canada (GRID:grid.505609.f) 
 University of California, Department of Cognitive Science, Irvine, USA (GRID:grid.266093.8) (ISNI:0000 0001 0668 7243) 
 Centre d’études avancées en médecine du sommeil, Montréal, Canada (GRID:grid.505609.f); Université de Montréal, Department of Psychiatry, Montréal, Canada (GRID:grid.14848.31) (ISNI:0000 0001 2292 3357) 
Publication year
2020
Publication date
2020
Publisher
Nature Publishing Group
e-ISSN
20524463
Source type
Scholarly Journal
Language of publication
English
ProQuest document ID
2489906662
Copyright
© The Author(s) 2020. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.