Validation of ‘Somnivore’, a machine learning algorithm for automated scoring and analysis of polysomnography data
journal contributionposted on 30.03.2022, 05:17 authored by Giancarlo Allocca, Sherie Ma, Davide Martelli, Matteo Cerri, Flavia Del Vecchio, Stefano Bastianini, Giovanna Zoccoli, Roberto Amici, Stephen R Morairty, Anne AulsebrookAnne Aulsebrook, Shaun Blackburn, John LeskuJohn Lesku, Niels C Rattenborg, Alexei L Vyssotski, Emma Wams, Kate Porcheret, Katharina Wulff, Russell Foster, Julia KM Chan, Christian L Nicholas, Dean R Freestone, Leigh A Johnston, Andrew L Gundlach
Manual scoring of polysomnography data is labor-intensive and time-consuming, and most existing software does not account for subjective differences and user variability. Therefore, we evaluated a supervised machine learning algorithm, SomnivoreTM, for automated wake–sleep stage classification. We designed an algorithm that extracts features from various input channels, following a brief session of manual scoring, and provides automated wake-sleep stage classification for each recording. For algorithm validation, polysomnography data was obtained from independent laboratories, and include normal, cognitively-impaired, and alcohol-treated human subjects (total n = 52), narcoleptic mice and drug-treated rats (total n = 56), and pigeons (n = 5). Training and testing sets for validation were previously scored manually by 1–2 trained sleep technologists from each laboratory. F-measure was used to assess precision and sensitivity for statistical analysis of classifier output and human scorer agreement. The algorithm gave high concordance with manual visual scoring across all human data (wake 0.91 ± 0.01; N1 0.57 ± 0.01; N2 0.81 ± 0.01; N3 0.86 ± 0.01; REM 0.87 ± 0.01), which was comparable to manual inter-scorer agreement on all stages. Similarly, high concordance was observed across all rodent (wake 0.95 ± 0.01; NREM 0.94 ± 0.01; REM 0.91 ± 0.01) and pigeon (wake 0.96 ± 0.006; NREM 0.97 ± 0.01; REM 0.86 ± 0.02) data. Effects of classifier learning from single signal inputs, simple stage reclassification, automated removal of transition epochs, and training set size were also examined. In summary, we have developed a polysomnography analysis program for automated sleep-stage classification of data from diverse species. Somnivore enables flexible, accurate, and high-throughput analysis of experimental and clinical sleep studies.
This research was supported by an Australian International Postgraduate Research Scholarship, the Ministero dell' Universita e della Ricerca Scientifica (MIUR) Project Grant 2008FY7K9S, Australian Research Council Grant DE140101075, Max Planck Society, University of Zurich, Wellcome Trust Strategic Award, National Institute for Health Research (NIHR) Oxford Biomedical Research Centre grants A90305 and A92181, National Health and Medical Research Council (NHMRC) Australia Project Grant APP1012195, the Australasian Sleep Association, an Australian Postgraduate Award, and the Australia and New Zealand Banking Group Limited (ANZ) Trustees Foundation. The funding institutes played no role in the design and conduct of the study; no role in the collection, management, analysis, or interpretation of data; and no role in the preparation, review, or approval of the manuscript.
JournalFrontiers in Neuroscience
PublisherFrontiers Media SA
Rights Statement© 2019 Allocca, Ma, Martelli, Cerri, Del Vecchio, Bastianini, Zoccoli, Amici, Morairty, Aulsebrook, Blackburn, Lesku, Rattenborg, Vyssotski, Wams, Porcheret, Wulﬀ, Foster, Chan, Nicholas, Freestone, Johnston and Gundlach. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Science & TechnologyLife Sciences & BiomedicineNeurosciencesNeurosciences & Neurologymachine learning algorithmspolysomnographysignal processing algorithmssleep stage classificationwake-sleep stage scoringSLEEP-DEPRIVATIONAMERICAN ACADEMYPOWER SPECTRAEEGCLASSIFICATIONRELIABILITYPERFORMANCEDURATIONIMPAIRMENTRECORDINGSwake–sleep stage scoring