{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T01:56:55Z","timestamp":1783994215243,"version":"3.55.0"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1011181","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T00:00:00Z","timestamp":1701043200000}}],"reference-count":37,"publisher":"Public Library of Science (PLoS)","issue":"11","license":[{"start":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T00:00:00Z","timestamp":1699833600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001729","name":"Stiftelsen f\u00f6r Strategisk Forskning","doi-asserted-by":"publisher","award":["SSF ARC19-0016"],"award-info":[{"award-number":["SSF ARC19-0016"]}],"id":[{"id":"10.13039\/501100001729","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001862","name":"the Swedish Research Council","doi-asserted-by":"crossref","award":["2022-06725"],"award-info":[{"award-number":["2022-06725"]}],"id":[{"id":"10.13039\/501100001862","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Reliable detection and classification of bacteria and other pathogens in the human body, animals, food, and water is crucial for improving and safeguarding public health. For instance, identifying the species and its antibiotic susceptibility is vital for effective bacterial infection treatment. Here we show that phase contrast time-lapse microscopy combined with deep learning is sufficient to classify four species of bacteria relevant to human health. The classification is performed on living bacteria and does not require fixation or staining, meaning that the bacterial species can be determined as the bacteria reproduce in a microfluidic device, enabling parallel determination of susceptibility to antibiotics. We assess the performance of convolutional neural networks and vision transformers, where the best model attained a class-average accuracy exceeding 98%. Our successful proof-of-principle results suggest that the methods should be challenged with data covering more species and clinically relevant isolates for future clinical use.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1011181","type":"journal-article","created":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T13:44:49Z","timestamp":1699883089000},"page":"e1011181","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":17,"title":["Label-free deep learning-based species classification of bacteria imaged by phase-contrast microscopy"],"prefix":"10.1371","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0426-3217","authenticated-orcid":true,"given":"Erik","family":"Hallstr\u00f6m","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vinodh","family":"Kandavalli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Petter","family":"Ranefall","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Johan","family":"Elf","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4139-7003","authenticated-orcid":true,"given":"Carolina","family":"W\u00e4hlby","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2023,11,13]]},"reference":[{"issue":"3","key":"pcbi.1011181.ref001","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1016\/j.tibtech.2018.08.005","article-title":"Deep learning with microfluidics for biotechnology","volume":"37","author":"J Riordon","year":"2019","journal-title":"Trends in biotechnology"},{"issue":"7","key":"pcbi.1011181.ref002","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1016\/j.tim.2021.01.006","article-title":"Deep learning for imaging and detection of microorganisms","volume":"29","author":"Y Zhang","year":"2021","journal-title":"Trends in Microbiology"},{"key":"pcbi.1011181.ref003","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"A Krizhevsky","year":"2012","journal-title":"Advances in neural information processing systems"},{"key":"pcbi.1011181.ref004","doi-asserted-by":"crossref","unstructured":"Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. Ieee; 2009. p. 248\u2013255.","DOI":"10.1109\/CVPR.2009.5206848"},{"issue":"7553","key":"pcbi.1011181.ref005","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"Y LeCun","year":"2015","journal-title":"nature"},{"key":"pcbi.1011181.ref006","unstructured":"Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980. 2014;."},{"key":"pcbi.1011181.ref007","unstructured":"Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. PMLR; 2015. p. 448\u2013456."},{"key":"pcbi.1011181.ref008","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"pcbi.1011181.ref009","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In: International Conference on Learning Representations; 2021. Available from: https:\/\/openreview.net\/forum?id=YicbFdNTTy."},{"key":"pcbi.1011181.ref010","article-title":"Attention is all you need","volume":"30","author":"A Vaswani","year":"2017","journal-title":"Advances in neural information processing systems"},{"key":"pcbi.1011181.ref011","doi-asserted-by":"crossref","unstructured":"Tran D, Wang H, Torresani L, Ray J, LeCun Y, Paluri M. A closer look at spatiotemporal convolutions for action recognition. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition; 2018. p. 6450\u20136459.","DOI":"10.1109\/CVPR.2018.00675"},{"key":"pcbi.1011181.ref012","doi-asserted-by":"crossref","unstructured":"Tran D, Wang H, Torresani L, Feiszli M. Video classification with channel-separated convolutional networks. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision; 2019. p. 5552\u20135561.","DOI":"10.1109\/ICCV.2019.00565"},{"key":"pcbi.1011181.ref013","doi-asserted-by":"crossref","unstructured":"Piergiovanni A, Kuo W, Angelova A. Rethinking video vits: Sparse video tubes for joint image and video learning. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition; 2023. p. 2214\u20132224.","DOI":"10.1109\/CVPR52729.2023.00220"},{"issue":"1","key":"pcbi.1011181.ref014","doi-asserted-by":"crossref","first-page":"6215","DOI":"10.1038\/s41467-022-33659-1","article-title":"Rapid antibiotic susceptibility testing and species identification for mixed samples","volume":"13","author":"V Kandavalli","year":"2022","journal-title":"Nature Communications"},{"key":"pcbi.1011181.ref015","first-page":"1","article-title":"A comprehensive review of image analysis methods for microorganism counting: from classical image processing to deep learning approaches","author":"J Zhang","year":"2022","journal-title":"Artificial Intelligence Review"},{"issue":"1","key":"pcbi.1011181.ref016","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1038\/s41377-020-00358-9","article-title":"Early detection and classification of live bacteria using time-lapse coherent imaging and deep learning","volume":"9","author":"H Wang","year":"2020","journal-title":"Light: Science & Applications"},{"key":"pcbi.1011181.ref017","doi-asserted-by":"crossref","unstructured":"Qiu Z, Yao T, Mei T. Learning spatio-temporal representation with pseudo-3d residual networks. In: proceedings of the IEEE International Conference on Computer Vision; 2017. p. 5533\u20135541.","DOI":"10.1109\/ICCV.2017.590"},{"key":"pcbi.1011181.ref018","doi-asserted-by":"crossref","unstructured":"Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 4700\u20134708.","DOI":"10.1109\/CVPR.2017.243"},{"issue":"9","key":"pcbi.1011181.ref019","doi-asserted-by":"crossref","first-page":"e0184554","DOI":"10.1371\/journal.pone.0184554","article-title":"Deep learning approach to bacterial colony classification","volume":"12","author":"B Zieli\u0144ski","year":"2017","journal-title":"PloS one"},{"issue":"23","key":"pcbi.1011181.ref020","doi-asserted-by":"crossref","first-page":"3005","DOI":"10.3390\/electronics10233005","article-title":"Small-Scale Depthwise Separable Convolutional Neural Networks for Bacteria Classification","volume":"10","author":"DT Mai","year":"2021","journal-title":"Electronics"},{"key":"pcbi.1011181.ref021","article-title":"Automated Interpretation of Blood Culture Gram Stains by Use of a Deep Convolutional Neural Network","volume":"56","author":"KP Smith","year":"2017","journal-title":"Journal of Clinical Microbiology"},{"issue":"12","key":"pcbi.1011181.ref022","doi-asserted-by":"crossref","first-page":"e1006628","DOI":"10.1371\/journal.pcbi.1006628","article-title":"Performance of convolutional neural networks for identification of bacteria in 3D microscopy datasets","volume":"14","author":"EA Hay","year":"2018","journal-title":"PLoS computational biology"},{"key":"pcbi.1011181.ref023","doi-asserted-by":"crossref","first-page":"e65151","DOI":"10.7554\/eLife.65151","article-title":"Misic, a general deep learning-based method for the high-throughput cell segmentation of complex bacterial communities","volume":"10","author":"S Panigrahi","year":"2021","journal-title":"Elife"},{"issue":"1","key":"pcbi.1011181.ref024","doi-asserted-by":"crossref","first-page":"6215","DOI":"10.1038\/s41467-022-33659-1","article-title":"Rapid antibiotic susceptibility testing and species identification for mixed samples","volume":"13","author":"V Kandavalli","year":"2022","journal-title":"Nature Communications"},{"issue":"34","key":"pcbi.1011181.ref025","doi-asserted-by":"crossref","first-page":"9170","DOI":"10.1073\/pnas.1708558114","article-title":"Antibiotic susceptibility testing in less than 30 min using direct single-cell imaging","volume":"114","author":"\u00d6 Baltekin","year":"2017","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"1","key":"pcbi.1011181.ref026","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/83.650848","article-title":"A pyramid approach to subpixel registration based on intensity","volume":"7","author":"P Thevenaz","year":"1998","journal-title":"IEEE transactions on image processing"},{"key":"pcbi.1011181.ref027","unstructured":"Hallstr\u00f6m E, Kandavalli V, Ranefall P, Elf J, W\u00e4hlby C. Phase contrast time-lapses of four bacteria species growing in microfluidic chips.; 2023. Available from: https:\/\/doi.org\/10.5281\/zenodo.7888071."},{"issue":"1","key":"pcbi.1011181.ref028","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-019-08987-4","article-title":"Unmasking Clever Hans predictors and assessing what machines really learn","volume":"10","author":"S Lapuschkin","year":"2019","journal-title":"Nature communications"},{"key":"pcbi.1011181.ref029","first-page":"15395","article-title":"Overinterpretation reveals image classification model pathologies","volume":"34","author":"B Carter","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"pcbi.1011181.ref030","unstructured":"Wightman R, Touvron H, Jegou H. ResNet strikes back: An improved training procedure in timm. In: NeurIPS 2021 Workshop on ImageNet: Past, Present, and Future; 2021. Available from: https:\/\/openreview.net\/forum?id=NG6MJnVl6M5."},{"key":"pcbi.1011181.ref031","unstructured":"Touvron H, Cord M, Douze M, Massa F, Sablayrolles A, J\u00e9gou H. Training data-efficient image transformers & distillation through attention. In: International conference on machine learning. PMLR; 2021. p. 10347\u201310357."},{"key":"pcbi.1011181.ref032","unstructured":"Mazzoli F. L\u00e1nczos interpolation explained; 2022. Available from: https:\/\/doi.org\/10.5281\/zenodo.7761401."},{"key":"pcbi.1011181.ref033","unstructured":"Kay W, Carreira J, Simonyan K, Zhang B, Hillier C, Vijayanarasimhan S, et al. The kinetics human action video dataset. arXiv preprint arXiv:170506950. 2017;."},{"key":"pcbi.1011181.ref034","unstructured":"Loshchilov I, Hutter F. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:160803983. 2016;."},{"key":"pcbi.1011181.ref035","article-title":"Pytorch: An imperative style, high-performance deep learning library","volume":"32","author":"A Paszke","year":"2019","journal-title":"Advances in neural information processing systems"},{"key":"pcbi.1011181.ref036","unstructured":"Wightman R, Raw N, Soare A, Arora A, Ha C, Reich C, et al. rwightman\/pytorch-image-models: v0.8.6dev0 Release; 2023. Available from: https:\/\/doi.org\/10.5281\/zenodo.7527889."},{"key":"pcbi.1011181.ref037","unstructured":"Garnier, Simon, Ross, Noam, Rudis, Robert, et al. viridis(Lite)\u2014Colorblind-Friendly Color Maps for R; 2023. Available from: https:\/\/sjmgarnier.github.io\/viridis\/."}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1011181","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T00:00:00Z","timestamp":1701043200000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011181","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T14:32:01Z","timestamp":1701095521000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011181"}},"subtitle":[],"editor":[{"given":"Pedro","family":"Mendes","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2023,11,13]]},"references-count":37,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2023,11,13]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1011181","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.05.14.540740","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,13]]}}}