{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,7]],"date-time":"2025-06-07T05:26:56Z","timestamp":1749274016766,"version":"3.37.3"},"reference-count":57,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,11,22]],"date-time":"2022-11-22T00:00:00Z","timestamp":1669075200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,11,22]],"date-time":"2022-11-22T00:00:00Z","timestamp":1669075200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R33 AR073552"],"award-info":[{"award-number":["R33 AR073552"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Radiologists today play a central role in making diagnostic decisions and labeling images for training and benchmarking artificial intelligence (AI) algorithms. A key concern is low inter-reader reliability (IRR) seen between experts when interpreting challenging cases. While team-based decisions are known to outperform individual decisions, inter-personal biases often creep up in group interactions which limit nondominant participants from expressing true opinions. To overcome the dual problems of low consensus and interpersonal bias, we explored a solution modeled on bee swarms. Two separate cohorts, three board-certified radiologists, (cohort 1), and five radiology residents (cohort 2) collaborated on a digital swarm platform in real time and in a blinded fashion, grading meniscal lesions on knee MR exams. These consensus votes were benchmarked against clinical (arthroscopy) and radiological (senior-most radiologist) standards of reference using Cohen\u2019s kappa. The IRR of the consensus votes was then compared to the IRR of the majority and most confident votes of the two cohorts. IRR was also calculated for predictions from a meniscal lesion detecting AI algorithm. The attending cohort saw an improvement of 23% in IRR of swarm votes (<jats:italic>k<\/jats:italic>= 0.34) over majority vote (<jats:italic>k<\/jats:italic>= 0.11). Similar improvement of 23% in IRR (<jats:italic>k<\/jats:italic>= 0.25) in 3-resident swarm votes over majority vote (<jats:italic>k<\/jats:italic>= 0.02) was observed. The 5-resident swarm had an even higher improvement of 30% in IRR (<jats:italic>k<\/jats:italic>= 0.37) over majority vote (<jats:italic>k<\/jats:italic>= 0.07). The swarm consensus votes outperformed individual and majority vote decision in both the radiologists and resident cohorts. The attending and resident swarms also outperformed predictions from a state-of-the-art AI algorithm.<\/jats:p>","DOI":"10.1007\/s10278-022-00662-3","type":"journal-article","created":{"date-parts":[[2022,11,23]],"date-time":"2022-11-23T02:29:41Z","timestamp":1669170581000},"page":"401-413","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Utilizing a Digital Swarm Intelligence Platform to Improve Consensus Among Radiologists and Exploring Its Applications"],"prefix":"10.1007","volume":"36","author":[{"given":"Rutwik","family":"Shah","sequence":"first","affiliation":[]},{"given":"Bruno","family":"Astuto Arouche Nunes","sequence":"additional","affiliation":[]},{"given":"Tyler","family":"Gleason","sequence":"additional","affiliation":[]},{"given":"Will","family":"Fletcher","sequence":"additional","affiliation":[]},{"given":"Justin","family":"Banaga","sequence":"additional","affiliation":[]},{"given":"Kevin","family":"Sweetwood","sequence":"additional","affiliation":[]},{"given":"Allen","family":"Ye","sequence":"additional","affiliation":[]},{"given":"Rina","family":"Patel","sequence":"additional","affiliation":[]},{"given":"Kevin","family":"McGill","sequence":"additional","affiliation":[]},{"given":"Thomas","family":"Link","sequence":"additional","affiliation":[]},{"given":"Jason","family":"Crane","sequence":"additional","affiliation":[]},{"given":"Valentina","family":"Pedoia","sequence":"additional","affiliation":[]},{"given":"Sharmila","family":"Majumdar","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,11,22]]},"reference":[{"key":"662_CR1","doi-asserted-by":"publisher","first-page":"979","DOI":"10.2105\/AJPH.74.9.979","volume":"74","author":"A Fink","year":"1984","unstructured":"Fink, A., Kosecoff, J., Chassin, M. & Brook, R. H. Consensus methods: characteristics and guidelines for use. American journal of public health 74, 979-983 (1984).","journal-title":"American journal of public health"},{"key":"662_CR2","unstructured":"Medicine, I. o., National Academies of Sciences, E. & Medicine. Improving Diagnosis in Health Care. (The National Academies Press, 2015)."},{"key":"662_CR3","doi-asserted-by":"publisher","first-page":"1694","DOI":"10.1002\/jmri.26555","volume":"49","author":"CP Smith","year":"2019","unstructured":"Smith, C. P. et al. Intra- and interreader reproducibility of PI-RADSv2: a multireader study. Journal of magnetic resonance imaging : JMRI 49, 1694-1703, https:\/\/doi.org\/10.1002\/jmri.26555 (2019).","journal-title":"Journal of magnetic resonance imaging : JMRI"},{"key":"662_CR4","doi-asserted-by":"publisher","first-page":"551","DOI":"10.3233\/bmr-150495","volume":"30","author":"CWJ van Tilburg","year":"2017","unstructured":"van Tilburg, C. W. J., Groeneweg, J. G., Stronks, D. L. & Huygen, F. Inter-rater reliability of diagnostic criteria for sacroiliac joint-, disc- and facet joint pain. Journal of back and musculoskeletal rehabilitation 30, 551-557, https:\/\/doi.org\/10.3233\/bmr-150495 (2017).","journal-title":"Journal of back and musculoskeletal rehabilitation"},{"key":"662_CR5","doi-asserted-by":"publisher","first-page":"209","DOI":"10.2214\/AJR.13.10952","volume":"203","author":"A Melsaether","year":"2014","unstructured":"Melsaether, A. et al. Inter- and intrareader agreement for categorization of background parenchymal enhancement at baseline and after training. American Journal of Roentgenology 203, 209-215, https:\/\/doi.org\/10.2214\/AJR.13.10952 (2014).","journal-title":"American Journal of Roentgenology"},{"key":"662_CR6","doi-asserted-by":"publisher","DOI":"10.1002\/jmri.27164","author":"R Tibrewala","year":"2020","unstructured":"Tibrewala, R. et al. Computer-aided detection AI reduces interreader variability in grading hip abnormalities with MRI. Journal of magnetic resonance imaging : JMRI, https:\/\/doi.org\/10.1002\/jmri.27164 (2020).","journal-title":"Journal of magnetic resonance imaging : JMRI"},{"key":"662_CR7","doi-asserted-by":"publisher","first-page":"1937","DOI":"10.1177\/0363546504264586","volume":"32","author":"WR Dunn","year":"2004","unstructured":"Dunn, W. R. et al. Multirater agreement of arthroscopic meniscal lesions. The American journal of sports medicine 32, 1937-1940, https:\/\/doi.org\/10.1177\/0363546504264586 (2004).","journal-title":"The American journal of sports medicine"},{"key":"662_CR8","doi-asserted-by":"publisher","first-page":"1668","DOI":"10.1148\/rg.2015150023","volume":"35","author":"MA Bruno","year":"2015","unstructured":"Bruno, M. A., Walker, E. A. & Abujudeh, H. H. Understanding and confronting our mistakes: the epidemiology of error in radiology and strategies for error reduction. RadioGraphics 35, 1668-1676, https:\/\/doi.org\/10.1148\/rg.2015150023 (2015).","journal-title":"RadioGraphics"},{"key":"662_CR9","doi-asserted-by":"publisher","first-page":"318","DOI":"10.1148\/radiol.2018171820","volume":"288","author":"G Choy","year":"2018","unstructured":"Choy, G. et al. Current applications and future impact of machine learning in radiology. Radiology 288, 318-328, https:\/\/doi.org\/10.1148\/radiol.2018171820 (2018).","journal-title":"Radiology"},{"key":"662_CR10","doi-asserted-by":"publisher","unstructured":"Demirer, M. et al. A User interface for optimizing radiologist engagement in image data curation for artificial intelligence. Radiology: Artificial Intelligence 1, e180095, https:\/\/doi.org\/10.1148\/ryai.2019180095 (2019).","DOI":"10.1148\/ryai.2019180095"},{"key":"662_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2020.101759","volume":"65","author":"D Karimi","year":"2020","unstructured":"Karimi, D., Dou, H., Warfield, S. K. & Gholipour, A. Deep learning with noisy labels: exploring techniques and remedies in medical image analysis. Medical Image Analysis 65, 101759, https:\/\/doi.org\/10.1016\/j.media.2020.101759 (2020).","journal-title":"Medical Image Analysis"},{"key":"662_CR12","doi-asserted-by":"publisher","first-page":"1313","DOI":"10.1109\/TMI.2016.2528120","volume":"35","author":"S Albarqouni","year":"2016","unstructured":"Albarqouni, S. et al. Aggnet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE transactions on medical imaging 35, 1313-1321 (2016).","journal-title":"IEEE transactions on medical imaging"},{"key":"662_CR13","doi-asserted-by":"crossref","unstructured":"Northcutt, C. G., Jiang, L. & Chuang, I. L. Confident learning: estimating uncertainty in dataset labels. Journal of Artificial Intelligence Research (2021).","DOI":"10.1613\/jair.1.12125"},{"key":"662_CR14","doi-asserted-by":"crossref","unstructured":"Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M.\u00a0\"Everyone wants to do the model work, not the data work\": data cascades in high-stakes AI. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-15) (2021, May).","DOI":"10.1145\/3411764.3445518"},{"key":"662_CR15","unstructured":"Northcutt, C. G., Wu, T. & Chuang, I. L. Learning with confident examples: rank pruning for robust classification with noisy labels. arXiv preprint http:\/\/arxiv.org\/abs\/1705.01936 (2017)."},{"key":"662_CR16","doi-asserted-by":"crossref","unstructured":"Lee, K.-H., He, X., Zhang, L. & Yang, L. Cleannet: Transfer learning for scalable image classifier training with label noise. In Proceedings of the IEEE conference on computer vision and pattern recognition\u00a05447\u20135456 (2018).","DOI":"10.1109\/CVPR.2018.00571"},{"key":"662_CR17","unstructured":"Thulasidasan, S., Bhattacharya, T., Bilmes, J., Chennupati, G. & Mohd-Yusof, J. Combating label noise in deep learning using abstention. arXiv preprint http:\/\/arxiv.org\/abs\/1905.10964 (2019)."},{"key":"662_CR18","doi-asserted-by":"crossref","unstructured":"Tanno, R., Saeedi, A., Sankaranarayanan, S., Alexander, D. C. & Silberman, N. Learning from noisy labels by regularized estimation of annotator confusion. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition\u00a011244-11253 (2019).","DOI":"10.1109\/CVPR.2019.01150"},{"key":"662_CR19","doi-asserted-by":"crossref","unstructured":"Veit, A., Nickel, M., Belongie, S., & van der Maaten, L. Separating self-expression and visual content in hashtag supervision. In Proceedings of the IEEE conference on computer vision and pattern recognition\u00a05919-5927 (2018).","DOI":"10.1109\/CVPR.2018.00620"},{"key":"662_CR20","unstructured":"Shen, Y. & Sanghavi, S. Learning with Bad Training Data via Iterative Trimmed Loss Minimization. Proceedings of the 36th International Conference on Machine Learning, in Proceeding of\u00a0Machine Learning Research\u00a097,\u00a05739-5748 (2019)."},{"key":"662_CR21","unstructured":"Ren, M., Zeng, W., Yang, B. & Urtasun, R. Learning to Reweight Examples for Robust Deep Learning. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Researchin 80,4334-4343 (2018)."},{"key":"662_CR22","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1148\/radiol.2018180694","volume":"290","author":"CD Lehman","year":"2019","unstructured":"Lehman, C. D. et al. Mammographic breast density assessment using deep learning: clinical implementation. Radiology 290, 52-58, https:\/\/doi.org\/10.1148\/radiol.2018180694 (2019).","journal-title":"Radiology"},{"key":"662_CR23","unstructured":"Yan, Y. et al. in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics Vol. 9 (eds Teh Yee Whye & Titterington Mike) 932--939 (PMLR, Proceedings of Machine Learning Research, 2010)."},{"key":"662_CR24","doi-asserted-by":"publisher","first-page":"8777","DOI":"10.1073\/pnas.1601827113","volume":"113","author":"RHJM Kurvers","year":"2016","unstructured":"Kurvers, R. H. J. M. et al. Boosting medical diagnostics by pooling independent judgments. Proceedings of the National Academy of Sciences 113, 8777-8782, https:\/\/doi.org\/10.1073\/pnas.1601827113 (2016).","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"662_CR25","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1016\/j.ejrad.2017.09.013","volume":"96","author":"M Posso","year":"2017","unstructured":"Posso, M. et al. Effectiveness and cost-effectiveness of double reading in digital mammography screening: a systematic review and meta-analysis. European journal of radiology 96, 40-49 (2017).","journal-title":"European journal of radiology"},{"key":"662_CR26","doi-asserted-by":"publisher","first-page":"1272","DOI":"10.1056\/NEJM197306142882405","volume":"288","author":"AV Milholland","year":"1973","unstructured":"Milholland, A. V., Wheeler, S. G. & Heieck, J. J. Medical assessment by a Delphi group opinion technic. New England Journal of Medicine 288, 1272-1275 (1973).","journal-title":"New England Journal of Medicine"},{"key":"662_CR27","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1148\/radiol.12111930","volume":"264","author":"N Mamisch","year":"2012","unstructured":"Mamisch, N. et al. Radiologic criteria for the diagnosis of spinal stenosis: results of a Delphi survey. Radiology 264, 174-179, https:\/\/doi.org\/10.1148\/radiol.12111930 (2012).","journal-title":"Radiology"},{"key":"662_CR28","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1511\/2006.59.220","volume":"94","author":"TD Seeley","year":"2006","unstructured":"Seeley, T. D., Visscher, P. K. & Passino, K. M. Group decision making in honey bee swarms: when 10,000 bees go house hunting, how do they cooperatively choose their new nesting site? American Scientist 94, 220-229 (2006).","journal-title":"American Scientist"},{"key":"662_CR29","doi-asserted-by":"crossref","unstructured":"Bonabeau, E. et al. Swarm Intelligence: From Natural to Artificial Systems. (OUP USA, 1999).","DOI":"10.1093\/oso\/9780195131581.001.0001"},{"key":"662_CR30","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1016\/j.tree.2009.06.016","volume":"25","author":"J Krause","year":"2010","unstructured":"Krause, J., Ruxton, G. D. & Krause, S. Swarm intelligence in animals and humans. Trends in Ecology & Evolution 25, 28-34, https:\/\/doi.org\/10.1016\/j.tree.2009.06.016 (2010).","journal-title":"Trends in Ecology & Evolution"},{"key":"662_CR31","doi-asserted-by":"publisher","first-page":"877","DOI":"10.1126\/science.1157679","volume":"320","author":"KJ Arrow","year":"2008","unstructured":"Arrow, K. J. et al. The promise of prediction markets. Science-new york then washington- 320, 877 (2008).","journal-title":"Science-new york then washington-"},{"key":"662_CR32","doi-asserted-by":"publisher","unstructured":"Rosenberg, L., Lungren, M., Halabi, S., Willcox, G., Baltaxe, D., & Lyons, M. Artificial Swarm Intelligence employed to Amplify Diagnostic Accuracy in Radiology. In S. Chakrabarti, & H. N. Saha (Eds.), 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference, IEMCON 2018 (pp. 1186-1191). [8614883] (2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference, IEMCON 2018). Institute of Electrical and Electronics Engineers Inc. https:\/\/doi.org\/10.1109\/IEMCON.2018.8614883 (2019).","DOI":"10.1109\/IEMCON.2018.8614883"},{"key":"662_CR33","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1023\/A:1022371810032","volume":"1","author":"W Sulis","year":"1997","unstructured":"Sulis, W. Fundamental concepts of collective intelligence. Nonlinear Dynamics, Psychology, and Life Sciences 1, 35-53, https:\/\/doi.org\/10.1023\/A:1022371810032 (1997).","journal-title":"Nonlinear Dynamics, Psychology, and Life Sciences"},{"key":"662_CR34","doi-asserted-by":"publisher","unstructured":"Galton, F. Vox Populi. Nature 75, 450\u2013451. https:\/\/doi.org\/10.1038\/075450a0 (1907).","DOI":"10.1038\/075450a0"},{"key":"662_CR35","unstructured":"Salminen, J. Collective intelligence in humans: a literature review. arXiv preprint http:\/\/arxiv.org\/abs\/1204.3401 (2012)."},{"key":"662_CR36","doi-asserted-by":"publisher","first-page":"1081","DOI":"10.1126\/science.1185718","volume":"329","author":"B Bahrami","year":"2010","unstructured":"Bahrami, B. et al. Optimally interacting minds. Science 329, 1081-1085 (2010).","journal-title":"Science"},{"key":"662_CR37","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1016\/0001-6918(92)90012-3","volume":"81","author":"J Shanteau","year":"1992","unstructured":"Shanteau, J. How much information does an expert use? Is it relevant? Acta psychologica 81, 75-86 (1992).","journal-title":"Acta psychologica"},{"key":"662_CR38","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1177\/1529100614525555","volume":"15","author":"M Kozhevnikov","year":"2014","unstructured":"Kozhevnikov, M., Evans, C. & Kosslyn, S. M. Cognitive style as environmentally sensitive individual differences in cognition: a modern synthesis and applications in education, business, and management. Psychological Science in the Public Interest 15, 3-33, https:\/\/doi.org\/10.1177\/1529100614525555 (2014).","journal-title":"Psychological Science in the Public Interest"},{"key":"662_CR39","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1037\/0022-3514.52.1.81","volume":"52","author":"RR McCrae","year":"1987","unstructured":"McCrae, R. R. & Costa, P. T. Validation of the five-factor model of personality across instruments and observers. Journal of personality and social psychology 52, 81 (1987).","journal-title":"Journal of personality and social psychology"},{"key":"662_CR40","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1111\/1469-7610.00715","volume":"42","author":"S Baron-Cohen","year":"2001","unstructured":"Baron\u2010Cohen, S., Wheelwright, S., Hill, J., Raste, Y. & Plumb, I. The \u201cReading the mind in the eyes\u201d test revised version: a study with normal adults, and adults with Asperger syndrome or high\u2010functioning autism. Journal of child psychology and psychiatry 42, 241-251 (2001).","journal-title":"Journal of child psychology and psychiatry"},{"key":"662_CR41","unstructured":"Rosenberg, L. & Willcox, G. 1054-1070 (Springer International Publishing)."},{"key":"662_CR42","doi-asserted-by":"crossref","unstructured":"Rosenberg, L. in 2016 International Joint Conference on Neural Networks (IJCNN). 2547-2551.","DOI":"10.1109\/IJCNN.2016.7727517"},{"key":"662_CR43","doi-asserted-by":"publisher","first-page":"688","DOI":"10.1002\/jor.23277","volume":"35","author":"C Russell","year":"2017","unstructured":"Russell, C. et al. Baseline cartilage quality is associated with voxel-based T1\u03c1 and T2 following ACL reconstruction: a multicenter pilot study. Journal of Orthopaedic Research 35, 688-698, https:\/\/doi.org\/10.1002\/jor.23277 (2017).","journal-title":"Journal of Orthopaedic Research"},{"key":"662_CR44","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1016\/j.joca.2003.11.003","volume":"12","author":"CG Peterfy","year":"2004","unstructured":"Peterfy, C. G. et al. Whole-organ magnetic resonance imaging score (WORMS) of the knee in osteoarthritis. Osteoarthritis Cartilage 12, 177-190, https:\/\/doi.org\/10.1016\/j.joca.2003.11.003 (2004).","journal-title":"Osteoarthritis Cartilage"},{"key":"662_CR45","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1038\/s41746-019-0189-7","volume":"2","author":"BN Patel","year":"2019","unstructured":"Patel, B. N. et al. Human-machine partnership with artificial intelligence for chest radiograph diagnosis. NPJ Digit Med 2, 111, https:\/\/doi.org\/10.1038\/s41746-019-0189-7 (2019).","journal-title":"NPJ Digit Med"},{"key":"662_CR46","doi-asserted-by":"publisher","unstructured":"Astuto, B. et al. Automatic deep learning assisted detection and grading of abnormalities in knee MRI studies. Radiology: Artificial Intelligence 0, e200165, https:\/\/doi.org\/10.1148\/ryai.2021200165\u00a0(2021)","DOI":"10.1148\/ryai.2021200165"},{"key":"662_CR47","doi-asserted-by":"publisher","first-page":"1525","DOI":"10.1007\/s00167-015-3861-8","volume":"24","author":"N Phelan","year":"2016","unstructured":"Phelan, N., Rowland, P., Galvin, R. & O'Byrne, J. M. A systematic review and meta-analysis of the diagnostic accuracy of MRI for suspected ACL and meniscal tears of the knee. Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA 24, 1525-1539, https:\/\/doi.org\/10.1007\/s00167-015-3861-8 (2016).","journal-title":"Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA"},{"key":"662_CR48","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1136\/jisakos-2019-000408","volume":"5","author":"J Shah","year":"2020","unstructured":"Shah, J. et al. Correlation of meniscus tears on MRI and arthroscopy using the ISAKOS classification provides satisfactory intermethod and inter-rater reliability. Journal of ISAKOS: Joint Disorders & Orthopaedic Sports Medicine 5, 201-207, https:\/\/doi.org\/10.1136\/jisakos-2019-000408 (2020).","journal-title":"Journal of ISAKOS: Joint Disorders & Orthopaedic Sports Medicine"},{"key":"662_CR49","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1016\/j.jacr.2015.11.003","volume":"13","author":"JA Harolds","year":"2016","unstructured":"Harolds, J. A., Parikh, J. R., Bluth, E. I., Dutton, S. C. & Recht, M. P. Burnout of radiologists: frequency, risk factors, and remedies: a report of the acr commission on human resources. Journal of the American College of Radiology 13, 411-416, https:\/\/doi.org\/10.1016\/j.jacr.2015.11.003 (2016).","journal-title":"Journal of the American College of Radiology"},{"key":"662_CR50","doi-asserted-by":"publisher","first-page":"1207","DOI":"10.1007\/s00256-020-03410-2","volume":"49","author":"B Fritz","year":"2020","unstructured":"Fritz, B., Marbach, G., Civardi, F., Fucentese, S. F. & Pfirrmann, C. W. A. Deep convolutional neural network-based detection of meniscus tears: comparison with radiologists and surgery as standard of reference. Skeletal radiology 49, 1207-1217, https:\/\/doi.org\/10.1007\/s00256-020-03410-2 (2020).","journal-title":"Skeletal radiology"},{"key":"662_CR51","doi-asserted-by":"publisher","unstructured":"Kim, S. H., Lee, H. J., Jang, Y. H., Chun, K. J. & Park, Y. B. Diagnostic accuracy of magnetic resonance imaging in the detection of type and location of meniscus tears: comparison with arthroscopic findings. Journal of clinical medicine 10, https:\/\/doi.org\/10.3390\/jcm10040606 (2021).","DOI":"10.3390\/jcm10040606"},{"key":"662_CR52","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1007\/s10606-009-9094-y","volume":"18","author":"B Kane","year":"2009","unstructured":"Kane, B. & Luz, S. Achieving diagnosis by consensus. Computer Supported Cooperative Work (CSCW) 18, 357-392, https:\/\/doi.org\/10.1007\/s10606-009-9094-y (2009).","journal-title":"Computer Supported Cooperative Work (CSCW)"},{"key":"662_CR53","doi-asserted-by":"publisher","first-page":"536","DOI":"10.1177\/0272989x15581615","volume":"36","author":"MW Kattan","year":"2016","unstructured":"Kattan, M. W., O\u2019Rourke, C., Yu, C. & Chagin, K. The wisdom of crowds of doctors: their average predictions outperform their individual ones. Medical Decision Making 36, 536-540, https:\/\/doi.org\/10.1177\/0272989x15581615 (2016).","journal-title":"Medical Decision Making"},{"key":"662_CR54","doi-asserted-by":"publisher","first-page":"1076","DOI":"10.3758\/s13423-014-0765-4","volume":"22","author":"AA Brennan","year":"2015","unstructured":"Brennan, A. A. & Enns, J. T. When two heads are better than one: Interactive versus independent benefits of collaborative cognition. Psychonomic Bulletin & Review 22, 1076-1082, https:\/\/doi.org\/10.3758\/s13423-014-0765-4 (2015).","journal-title":"Psychonomic Bulletin & Review"},{"key":"662_CR55","doi-asserted-by":"publisher","first-page":"9020","DOI":"10.1073\/pnas.1008636108","volume":"108","author":"J Lorenz","year":"2011","unstructured":"Lorenz, J., Rauhut, H., Schweitzer, F. & Helbing, D. How social influence can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences 108, 9020-9025, https:\/\/doi.org\/10.1073\/pnas.1008636108 (2011).","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"662_CR56","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1126\/science.1221403","volume":"336","author":"R Hertwig","year":"2012","unstructured":"Hertwig, R. Tapping into the wisdom of the crowd\u2013with confidence. Science 336, 303-304 (2012).","journal-title":"Science"},{"key":"662_CR57","doi-asserted-by":"crossref","unstructured":"Perry, N. et al. European guidelines for quality assurance in breast cancer screening and diagnosis. -summary document. Oncology in Clinical Practice 4, 74-86 (2008).","DOI":"10.1093\/annonc\/mdm481"}],"container-title":["Journal of Digital Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-022-00662-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-022-00662-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-022-00662-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,9]],"date-time":"2024-10-09T10:38:25Z","timestamp":1728470305000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-022-00662-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,22]]},"references-count":57,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["662"],"URL":"https:\/\/doi.org\/10.1007\/s10278-022-00662-3","relation":{},"ISSN":["1618-727X"],"issn-type":[{"type":"electronic","value":"1618-727X"}],"subject":[],"published":{"date-parts":[[2022,11,22]]},"assertion":[{"value":"21 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 April 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 May 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 November 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The following study was conducted in accordance with IRB process at the primary medical center. The study utilized retrospectively acquired data only, and the IRB committee determined that our study did not require an ethics approval.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}]}}