{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T23:03:47Z","timestamp":1781737427398,"version":"3.54.5"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,2,5]],"date-time":"2021-02-05T00:00:00Z","timestamp":1612483200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,2,5]],"date-time":"2021-02-05T00:00:00Z","timestamp":1612483200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002954","name":"Universit\u00e0 degli Studi di Milano - Bicocca","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100002954","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Health Inf Sci Syst"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>The integration of Artificial Intelligence into medical practices has recently been advocated for the promise to bring increased efficiency and effectiveness to these practices. Nonetheless, little research has so far been aimed at understanding the best human-AI interaction protocols in collaborative tasks, even in currently more viable settings, like independent double-reading screening tasks.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>To this aim, we report about a retrospective case\u2013control study, involving 12 board-certified radiologists, in the detection of knee lesions by means of Magnetic Resonance Imaging, in which we simulated the serial combination of two Deep Learning models with humans in eight double-reading protocols. Inspired by the so-called Kasparov\u2019s Laws, we investigate whether the combination of humans and AI models could achieve better performance than AI models alone, and whether weak reader, when supported by fit-for-use interaction protocols, could out-perform stronger readers.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>We discuss two main findings: groups of humans who perform significantly worse than a state-of-the-art AI can significantly outperform it if their judgements are aggregated by majority voting (in concordance with the first part of the Kasparov\u2019s law); small ensembles of significantly weaker readers can significantly outperform teams of stronger readers, supported by the same computational tool, when the judgments of the former ones are combined within \u201cfit-for-use\u201d protocols (in concordance with the second part of the Kasparov\u2019s law).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>Our study shows that good interaction protocols can guarantee improved decision performance that easily surpasses the performance of individual agents, even of realistic super-human AI systems. This finding highlights the importance of focusing on how to guarantee better co-operation within human-AI teams, so to enable safer and more human sustainable care practices.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s13755-021-00138-8","type":"journal-article","created":{"date-parts":[[2021,2,8]],"date-time":"2021-02-08T21:22:37Z","timestamp":1612819357000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Studying human-AI collaboration protocols: the case of the Kasparov\u2019s law in radiological double reading"],"prefix":"10.1007","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4065-3415","authenticated-orcid":false,"given":"Federico","family":"Cabitza","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Campagner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luca Maria","family":"Sconfienza","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,2,5]]},"reference":[{"issue":"1","key":"138_CR1","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1111\/1754-9485.12828","volume":"63","author":"M Adams","year":"2019","unstructured":"Adams M, Chen W, Holcdorf D, McCusker MW, Howe PD, Gaillard F. Computer vs human: deep learning versus perceptual training for the detection of neck of femur fractures. J Med Imaging Radiat Oncol. 2019;63(1):27\u201332.","journal-title":"J Med Imaging Radiat Oncol"},{"issue":"8","key":"138_CR2","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/MC.2020.2996587","volume":"53","author":"Z Akata","year":"2020","unstructured":"Akata Z, Balliet D, de Rijke M, Dignum F, Dignum V, Eiben G, Fokkens A, Grossi D, Hindriks K, Hoos H, Hung H, Jonker C, Monz C, Neerincx M, Oliehoek F, Prakken H, Schlobach S, van der Gaag L, van Harmelen F, van Hoof H, van Riemsdijk B, van Wynsberghe A, Verbrugge R, Verheij B, Vossen P, Welling M. A research agenda for hybrid intelligence: augmenting human intellect with collaborative, adaptive, responsible, and explainable artificial intelligence. Computer. 2020;53(8):18\u201328. https:\/\/doi.org\/10.1109\/MC.2020.2996587.","journal-title":"Computer"},{"issue":"8","key":"138_CR3","doi-asserted-by":"publisher","first-page":"909","DOI":"10.1016\/j.acra.2004.05.012","volume":"11","author":"E Alberdi","year":"2004","unstructured":"Alberdi E, Povyakalo A, Strigini L, Ayton P. Effects of incorrect computer-aided detection (cad) output on human decision-making in mammography. Acad Radiol. 2004;11(8):909\u201318.","journal-title":"Acad Radiol"},{"issue":"3","key":"138_CR4","doi-asserted-by":"publisher","first-page":"e190096","DOI":"10.1001\/jamanetworkopen.2019.0096","volume":"2","author":"ML Barnett","year":"2019","unstructured":"Barnett ML, Boddupalli D, Nundy S, Bates DW. Comparative accuracy of diagnosis by collective intelligence of multiple physicians vs individual physicians. JAMA Netw Open. 2019;2(3):e190096.","journal-title":"JAMA Netw Open"},{"key":"138_CR5","volume-title":"The second machine age: work, progress, and prosperity in a time of brilliant technologies","author":"E Brynjolfsson","year":"2014","unstructured":"Brynjolfsson E, McAfee A. The second machine age: work, progress, and prosperity in a time of brilliant technologies. New York: WW Norton & Company; 2014."},{"key":"138_CR6","doi-asserted-by":"crossref","unstructured":"Cabitza, F. Biases affecting human decision making in AI-supported second opinion settings. Lecture notes in computer science 11676 LNAI, 2019 pp. 283\u2013294.","DOI":"10.1007\/978-3-030-26773-5_25"},{"key":"138_CR7","unstructured":"Cabitza F, Campagner A, Del\u00a0Zotti F, Ravizza A, Sternini F. All you need is higher accuracy? on the quest for minimum acceptable accuracy for medical artificial intelligence. In: eHealth 2020: Proceedings of eHealth, the 12th International Conference on e-Health. 2020; 21\u201323 July 2020."},{"issue":"1","key":"138_CR8","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1186\/s12911-020-01224-9","volume":"20","author":"F Cabitza","year":"2020","unstructured":"Cabitza F, Campagner A, Sconfienza LM. As if sand were stone, new concepts and metrics to probe the ground on which to build trustable AI. BMC Med Inf Decis Making. 2020;20(1):219. https:\/\/doi.org\/10.1186\/s12911-020-01224-9.","journal-title":"BMC Med Inf Decis Making"},{"issue":"6","key":"138_CR9","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1001\/jama.2017.7797","volume":"318","author":"F Cabitza","year":"2017","unstructured":"Cabitza F, Rasoini R, Gensini GF. Unintended consequences of machine learning in medicine. JAMA. 2017;318(6):517\u20138.","journal-title":"JAMA"},{"key":"138_CR10","doi-asserted-by":"crossref","unstructured":"Campagner A, Ciucci D, Cabitza F. Ensemble learning, social choice and collective intelligence: An experimental comparison of aggregation techniques. Lecture Notes in Computer Science 12256 LNAI, 2020 pp. 53\u201365.","DOI":"10.1007\/978-3-030-57524-3_5"},{"key":"138_CR11","unstructured":"Caritat MJA. Essai sur l\u2019application de l\u2019analyse \u00e0 la probabilit\u00e9 des d\u00e9cisions rendues \u00e0 la pluralit\u00e9 des voix. De l\u2019Imprimerie royale 1785."},{"key":"138_CR12","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1148\/radiol.2020201495","volume":"296","author":"JJ Cavallo","year":"2020","unstructured":"Cavallo JJ, Forman HP. The economic impact of the covid-19 pandemic on radiology practices. Radiology. 2020;296:141.","journal-title":"Radiology"},{"issue":"8","key":"138_CR13","doi-asserted-by":"publisher","first-page":"963","DOI":"10.1093\/jamia\/ocy028","volume":"25","author":"E Coiera","year":"2018","unstructured":"Coiera E, Ammenwerth E, Georgiou A, Magrabi F. Does health informatics have a replication crisis? J Am Med Inf Assoc. 2018;25(8):963\u20138.","journal-title":"J Am Med Inf Assoc"},{"issue":"10","key":"138_CR14","doi-asserted-by":"publisher","first-page":"1260","DOI":"10.1016\/j.acra.2012.05.013","volume":"19","author":"T Drew","year":"2012","unstructured":"Drew T, Cunningham C, Wolfe JM. When and why might a computer-aided detection (cad) system interfere with visual search? an eye-tracking study. Acad Radiol. 2012;19(10):1260\u20137.","journal-title":"Acad Radiol"},{"key":"138_CR15","doi-asserted-by":"crossref","unstructured":"Gale W, Oakden-Rayner L, Carneiro G, Palmer LJ, Bradley AP. Producing radiologist-quality reports for interpretable deep learning. In: 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019), 2019; pp. 1275\u20131279. IEEE.","DOI":"10.1109\/ISBI.2019.8759236"},{"issue":"3","key":"138_CR16","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1007\/s13244-018-0599-0","volume":"9","author":"H Geijer","year":"2018","unstructured":"Geijer H, Geijer M. Added value of double reading in diagnostic radiology, a systematic review. Insights Imaging. 2018;9(3):287\u2013301.","journal-title":"Insights Imaging"},{"issue":"2","key":"138_CR17","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1002\/jmri.25970","volume":"48","author":"E Gong","year":"2018","unstructured":"Gong E, Pauly JM, Wintermark M, Zaharchuk G. Deep learning enables reduced gadolinium dose for contrast-enhanced brain MRI. J Magn Reson Imaging. 2018;48(2):330\u201340.","journal-title":"J Magn Reson Imaging"},{"issue":"11","key":"138_CR18","doi-asserted-by":"publisher","first-page":"e651","DOI":"10.1016\/S1470-2045(17)30806-9","volume":"18","author":"E Gourd","year":"2017","unstructured":"Gourd E. UK radiologist staffing crisis reaches critical levels. Lancet Oncol. 2017;18(11):e651.","journal-title":"Lancet Oncol"},{"key":"138_CR19","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1016\/j.jclinepi.2020.03.005","volume":"122","author":"BY Gravesteijn","year":"2020","unstructured":"Gravesteijn BY, Nieboer D, Ercole A, Lingsma HF, Nelson D, Van Calster B, Steyerberg EW, \u00c5kerlund C, Amrein K, Andelic N, et al. Machine learning algorithms performed no better than regression models for prognostication in traumatic brain injury. J Clin Epidemiol. 2020;122:95\u2013107.","journal-title":"J Clin Epidemiol"},{"issue":"8","key":"138_CR20","doi-asserted-by":"publisher","first-page":"1836","DOI":"10.1093\/annonc\/mdy166","volume":"29","author":"HA Haenssle","year":"2018","unstructured":"Haenssle HA, Fink C, Schneiderbauer R, Toberer F, Buhl T, Blum A, Kalloo A, Hassen ABH, Thomas L, Enk A, et al. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol. 2018;29(8):1836\u201342.","journal-title":"Ann Oncol"},{"key":"138_CR21","doi-asserted-by":"publisher","DOI":"10.1177\/1591019920953055","author":"AE Hassan","year":"2020","unstructured":"Hassan AE, Ringheanu VM, Rabah RR, Preston L, Tekle WG, Qureshi AI. Early experience utilizing artificial intelligence shows significant reduction in transfer times and length of stay in a hub and spoke model. Intervent Neuroradiol. 2020;. https:\/\/doi.org\/10.1177\/1591019920953055.","journal-title":"Intervent Neuroradiol"},{"key":"138_CR22","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1016\/j.ejca.2019.07.019","volume":"120","author":"A Hekler","year":"2019","unstructured":"Hekler A, Utikal JS, Enk AH, Hauschild A, Weichenthal M, Maron RC, Berking C, Haferkamp S, Klode J, Schadendorf D, et al. Superior skin cancer classification by the combination of human and artificial intelligence. Eur J Cancer. 2019;120:114\u201321.","journal-title":"Eur J Cancer"},{"issue":"2","key":"138_CR23","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1007\/s40708-016-0042-6","volume":"3","author":"A Holzinger","year":"2016","unstructured":"Holzinger A. Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Inf. 2016;3(2):119\u201331.","journal-title":"Brain Inf"},{"key":"138_CR24","unstructured":"Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 2017."},{"issue":"4","key":"138_CR25","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1111\/medu.12943","volume":"50","author":"A Kaba","year":"2016","unstructured":"Kaba A, Wishart I, Fraser K, Coderre S, McLaughlin K. Are we at risk of groupthink in our approach to teamwork interventions in health care? Med Educ. 2016;50(4):400\u20138.","journal-title":"Med Educ"},{"key":"138_CR26","unstructured":"Kasparov G. Deep thinking: where machine intelligence ends and human creativity begins. Hachette, UK, 2017."},{"issue":"4","key":"138_CR27","doi-asserted-by":"publisher","first-page":"536","DOI":"10.1177\/0272989X15581615","volume":"36","author":"MW Kattan","year":"2016","unstructured":"Kattan MW, O\u2019Rourke C, Yu C, Chagin K. The wisdom of crowds of doctors: their average predictions outperform their individual ones. Med Decis Making. 2016;36(4):536\u201340.","journal-title":"Med Decis Making"},{"issue":"3","key":"138_CR28","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1038\/s41571-020-0329-7","volume":"17","author":"D Killock","year":"2020","unstructured":"Killock D. Ai outperforms radiologists in mammographic screening. Nat Rev Clin Oncol. 2020;17(3):134.","journal-title":"Nat Rev Clin Oncol"},{"issue":"11","key":"138_CR29","doi-asserted-by":"publisher","first-page":"1828","DOI":"10.1001\/jamainternmed.2015.5231","volume":"175","author":"CD Lehman","year":"2015","unstructured":"Lehman CD, Wellman RD, Buist DS, Kerlikowske K, Tosteson AN, Miglioretti DL. Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Intern Med. 2015;175(11):1828\u201337.","journal-title":"JAMA Intern Med"},{"issue":"6","key":"138_CR30","doi-asserted-by":"publisher","first-page":"e271","DOI":"10.1016\/S2589-7500(19)30123-2","volume":"1","author":"X Liu","year":"2019","unstructured":"Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health. 2019;1(6):e271\u201397.","journal-title":"Lancet Digit Health"},{"issue":"2","key":"138_CR31","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1093\/jamia\/ocw105","volume":"24","author":"D Lyell","year":"2017","unstructured":"Lyell D, Coiera E. Automation bias and verification complexity: a systematic review. J Am Med Inf Assoc. 2017;24(2):423\u201331.","journal-title":"J Am Med Inf Assoc"},{"issue":"8","key":"138_CR32","doi-asserted-by":"publisher","first-page":"739","DOI":"10.1001\/jama.2019.0286","volume":"321","author":"KD Mandl","year":"2019","unstructured":"Mandl KD, Manrai AK. Potential excessive testing at scale: biomarkers, genomics, and machine learning. JAMA. 2019;321(8):739\u201340.","journal-title":"JAMA"},{"key":"138_CR33","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1136\/bmj.m689","volume":"368","author":"M Nagendran","year":"2020","unstructured":"Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, Topol EJ, Ioannidis JP, Collins GS, Maruthappu M. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ. 2020;368:689.","journal-title":"BMJ"},{"issue":"3","key":"138_CR34","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1515\/dx-2020-0020","volume":"7","author":"AP Olson","year":"2020","unstructured":"Olson AP, Durning SJ, Branson CF, Sick B, Lane KP, Rencic JJ. Teamwork in clinical reasoning-cooperative or parallel play? Diagnosis. 2020;7(3):307\u201312.","journal-title":"Diagnosis"},{"issue":"7","key":"138_CR35","doi-asserted-by":"publisher","first-page":"e0159806","DOI":"10.1371\/journal.pone.0159806","volume":"11","author":"M Posso","year":"2016","unstructured":"Posso M, Carles M, Ru\u00e9 M, Puig T, Bonfill X. Cost-effectiveness of double reading versus single reading of mammograms in a breast cancer screening programme. PLoS ONE. 2016;11(7):e0159806.","journal-title":"PLoS ONE"},{"issue":"7638","key":"138_CR36","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1038\/nature21054","volume":"541","author":"D Prelec","year":"2017","unstructured":"Prelec D, Seung HS, McCoy J. A solution to the single-question crowd wisdom problem. Nature. 2017;541(7638):532\u20135.","journal-title":"Nature"},{"issue":"18","key":"138_CR37","doi-asserted-by":"publisher","first-page":"1765","DOI":"10.1001\/jama.2019.15064","volume":"322","author":"WN Price","year":"2019","unstructured":"Price WN, Gerke S, Cohen IG. Potential liability for physicians using artificial intelligence. JAMA. 2019;322(18):1765\u20136.","journal-title":"JAMA"},{"issue":"1","key":"138_CR38","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1186\/s12911-019-0882-0","volume":"19","author":"K Radcliffe","year":"2019","unstructured":"Radcliffe K, Lyson HC, Barr-Walker J, Sarkar U. Collective intelligence in medical decision-making: a systematic scoping review. BMC Med Inf Decis Making. 2019;19(1):158.","journal-title":"BMC Med Inf Decis Making"},{"key":"138_CR39","doi-asserted-by":"publisher","first-page":"1581","DOI":"10.1001\/jamaoncol.2020.3321","volume":"6","author":"M Salim","year":"2020","unstructured":"Salim M, W\u00e5hlin E, Dembrower K, Azavedo E, Foukakis T, Liu Y, Smith K, Eklund M, Strand F. External evaluation of 3 commercial artificial intelligence algorithms for independent assessment of screening mammograms. JAMA Oncol. 2020;6:1581\u20138.","journal-title":"JAMA Oncol"},{"key":"138_CR40","doi-asserted-by":"crossref","unstructured":"Schaekermann M, Beaton G, Sanoubari E, Lim A, Larson K, Law E. Ambiguity-aware ai assistants for medical data analysis. In: Proceedings of the 2020 CHI conference on human factors in computing systems, 2020; pp. 1\u201314.","DOI":"10.1145\/3313831.3376506"},{"issue":"2\u20133","key":"138_CR41","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1007\/BF00133655","volume":"5","author":"K Schmidt","year":"1996","unstructured":"Schmidt K, Simonee C. Coordination mechanisms: towards a conceptual foundation of CSCW systems design. Comput Support Cooper Work. 1996;5(2\u20133):155\u2013200.","journal-title":"Comput Support Cooper Work"},{"issue":"2","key":"138_CR42","doi-asserted-by":"publisher","first-page":"103174","DOI":"10.1016\/j.im.2019.103174","volume":"57","author":"I Seeber","year":"2020","unstructured":"Seeber I, Bittner E, Briggs RO, de Vreede T, De Vreede GJ, Elkins A, Maier R, Merz AB, Oeste-Rei\u00df S, Randrup N, et al. Machines as teammates: a research agenda on ai in team collaboration. Inf Manag. 2020;57(2):103174.","journal-title":"Inf Manag"},{"issue":"1","key":"138_CR43","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1207\/S15327108IJAP1001_5","volume":"10","author":"LJ Skitka","year":"2000","unstructured":"Skitka LJ, Mosier KL, Burdick M, Rosenblatt B. Automation bias and errors: are crews better than individuals? Int J Aviation Psychol. 2000;10(1):85\u201397.","journal-title":"Int J Aviation Psychol"},{"key":"138_CR44","doi-asserted-by":"publisher","first-page":"e100081","DOI":"10.1136\/bmjhci-2019-100081","volume":"1","author":"M Sujan","year":"2019","unstructured":"Sujan M, White S, Furniss D, Habli I, Grundy K, Grundy H, Nelson D, Elliott M, Reynolds N. Human factors challenges for the safe use of artificial intelligence in patient care. BMJ Health Care Inf. 2019;1:e100081.","journal-title":"BMJ Health Care Inf"},{"issue":"2","key":"138_CR45","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1001\/amajethics.2019.160","volume":"21","author":"HR Sullivan","year":"2019","unstructured":"Sullivan HR, Schweikart SJ. Are current tort liability doctrines adequate for addressing injury caused by ai? AMA J Ethics. 2019;21(2):160\u20136.","journal-title":"AMA J Ethics"},{"key":"138_CR46","doi-asserted-by":"crossref","unstructured":"Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016; pp. 2818\u20132826.","DOI":"10.1109\/CVPR.2016.308"},{"key":"138_CR47","unstructured":"Tenner E. The efficiency paradox: what Big Data can\u2019t do. Vintage; 2018."},{"key":"138_CR48","doi-asserted-by":"publisher","first-page":"104231","DOI":"10.1016\/j.ijmedinf.2020.104231","volume":"141","author":"E Uchino","year":"2020","unstructured":"Uchino E, Suzuki K, Sato N, Kojima R, Tamada Y, Hiragi S, Yokoi H, Yugami N, Minamiguchi S, Haga H, et al. Classification of glomerular pathological findings using deep learning and nephrologist-ai collective intelligence approach. Int J Med Inf. 2020;141:104231.","journal-title":"Int J Med Inf"},{"issue":"8","key":"138_CR49","doi-asserted-by":"publisher","first-page":"e0134269","DOI":"10.1371\/journal.pone.0134269","volume":"10","author":"M Wolf","year":"2015","unstructured":"Wolf M, Krause J, Carney PA, Bogart A, Kurvers RH. Collective intelligence meets medical decision-making: the collective outperforms the best radiologist. PLoS ONE. 2015;10(8):e0134269.","journal-title":"PLoS ONE"},{"key":"138_CR50","doi-asserted-by":"publisher","first-page":"869","DOI":"10.3389\/fneur.2019.00869","volume":"10","author":"G Zhu","year":"2019","unstructured":"Zhu G, Jiang B, Tong L, Xie Y, Zaharchuk G, Wintermark M. Applications of deep learning to neuro-imaging techniques. Front Neurol. 2019;10:869.","journal-title":"Front Neurol"}],"container-title":["Health Information Science and Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13755-021-00138-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s13755-021-00138-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13755-021-00138-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,3]],"date-time":"2021-12-03T14:18:57Z","timestamp":1638541137000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s13755-021-00138-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,5]]},"references-count":50,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["138"],"URL":"https:\/\/doi.org\/10.1007\/s13755-021-00138-8","relation":{},"ISSN":["2047-2501"],"issn-type":[{"value":"2047-2501","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,5]]},"assertion":[{"value":"17 September 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 January 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 February 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of interest"}},{"value":"No permissions were required to use the MRNet dataset (all rights reserved by the Stanford University School of Medicine) as this was used according to the Research Use Agreement. We obtained the consent to publish the original annotations of this dataset by the involved clinicians. These latter ones are mentioned in the acknowledgements section below.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"The code of the scripts used in the experimental part, and all data are available shared at:","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code and data availability"}}],"article-number":"8"}}