{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,9]],"date-time":"2026-08-09T10:02:59Z","timestamp":1786269779982,"version":"build-2736575974"},"reference-count":21,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T00:00:00Z","timestamp":1614038400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T00:00:00Z","timestamp":1614038400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Dres. Jutta"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In many research areas, scientific progress is accelerated by multidisciplinary access to image data and their interdisciplinary annotation. However, keeping track of these annotations to ensure a high-quality multi-purpose data set is a challenging and labour intensive task. We developed the open-source online platform EXACT (EXpert Algorithm Collaboration Tool) that enables the collaborative interdisciplinary analysis of images from different domains online and offline. EXACT supports multi-gigapixel medical whole slide images as well as image series with thousands of images. The software utilises a flexible plugin system that can be adapted to diverse applications such as counting mitotic figures with a screening mode, finding false annotations on a novel validation view, or using the latest deep learning image analysis technologies. This is combined with a version control system which makes it possible to keep track of changes in the data sets and, for example, to link the results of deep learning experiments to specific data set versions. EXACT is freely available and has already been successfully applied to a broad range of annotation tasks, including highly diverse applications like deep learning supported cytology scoring, interdisciplinary multi-centre whole slide image tumour annotation, and highly specialised whale sound spectroscopy clustering.<\/jats:p>","DOI":"10.1038\/s41598-021-83827-4","type":"journal-article","created":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T11:05:33Z","timestamp":1614078333000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":41,"title":["EXACT: a collaboration toolset for algorithm-aided annotation of images with annotation version control"],"prefix":"10.1038","volume":"11","author":[{"given":"Christian","family":"Marzahl","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc","family":"Aubreville","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christof A.","family":"Bertram","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jennifer","family":"Maier","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Bergler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christine","family":"Kr\u00f6ger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"J\u00f6rn","family":"Voigt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Katharina","family":"Breininger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Klopfleisch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas","family":"Maier","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,2,23]]},"reference":[{"key":"83827_CR1","doi-asserted-by":"crossref","unstructured":"Aubreville, M., Bertram, C., Klopfleisch, R. & Maier, A. Sliderunner. In Bildverarbeitung f\u00fcr die Medizin 2018 (eds Maier, A., Deserno, T., Handels, H., Maier-Hein, K., Palm, C. & Tolxdorff, T.) 309\u2013314 (Springer Vieweg, Berlin, Heidelberg, 2018).","DOI":"10.1007\/978-3-662-56537-7_81"},{"key":"83827_CR2","doi-asserted-by":"crossref","unstructured":"Hollandi, R., Di\u00f3sdi, \u00c1., Hollandi, G., Moshkov, N. & Horvath, P. Annotatorj: An imagej plugin to ease hand-annotation of cellular compartments. Mol. Biol. Cell 31(20), 2179\u20132186 (2020).","DOI":"10.1091\/mbc.E20-02-0156"},{"key":"83827_CR3","doi-asserted-by":"publisher","first-page":"690","DOI":"10.1038\/nmeth.2075","volume":"9","author":"F De Chaumont","year":"2012","unstructured":"De Chaumont, F. et al. Icy: An open bioimage informatics platform for extended reproducible research. Nat. Methods 9, 690 (2012).","journal-title":"Nat. Methods"},{"key":"83827_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-017-17204-5","volume":"7","author":"P Bankhead","year":"2017","unstructured":"Bankhead, P. et al. Qupath: Open source software for digital pathology image analysis. Sci. Rep. 7, 1\u20137 (2017).","journal-title":"Sci. Rep."},{"key":"83827_CR5","doi-asserted-by":"publisher","first-page":"1395","DOI":"10.1093\/bioinformatics\/btw013","volume":"32","author":"R Mar\u00e9e","year":"2016","unstructured":"Mar\u00e9e, R. et al. Collaborative analysis of multi-gigapixel imaging data using cytomine. Bioinformatics 32, 1395\u20131401 (2016).","journal-title":"Bioinformatics"},{"key":"83827_CR6","doi-asserted-by":"publisher","unstructured":"Puttapirat, P. et\u00a0al. Openhi\u2014An open source framework for annotating histopathological image. In 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 1076\u20131082 https:\/\/doi.org\/10.1109\/BIBM.2018.8621393 (2018).","DOI":"10.1109\/BIBM.2018.8621393"},{"key":"83827_CR7","doi-asserted-by":"crossref","unstructured":"Obando, D. F.\u00a0G., Mandache, D., Olivo-Marin, J.-C. & Meas-Yedid, V. Icytomine: A user-friendly tool for integrating workflows on whole slide images. In Digital Pathology (eds Reyes-Aldasoro, C. C., Janowczyk, A., Veta, M., Bankhead, P. & Sirinukunwattana, K.) 181\u2013189 (Springer International Publishing, Cham, 2019).","DOI":"10.1007\/978-3-030-23937-4_21"},{"key":"83827_CR8","doi-asserted-by":"crossref","unstructured":"Fiedler, N., Bestmann, M. & Hendrich, N. Imagetagger: An open source online platform for collaborative image labeling. In RoboCup 2018: Robot World Cup XXII (eds Holz, D., Genter, K., Saad, M. & von Stryk, O.) 162\u2013169 (Springer International Publishing, Cham, 2018).","DOI":"10.1007\/978-3-030-27544-0_13"},{"key":"83827_CR9","doi-asserted-by":"crossref","unstructured":"Goode, A., Gilbert, B., Harkes, J., Jukic, D. & Satyanarayanan, M. Openslide: A vendor-neutral software foundation for digital pathology. J. Pathol. Inform. 4(1), 27 (2013).","DOI":"10.4103\/2153-3539.119005"},{"key":"83827_CR10","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1111\/j.1939-1676.2002.tb02370.x","volume":"16","author":"MY Doucet","year":"2002","unstructured":"Doucet, M. Y. & Viel, L. Alveolar macrophage graded hemosiderin score from bronchoalveolar lavage in horses with exercise-induced pulmonary hemorrhage and controls. J. Vet. Intern. Med. 16, 281\u2013286 (2002).","journal-title":"J. Vet. Intern. Med."},{"key":"83827_CR11","doi-asserted-by":"crossref","unstructured":"Maier-Hein, L. et\u00a0al. Heidelberg colorectal data set for surgical data science in the sensor operating room. arXiv:2005.03501 (2020).","DOI":"10.1038\/s41597-021-00882-2"},{"key":"83827_CR12","first-page":"2579","volume":"9","author":"LVD Maaten","year":"2008","unstructured":"Maaten, L. V. D. & Hinton, G. Visualizing data using t-sne. J. Mach. Learn. Res. 9, 2579\u20132605 (2008).","journal-title":"J. Mach. Learn. Res."},{"key":"83827_CR13","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/0169-7439(87)80084-9","volume":"2","author":"S Wold","year":"1987","unstructured":"Wold, S., Esbensen, K. & Geladi, P. Principal component analysis. Chemometr. Intell. Lab. 2, 37\u201352 (1987).","journal-title":"Chemometr. Intell. Lab."},{"key":"83827_CR14","doi-asserted-by":"crossref","unstructured":"McInnes, L., Healy, J. & Melville, J. UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426 (2018).","DOI":"10.21105\/joss.00861"},{"key":"83827_CR15","doi-asserted-by":"crossref","unstructured":"\u00d8rting, S. N., Doyle, A., van Hilten, A., Hirth, M., Inel, O., Madan, C. R., Mavridis, P., Spiers, H. & Cheplygina, V. A survey of crowdsourcing in medical image analysis. Hum. Comput. 7(1), 1\u201326 (2020).","DOI":"10.15346\/hc.v7i1.111"},{"key":"83827_CR16","doi-asserted-by":"crossref","unstructured":"Marzahl, C. et al. Is crowd-algorithm collaboration an advanced alternative to crowd-sourcing on cytology slides? In Bildverarbeitung f\u00fcr die Medizin 2020 (eds Tolxdorff, T., Deserno, T., Handels, H., Maier, A., Maier-Hein, K. & Palm, C.) 26\u201331 (Springer Fachmedien Wiesbaden, Wiesbaden, 2020).","DOI":"10.1007\/978-3-658-29267-6_5"},{"key":"83827_CR17","unstructured":"Marzahl, C. et\u00a0al. Are fast labeling methods reliable? A case study of computer-aided expert annotations on microscopy slides. In Medical Image Computing and Computer Assisted Intervention \u2013 MICCAI 2020 (eds Martel, A. L.,  Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M. A., Zhou, S. K., Racoceanu, D. & Joskowicz, L.) 24\u201332 (Springer International Publishing, Cham, 2020)."},{"key":"83827_CR18","doi-asserted-by":"publisher","first-page":"166","DOI":"10.1136\/bmj.2.5964.166","volume":"2","author":"DW Golde","year":"1975","unstructured":"Golde, D.\u00a0W., Drew, W.\u00a0L., Klein, H.\u00a0Z. et al. Occult pulmonary haemorrhage in leukaemia. Br. Med. J. 2, 166\u2013168 (1975).","journal-title":"Br. Med. J."},{"key":"83827_CR19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-020-65958-2","volume":"10","author":"C Marzahl","year":"2020","unstructured":"Marzahl, C. et al. Deep learning-based quantification of pulmonary hemosiderophages in cytology slides. Sci. Rep. 10, 1\u201310 (2020).","journal-title":"Sci. Rep."},{"key":"83827_CR20","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-47335-w","author":"C Bergler","year":"2019","unstructured":"Bergler, C. et\u00a0al. ORCA-SPOT: An automatic killer whale sound detection toolkit using deep learning. Sci. Rep. https:\/\/doi.org\/10.1038\/s41598-019-47335-w. (2019).","journal-title":"Sci. Rep."},{"key":"83827_CR21","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1038\/nmeth.1896","volume":"9","author":"C Allan","year":"2012","unstructured":"Allan, C. et al. Omero: flexible, model-driven data management for experimental biology. Nat. Methods 9, 245\u2013253 (2012).","journal-title":"Nat. Methods"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-021-83827-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-021-83827-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-021-83827-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,3]],"date-time":"2022-12-03T04:00:33Z","timestamp":1670040033000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-021-83827-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,23]]},"references-count":21,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["83827"],"URL":"https:\/\/doi.org\/10.1038\/s41598-021-83827-4","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,23]]},"assertion":[{"value":"29 April 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 February 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 February 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"4343"}}