{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T17:32:08Z","timestamp":1785605528970,"version":"3.56.0"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,9,22]],"date-time":"2020-09-22T00:00:00Z","timestamp":1600732800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,9,22]],"date-time":"2020-09-22T00:00:00Z","timestamp":1600732800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007751","name":"Akademia G\u00f3rniczo-Hutnicza im. Stanislawa Staszica","doi-asserted-by":"publisher","award":["16.16.120.773"],"award-info":[{"award-number":["16.16.120.773"]}],"id":[{"id":"10.13039\/501100007751","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Ultragen Rafa\u0142 Obuchowicz","award":["n.a."],"award-info":[{"award-number":["n.a."]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Imaging"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n<jats:title>Background<\/jats:title>\n<jats:p>The perceptual quality of magnetic resonance (MR) images influences diagnosis and may compromise the treatment. The purpose of this study was to evaluate how the image quality changes influence the interobserver variability of their assessment.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Methods<\/jats:title>\n<jats:p>For the variability evaluation, a dataset containing distorted MRI images was prepared and then assessed by 31 experienced medical professionals (radiologists). Differences between observers were analyzed using the Fleiss\u2019 kappa. However, since the kappa evaluates the agreement among radiologists taking into account aggregated decisions, a typically employed criterion of the image quality assessment (IQA) performance was used to provide a more thorough analysis. The IQA performance of radiologists was evaluated by comparing the Spearman correlation coefficients, <jats:italic>\u03c1<\/jats:italic>, between individual scores with the mean opinion scores (MOS) composed of the subjective opinions of the remaining professionals.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Results<\/jats:title>\n<jats:p>The experiments show that there is a significant agreement among radiologists (<jats:italic>\u03ba<\/jats:italic>=0.12; 95% confidence interval [CI]: 0.118, 0.121; <jats:italic>P<\/jats:italic>&lt;0.001) on the quality of the assessed images. The resulted <jats:italic>\u03ba<\/jats:italic> is strongly affected by the subjectivity of the assigned scores, separately presenting close scores. Therefore, the <jats:italic>\u03c1<\/jats:italic> was used to identify poor performance cases and to confirm the consistency of the majority of collected scores (<jats:italic>\u03c1<\/jats:italic><jats:sub><jats:italic>mean<\/jats:italic><\/jats:sub> = 0.5706). The results for interns (<jats:italic>\u03c1<\/jats:italic><jats:sub><jats:italic>mean<\/jats:italic><\/jats:sub> = 0.6868) supports the finding that the quality assessment of MR images can be successfully taught.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Conclusions<\/jats:title>\n<jats:p>The agreement observed among radiologists from different imaging centers confirms the subjectivity of the perception of MR images. It was shown that the image content and severity of distortions affect the IQA. Furthermore, the study highlights the importance of the psychosomatic condition of the observers and their attitude.<\/jats:p>\n<\/jats:sec>","DOI":"10.1186\/s12880-020-00505-z","type":"journal-article","created":{"date-parts":[[2020,10,2]],"date-time":"2020-10-02T15:34:28Z","timestamp":1601652868000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Interobserver variability in quality assessment of magnetic resonance images"],"prefix":"10.1186","volume":"20","author":[{"given":"Rafal","family":"Obuchowicz","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mariusz","family":"Oszust","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4773-5322","authenticated-orcid":false,"given":"Adam","family":"Piorkowski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,9,22]]},"reference":[{"issue":"3","key":"505_CR1","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1364\/JOSAA.18.000473","volume":"18","author":"CK Abbey","year":"2001","unstructured":"Abbey CK, Barrett HH. Human- and model-observer performance in ramp-spectrum noise: effects of regularization and object variability. J Opt Soc Am A. 2001; 18(3):473\u201388. https:\/\/doi.org\/10.1364\/JOSAA.18.000473.","journal-title":"J Opt Soc Am A"},{"issue":"5","key":"505_CR2","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1364\/JOSAA.12.000834","volume":"12","author":"HH Barrett","year":"1995","unstructured":"Barrett HH, Denny JL, Wagner RF, Myers KJ. Objective assessment of image quality. ii. fisher information, fourier crosstalk, and figures of merit for task performance. J Opt Soc Am A. 1995; 12(5):834\u201352. https:\/\/doi.org\/10.1364\/JOSAA.12.000834.","journal-title":"J Opt Soc Am A"},{"issue":"3","key":"505_CR3","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1148\/radiol.2019181343","volume":"292","author":"M Buda","year":"2019","unstructured":"Buda M, Wildman-Tobriner B, Hoang JK, Thayer D, Tessler FN, Middleton WD, Mazurowski MA. Management of thyroid nodules seen on us images: Deep learning may match performance of radiologists. Radiology. 2019; 292(3):695\u2013701.","journal-title":"Radiology"},{"key":"505_CR4","doi-asserted-by":"publisher","unstructured":"Chaddha N, Meng T. Psycho-visual based distortion measures for monochrome image and video compression. 1993:841\u20135. https:\/\/doi.org\/10.1109\/ACSSC.1993.342451.","DOI":"10.1109\/ACSSC.1993.342451"},{"key":"505_CR5","doi-asserted-by":"publisher","unstructured":"Chalavi S, Simmons A, Dijkstra H, Barker GJ, Reinders AA. Quantitative and qualitative assessment of structural magnetic resonance imaging data in a two-center study. BMC Med Imaging. 2012; 12(27). https:\/\/doi.org\/10.1186\/1471-2342-12-27.","DOI":"10.1186\/1471-2342-12-27"},{"key":"505_CR6","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1117\/12.480331","volume":"5034","author":"L Chen","year":"2003","unstructured":"Chen L, Barrett H. Optimizing lens-coupled digital radiographic imaging systems based on model observers\u2019 performance. Proc SPIE Int Soc Opt Eng. 2003; 5034:63\u201370. https:\/\/doi.org\/10.1117\/12.480331.","journal-title":"Proc SPIE Int Soc Opt Eng"},{"issue":"11","key":"505_CR7","doi-asserted-by":"publisher","first-page":"1783","DOI":"10.1364\/AO.39.001783","volume":"39","author":"E Clarkson","year":"2000","unstructured":"Clarkson E, Barrett HH. Approximations to ideal-observer performance on signal-detection tasks. Appl Opt. 2000; 39(11):1783\u201393. https:\/\/doi.org\/10.1364\/AO.39.001783.","journal-title":"Appl Opt"},{"key":"505_CR8","doi-asserted-by":"publisher","unstructured":"Daly SJ. Visible differences predictor: an algorithm for the assessment of image fidelity. 1992. https:\/\/doi.org\/10.1117\/12.135952.","DOI":"10.1117\/12.135952"},{"key":"505_CR9","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1006\/nimg.2000.0601","volume":"12","author":"R Deichmann","year":"2000","unstructured":"Deichmann R, Good C, Josephs O, Ashburner J, Turner R. Optimization of 3d mp-rage sequence for structural brain imaging. NeuroImage. 2000; 12:112\u201327. https:\/\/doi.org\/10.1006\/nimg.2000.0601.","journal-title":"NeuroImage"},{"issue":"1","key":"505_CR10","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1002\/jmri.23639","volume":"36","author":"A Deshmane","year":"2012","unstructured":"Deshmane A, Gulani V, Griswold MA, Seiberlich N. Parallel MR imaging. J Magn Reson Imaging. 2012; 36(1):55\u201372. https:\/\/doi.org\/10.1002\/jmri.23639.","journal-title":"J Magn Reson Imaging"},{"issue":"10","key":"505_CR11","doi-asserted-by":"publisher","first-page":"6098","DOI":"10.1118\/1.4929973","volume":"42","author":"BL Eck","year":"2015","unstructured":"Eck BL, Fahmi R, Brown KM, Zabic S, Raihani N, Miao J, Wilson DL. Computational and human observer image quality evaluation of low dose, knowledge-based ct iterative reconstruction. Med Phys. 2015; 42(10):6098\u2013111. https:\/\/doi.org\/10.1118\/1.4929973.","journal-title":"Med Phys"},{"issue":"2","key":"505_CR12","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1212\/WNL.49.2.371","volume":"49","author":"M Filippi","year":"1997","unstructured":"Filippi M, van Waesberghe JH, Horsfield MA, Bressi S, Gasperini C, Yousry TA, Gawne-Cain ML, Morrissey SP, Rocca MA, Barkhof F, Lycklama \u00e0 Nijeholt GJ, Bastianello S, Miller DH. Interscanner variation in brain mri lesion load measurements in ms: Implications for clinical trials. Neurology. 1997; 49(2):371\u20137. https:\/\/doi.org\/10.1212\/WNL.49.2.371.","journal-title":"Neurology"},{"issue":"5","key":"505_CR13","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1037\/h0031619","volume":"76","author":"JL Fleiss","year":"1971","unstructured":"Fleiss JL. Measuring nominal scale agreement among many raters. Psychological bulletin. 1971; 76(5):378.","journal-title":"Psychological bulletin"},{"issue":"1","key":"505_CR14","doi-asserted-by":"publisher","first-page":"372","DOI":"10.1109\/TIP.2015.2500021","volume":"25","author":"D Ghadiyaram","year":"2016","unstructured":"Ghadiyaram D, Bovik AC. Massive online crowdsourced study of subjective and objective picture quality. IEEE T. Image Process. 2016; 25(1):372\u2013387. https:\/\/doi.org\/10.1109\/TIP.2015.2500021.","journal-title":"IEEE T. Image Process"},{"issue":"2","key":"505_CR15","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/j.mri.2005.10.018","volume":"24","author":"D Huo","year":"2006","unstructured":"Huo D, Xu D, Liang ZP, Wilson D. Application of perceptual difference model on regularization techniques of parallel mr imaging. Magn Reson Imaging. 2006; 24(2):123\u201332. https:\/\/doi.org\/10.1016\/j.mri.2005.10.018.","journal-title":"Magn Reson Imaging"},{"key":"505_CR16","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1016\/j.neuroimage.2009.02.010","volume":"46","author":"J Jovicich","year":"2009","unstructured":"Jovicich J, Czanner S, Han X, Salat D, van der Kouwe A, Quinn B, Pacheco J, Albert M, Killiany R, Blacker D, Maguire R, Rosas H, Makris N, Gollub R, Dale A, Dickerson B, Fischl B. Mri-derived measurements of human subcortical, ventricular and intracranial brain volumes: Reliability effects of scan sessions, acquisition sequences, data analyses, scanner upgrade, scanner vendors and field strengths. NeuroImage. 2009; 46:177\u201392. https:\/\/doi.org\/10.1016\/j.neuroimage.2009.02.010.","journal-title":"NeuroImage"},{"issue":"1","key":"505_CR17","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1148\/rg.331125096","volume":"33","author":"GC Kagadis","year":"2013","unstructured":"Kagadis GC, Walz-Flannigan A, Krupinski EA, Nagy PG, Katsanos K, Diamantopoulos A, Langer SG. Medical imaging displays and their use in image interpretation. Radiographics. 2013; 33(1):275\u201390.","journal-title":"Radiographics"},{"key":"505_CR18","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1007\/s002619910004","volume":"25","author":"M Kato","year":"2000","unstructured":"Kato M, Saji S, Kanematsu M, Fukada D, Miya K, Umemoto T, Kunieda K, Sugiyama Y, Takao H, Kawaguchi Y, Takagi Y, Kondo H, Hoshi H. Detection of lymph-node metastases in patients with gastric carcinoma: Comparison of three mr imaging pulse sequences. Abdom Imaging. 2000; 25:25\u20139. https:\/\/doi.org\/10.1007\/s002619910004.","journal-title":"Abdom Imaging"},{"key":"505_CR19","doi-asserted-by":"publisher","first-page":"2123","DOI":"10.1016\/j.neuroimage.2009.11.006","volume":"49","author":"F Kruggel","year":"2009","unstructured":"Kruggel F, Turner J, Tugan Muftuler L. Impact of scanner hardware and imaging protocol on image quality and compartment volume precision in the adni cohort. NeuroImage. 2009; 49:2123\u201333. https:\/\/doi.org\/10.1016\/j.neuroimage.2009.11.006.","journal-title":"NeuroImage"},{"key":"505_CR20","doi-asserted-by":"publisher","first-page":"93","DOI":"10.12659\/PJR.892628","volume":"80","author":"K Krupa","year":"2015","unstructured":"Krupa K, Bekiesinska-Figatowska M. Artifacts in magnetic resonance imaging. Pol J Radiol. 2015; 80:93\u2013106. https:\/\/doi.org\/10.12659\/PJR.892628.","journal-title":"Pol J Radiol"},{"key":"505_CR21","doi-asserted-by":"publisher","first-page":"1205","DOI":"10.3758\/APP.72.5.1205","volume":"72","author":"EA Krupinski","year":"2010","unstructured":"Krupinski EA. Current perspectives in medical image perception. Atten Percept Psychophys. 2010; 72:1205\u201317.","journal-title":"Atten Percept Psychophys"},{"key":"505_CR22","doi-asserted-by":"publisher","unstructured":"Kundel HL. Reader error, object recognition, and visual search; 2004. https:\/\/doi.org\/10.1117\/12.542717.","DOI":"10.1117\/12.542717"},{"key":"505_CR23","doi-asserted-by":"publisher","first-page":"453","DOI":"10.1148\/radiol.2222010276","volume":"222","author":"K Ludwig","year":"2002","unstructured":"Ludwig K, Lenzen H, Kamm KF, Link T, Diederich S, Wormanns D, Heindel W. Performance of a flat-panel detector in detecting artificial bone lesions: Comparison with conventional screen-film and storage-phosphor radiography1. Radiology. 2002; 222:453\u20139.","journal-title":"Radiology"},{"issue":"915","key":"505_CR24","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1259\/bjr\/28883951","volume":"77","author":"DJ Manning","year":"2004","unstructured":"Manning DJ, Ethell SC, Donovan T. Detection or decision errors? missed lung cancer from the posteroanterior chest radiograph. Br J Radiol. 2004; 77(915):231\u20135. https:\/\/doi.org\/10.1259\/bjr\/28883951. PMID: 15020365.","journal-title":"Br J Radiol"},{"issue":"932","key":"505_CR25","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1259\/bjr\/72087985","volume":"78","author":"DJ Manning","year":"2005","unstructured":"Manning DJ, Gale A, Krupinski EA. Perception research in medical imaging. Br J Radiol. 2005; 78(932):683\u20135. https:\/\/doi.org\/10.1259\/bjr\/72087985.","journal-title":"Br J Radiol"},{"key":"505_CR26","volume-title":"version R2018b","author":"MATLAB","year":"2018","unstructured":"MATLAB. version R2018b. Natick, Massachusetts: The MathWorks Inc.; 2018."},{"issue":"4","key":"505_CR27","doi-asserted-by":"publisher","first-page":"596","DOI":"10.1016\/j.mri.2012.09.009","volume":"31","author":"J Miao","year":"2013","unstructured":"Miao J, Huang F, Narayan S, Wilson DL. A new perceptual difference model for diagnostically relevant quantitative image quality evaluation: A preliminary study. Magn Reson Imaging. 2013; 31(4):596\u2013603. https:\/\/doi.org\/10.1016\/j.mri.2012.09.009.","journal-title":"Magn Reson Imaging"},{"issue":"6Part1","key":"505_CR28","doi-asserted-by":"publisher","first-page":"2541","DOI":"10.1118\/1.2903207","volume":"35","author":"J Miao","year":"2008","unstructured":"Miao J, Huo D, Wilson DL. Quantitative image quality evaluation of mr images using perceptual difference models. Med Phys. 2008; 35(6Part1):2541\u201353. https:\/\/doi.org\/10.1118\/1.2903207.","journal-title":"Med Phys"},{"key":"505_CR29","doi-asserted-by":"publisher","first-page":"334","DOI":"10.1016\/j.ins.2019.01.034","volume":"482","author":"M Oszust","year":"2019","unstructured":"Oszust M. No-reference quality assessment of noisy images with local features and visual saliency models. Inf Sci. 2019; 482:334\u201349. https:\/\/doi.org\/10.1016\/j.ins.2019.01.034.","journal-title":"Inf Sci"},{"key":"505_CR30","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1016\/j.ejro.2019.03.002","volume":"6","author":"Z Pang","year":"2019","unstructured":"Pang Z, Margolis M, Menezes RJ, Maan H, Ghai S. Diagnostic performance of 2015 American thyroid association guidelines and inter-observer variability in assigning risk category. Eur J Radiol Open. 2019; 6:122\u20137. https:\/\/doi.org\/10.1016\/j.ejro.2019.03.002.","journal-title":"Eur J Radiol Open"},{"key":"505_CR31","volume-title":"Computational Color Imaging","author":"M Pedersen","year":"2009","unstructured":"Pedersen M, Hardeberg JY. A new spatial hue angle metric for perceptual image difference In: Tr\u00e9meau A, Schettini R, Tominaga S, editors. Computational Color Imaging. Berlin: Springer Berlin Heidelberg: 2009. p. 81\u201390."},{"key":"505_CR32","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1016\/j.jacr.2003.12.011","volume":"1","author":"A R Margulis","year":"2004","unstructured":"R Margulis A, Dirk Sostman H. Radiologist-patient contact during the performance of cross-sectional examinations. J Am Coll Radiol JACR. 2004; 1:162\u20133. https:\/\/doi.org\/10.1016\/j.jacr.2003.12.011.","journal-title":"J Am Coll Radiol JACR"},{"issue":"1","key":"505_CR33","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1148\/radiol.12120674","volume":"266","author":"EA Rafferty","year":"2013","unstructured":"Rafferty EA, Park JM, Philpotts LE, Poplack SP, Sumkin JH, Halpern EF, Niklason LT. Assessing radiologist performance using combined digital mammography and breast tomosynthesis compared with digital mammography alone: Results of a multicenter, multireader trial. Radiology. 2013; 266(1):104\u201313. https:\/\/doi.org\/10.1148\/radiol.12120674.","journal-title":"Radiology"},{"key":"505_CR34","doi-asserted-by":"publisher","unstructured":"Rajashekar U, Wang Z, Simoncelli1 EP. Quantifying color image distortions based on adaptive spatio-chromatic signal decompositions. In: 2009 16th IEEE International Conference on Image Processing (ICIP): 2009. p. 2213\u20136. https:\/\/doi.org\/10.1109\/ICIP.2009.5413889.","DOI":"10.1109\/ICIP.2009.5413889"},{"issue":"2","key":"505_CR35","doi-asserted-by":"publisher","first-page":"224","DOI":"10.1117\/1.1453412","volume":"11","author":"KA Salem","year":"2002","unstructured":"Salem KA, Lewin JS, Aschoff AJ, Duerk JL, Wilson DL. Validation of a human vision model for image quality evaluation of fast interventional magnetic resonance imaging. J Electron Imaging. 2002; 11(2):224\u2013235\u201312. https:\/\/doi.org\/10.1117\/1.1453412.","journal-title":"J Electron Imaging"},{"issue":"2","key":"505_CR36","doi-asserted-by":"publisher","first-page":"430","DOI":"10.1109\/TIP.2005.859378","volume":"15","author":"HR Sheikh","year":"2006","unstructured":"Sheikh HR, Bovik AC. Image information and visual quality. IEEE T. Image Process. 2006; 15(2):430\u201344. https:\/\/doi.org\/10.1109\/TIP.2005.859378.","journal-title":"IEEE T. Image Process"},{"issue":"11","key":"505_CR37","doi-asserted-by":"publisher","first-page":"3440","DOI":"10.1109\/TIP.2006.881959","volume":"15","author":"HR Sheikh","year":"2006","unstructured":"Sheikh HR, Sabir MF, Bovik AC. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE T. Image Process. 2006; 15(11):3440\u201351. https:\/\/doi.org\/10.1109\/tip.2006.881959.","journal-title":"IEEE T. Image Process"},{"issue":"2","key":"505_CR38","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1615\/CritRevBiomedEng.v38.i2.20","volume":"38","author":"N Sinha","year":"2010","unstructured":"Sinha N, Ramakrishnan A. Quality assessment in magnetic resonance images. Crit Rev Trade Biomed Eng. 2010; 38(2):127\u201341. https:\/\/doi.org\/10.1615\/CritRevBiomedEng.v38.i2.20.","journal-title":"Crit Rev Trade Biomed Eng"},{"issue":"1082","key":"505_CR39","first-page":"20170,611","volume":"91","author":"RJI Sweeney","year":"2018","unstructured":"Sweeney RJI, Lewis SJ, Hogg P, McEntee MF. A review of mammographic positioning image quality criteria for the craniocaudal projection. Br J Radiol. 2018; 91(1082):20170,611.","journal-title":"Br J Radiol"},{"key":"505_CR40","unstructured":"Video Quality ExpertsGroup. Final report from the video quality experts group on the validation of objective models of video quality assessment, phase ii (fr_tv2). 2003. https:\/\/www.itu.int\/ITU-T\/studygroups\/com09\/docs\/tutorial_opavc.pdf, Accessed 07 June 2020."},{"key":"505_CR41","doi-asserted-by":"publisher","unstructured":"Wang Z, Bovik AC, Lu L. Why is image quality assessment so difficult? In: 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 4: 2002. p. IV\u20133313\u2013IV\u20133316. https:\/\/doi.org\/10.1109\/ICASSP.2002.5745362.","DOI":"10.1109\/ICASSP.2002.5745362"},{"issue":"4","key":"505_CR42","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/TIP.2003.819861","volume":"13","author":"Z Wang","year":"2004","unstructured":"Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: From error visibility to structural similarity. IEEE T Image Process. 2004; 13(4):600\u201312. https:\/\/doi.org\/10.1109\/tip.2003.819861.","journal-title":"IEEE T Image Process"},{"key":"505_CR43","doi-asserted-by":"publisher","unstructured":"Williams MC, Golay SK, Hunter A, Weir-McCall JR, Mlynska L, Dweck MR, Uren NG, Reid JH, Lewis SC, Berry C, van Beek EJR, Roditi G, Newby DE, Mirsadraee S. Observer variability in the assessment of ct coronary angiography and coronary artery calcium score: substudy of the scottish computed tomography of the heart (scot-heart) trial. Open Heart. 2015; 2(1). https:\/\/doi.org\/10.1136\/openhrt-2014-000234.","DOI":"10.1136\/openhrt-2014-000234"},{"issue":"2","key":"505_CR44","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1109\/21.3468","volume":"18","author":"R Wilson","year":"1988","unstructured":"Wilson R, Knutsson H. Uncertainty and inference in the visual system. IEEE Trans Syst Man Cybern. 1988; 18(2):305\u201312. https:\/\/doi.org\/10.1109\/21.3468.","journal-title":"IEEE Trans Syst Man Cybern"},{"key":"505_CR45","doi-asserted-by":"publisher","first-page":"1324","DOI":"10.1016\/j.neuroimage.2008.10.037","volume":"44","author":"J Wonderlick","year":"2008","unstructured":"Wonderlick J, Ziegler D, Hosseini-Varnamkhasti P, Locascio J, Bakkour A, van der Kouwe A, Triantafyllou C, Corkin S, Dickerson B. Reliability of mri-derived cortical and subcortical morphometric measures: Effects of pulse sequence, voxel geometry, and parallel imaging. NeuroImage. 2008; 44:1324\u201333. https:\/\/doi.org\/10.1016\/j.neuroimage.2008.10.037.","journal-title":"NeuroImage"}],"container-title":["BMC Medical Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12880-020-00505-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12880-020-00505-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12880-020-00505-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,21]],"date-time":"2021-09-21T23:14:07Z","timestamp":1632266047000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedimaging.biomedcentral.com\/articles\/10.1186\/s12880-020-00505-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,22]]},"references-count":45,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["505"],"URL":"https:\/\/doi.org\/10.1186\/s12880-020-00505-z","relation":{},"ISSN":["1471-2342"],"issn-type":[{"value":"1471-2342","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,22]]},"assertion":[{"value":"13 April 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 September 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 September 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Ethical approval: The study protocol was designed according to the guidelines of the Declaration of Helsinki and the Good Clinical Practice standard. Written acceptance for conducting the study was obtained from the Ethics Committee of Jagiellonian University (no. 1072.6120.15.2017).Ethical Statement: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committee in accordance to 1964 Helsinki declaration and its latter amandements.Consent to participate: Written consent to participate in the study was obtained from all participants. Furthermore, written consent for publication of anonymised data was obtained from all participants.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not Applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"109"}}