{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T03:16:48Z","timestamp":1777778208979,"version":"3.51.4"},"reference-count":44,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[2020,1,25]],"date-time":"2020-01-25T00:00:00Z","timestamp":1579910400000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.sagepub.com\/licence-information-for-chorus"}],"funder":[{"DOI":"10.13039\/100000145","name":"Division of Information and Intelligent Systems","doi-asserted-by":"publisher","award":["1704018"],"award-info":[{"award-number":["1704018"]}],"id":[{"id":"10.13039\/100000145","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Information Visualization"],"published-print":{"date-parts":[[2019,10]]},"abstract":"<jats:p>Data visualizations typically show a representation of a data set with little to no focus on the repeatability or generalizability of the displayed trends and patterns. However, insights gleaned from these visualizations are often used as the basis for decisions about future events. Visualizations of retrospective data therefore often serve as \u201cvisual predictive models.\u201d However, this visual predictive model approach can lead to invalid inferences. In this article, we describe an approach to visual model validation called Inline Replication. Inline Replication is closely related to the statistical techniques of bootstrap sampling and cross-validation and, like those methods, provides a non-parametric and broadly applicable technique for assessing the variance of findings from visualizations. This article describes the overall Inline Replication process and outlines how it can be integrated into both traditional and emerging \u201cbig data\u201d visualization pipelines. It also provides examples of how Inline Replication can be integrated into common visualization techniques such as bar charts and linear regression lines. Results from an empirical evaluation of the technique and two prototype Inline Replication\u2013based visual analysis systems are also described. The empirical evaluation demonstrates the impact of Inline Replication under different conditions, showing that both (1) the level of partitioning and (2) the approach to aggregation have a major influence over its behavior. The results highlight the trade-offs in choosing Inline Replication parameters but suggest that using [Formula: see text] partitions is a reasonable default.<\/jats:p>","DOI":"10.1177\/1473871618821747","type":"journal-article","created":{"date-parts":[[2019,1,25]],"date-time":"2019-01-25T04:36:18Z","timestamp":1548390978000},"page":"405-425","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":6,"title":["Visualization model validation via inline replication"],"prefix":"10.1177","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6424-7374","authenticated-orcid":false,"given":"David","family":"Gotz","sequence":"first","affiliation":[{"name":"School of Information and Library Science, The University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"}]},{"given":"Wenyuan","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information and Library Science, The University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"}]},{"given":"Annie T","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0162-4080","authenticated-orcid":false,"given":"David","family":"Borland","sequence":"additional","affiliation":[{"name":"RENCI, The University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"}]}],"member":"179","published-online":{"date-parts":[[2019,1,25]]},"reference":[{"key":"bibr1-1473871618821747","first-page":"336","volume-title":"Proceedings of IEEE symposium on visual languages","author":"Shneiderman B"},{"key":"bibr2-1473871618821747","volume-title":"Readings in information visualization: using vision to think","author":"Card S","year":"1999"},{"key":"bibr3-1473871618821747","first-page":"69","volume-title":"IEEE symposium on information visualization","author":"Chi EH"},{"key":"bibr4-1473871618821747","volume-title":"Statistics in research","author":"Ostle B.","year":"1963","edition":"2"},{"key":"bibr5-1473871618821747","unstructured":"Perer A, Bertini E, Maciejewski R, et al. An IEEE VIS 2014 workshop on visualization for predictive analytics, 2014, http:\/\/predictive-workshop.github.io\/"},{"key":"bibr6-1473871618821747","unstructured":"Gotz D, Price BA, Chen AT. Visual model validation via inline replication. arXiv:1605.08749, 2016, http:\/\/arxiv.org\/abs\/1605.08749"},{"key":"bibr7-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346682"},{"key":"bibr8-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1057\/ivs.2009.1"},{"key":"bibr9-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-32677-6_15"},{"key":"bibr10-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.279"},{"key":"bibr11-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2009.114"},{"key":"bibr12-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.247"},{"key":"bibr13-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2013.14"},{"key":"bibr14-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4471-2804-5_6"},{"key":"bibr15-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2018.2874782"},{"key":"bibr16-1473871618821747","first-page":"85","volume-title":"Proceedings of the 21st international conference on intelligent user interfaces","author":"Gotz D"},{"issue":"4","key":"bibr17-1473871618821747","volume":"7","author":"Gotz D","year":"2017","journal-title":"ACM T Interact Intell Syst"},{"key":"bibr18-1473871618821747","first-page":"151","volume-title":"IEEE conference on visual analytics science and technology (VAST)","author":"van den Elzen S"},{"key":"bibr19-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.125"},{"key":"bibr20-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.138"},{"key":"bibr21-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1093\/toxsci\/kfr220"},{"key":"bibr22-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2012.48"},{"key":"bibr23-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346574"},{"key":"bibr24-1473871618821747","first-page":"1673","volume-title":"Proceedings of the SIGCHI conference on human factors in computing systems (CHI \u201912)","author":"Fisher D"},{"key":"bibr25-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1201\/9781420036268"},{"key":"bibr26-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1080\/00031305.1981.10479331"},{"key":"bibr27-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1097\/00001648-199001000-00010"},{"key":"bibr28-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2015.2410278"},{"key":"bibr29-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2010.176"},{"key":"bibr30-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.285"},{"key":"bibr31-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.2013.808157"},{"key":"bibr32-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2010.161"},{"key":"bibr33-1473871618821747","first-page":"51","volume-title":"IEEE symposium on visual analytics science and technology","author":"Correa C"},{"key":"bibr34-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1201\/b17511"},{"key":"bibr35-1473871618821747","volume-title":"Envisioning information","author":"Tufte ER.","year":"1990"},{"key":"bibr36-1473871618821747","volume-title":"Information visualization: perception for design (interactive technologies)","author":"Ware C.","year":"2004"},{"key":"bibr37-1473871618821747","unstructured":"Cammegh Limited. Cammegh\u2014the world\u2019s finest roulette wheel, 2015, http:\/\/www.cammegh.com\/product.php?product=displays"},{"key":"bibr38-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1177\/1473871612445832"},{"key":"bibr39-1473871618821747","volume-title":"Proceedings of the CHI conference on human factors in computing systems","author":"Zgraggen E"},{"key":"bibr40-1473871618821747","first-page":"1137","volume-title":"Proceedings of the 14th international joint conference on artificial intelligence (IJCAI\u201995)","volume":"2","author":"Kohavi R"},{"key":"bibr41-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176344552"},{"key":"bibr42-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1109\/3468.618255"},{"key":"bibr43-1473871618821747","doi-asserted-by":"publisher","DOI":"10.1161\/01.CIR.101.23.e215"},{"key":"bibr44-1473871618821747","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-130-12-199906150-00008"}],"container-title":["Information Visualization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1473871618821747","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/1473871618821747","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1473871618821747","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1473871618821747","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T19:18:57Z","timestamp":1777490337000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1473871618821747"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,25]]},"references-count":44,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,10]]}},"alternative-id":["10.1177\/1473871618821747"],"URL":"https:\/\/doi.org\/10.1177\/1473871618821747","relation":{},"ISSN":["1473-8716","1473-8724"],"issn-type":[{"value":"1473-8716","type":"print"},{"value":"1473-8724","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,25]]}}}