{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T15:46:41Z","timestamp":1753890401833,"version":"3.41.2"},"reference-count":67,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,5,25]],"date-time":"2023-05-25T00:00:00Z","timestamp":1684972800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Bioinform."],"abstract":"<jats:p>We present a general purpose visual analysis system that can be used for exploring parameters of a variety of computer models. Our proposed system offers key components of a visual parameter analysis framework including parameter sampling, deriving output summaries, and an exploration interface. It also provides an API for rapid development of parameter space exploration solutions as well as the flexibility to support custom workflows for different application domains. We evaluate the effectiveness of our system by demonstrating it in three domains: data mining, machine learning and specific application in bioinformatics.<\/jats:p>","DOI":"10.3389\/fbinf.2023.1153800","type":"journal-article","created":{"date-parts":[[2023,5,25]],"date-time":"2023-05-25T10:45:49Z","timestamp":1685011549000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["ModEx: a general purpose computer model exploration system"],"prefix":"10.3389","volume":"3","author":[{"given":"Hamid","family":"Younesy","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph","family":"Pober","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Torsten","family":"M\u00f6ller","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohammad M.","family":"Karimi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,5,25]]},"reference":[{"key":"B1","first-page":"265","article-title":"TensorFlow: A system for large-scale machine learning","volume-title":"12th {USENIX} symposium on operating systems design and implementation","author":"Abadi","year":"2016"},{"key":"B2","first-page":"191","article-title":"Visual analytics decision support environment for epidemic modeling and response evaluation","author":"Afzal","year":"2011"},{"key":"B3","doi-asserted-by":"publisher","first-page":"R106","DOI":"10.1186\/gb-2010-11-10-r106","article-title":"Differential expression analysis for sequence count data","volume":"11","author":"Anders","year":"2010","journal-title":"Genome Biol."},{"key":"B4","doi-asserted-by":"publisher","first-page":"1765","DOI":"10.1038\/nprot.2013.099","article-title":"Count-based differential expression analysis of RNA sequencing data using R and Bioconductor","volume":"8","author":"Anders","year":"2013","journal-title":"Nat. Protoc."},{"key":"B5","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1186\/1471-2105-11-121","article-title":"Mayday - integrative analytics for expression data","volume":"11","author":"Battke","year":"2010","journal-title":"BMC Bioinforma."},{"key":"B6","doi-asserted-by":"publisher","first-page":"1499","DOI":"10.1109\/tvcg.2013.61","article-title":"ParaGlide: Interactive parameter space partitioning for computer simulations","volume":"19","author":"Bergner","year":"2013","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B7","doi-asserted-by":"publisher","first-page":"1235","DOI":"10.1111\/j.1467-8659.2012.03116.x","article-title":"Vismon: Facilitating analysis of trade-offs, uncertainty, and sensitivity in fisheries management decision making","volume":"31","author":"Booshehrian","year":"2012","journal-title":"Comput. Graph. Forum"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1121","DOI":"10.1109\/tvcg.2009.174","article-title":"Protovis: A graphical toolkit for visualization","volume":"15","author":"Bostock","year":"2009","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B9","doi-asserted-by":"publisher","first-page":"2301","DOI":"10.1109\/tvcg.2011.185","article-title":"D3 data-driven documents","volume":"17","author":"Bostock","year":"2011","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B10","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1177\/1473871615571951","article-title":"Interactive and iterative visual clustering","volume":"15","author":"Boudjeloud-Assala","year":"2016","journal-title":"Inf. Vis."},{"key":"B11","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1016\/j.neucom.2014.09.062","article-title":"Cluster Sculptor, an interactive visual clustering system","volume":"150","author":"Bruneau","year":"2015","journal-title":"Neurocomputing"},{"key":"B12","doi-asserted-by":"publisher","first-page":"2581","DOI":"10.1109\/tvcg.2011.188","article-title":"Dicon: Interactive visual analysis of multidimensional clusters","volume":"17","author":"Cao","year":"2011","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v061.i06","article-title":"NbClust: An R package for determining the relevant number of clusters in a data set","volume":"61","author":"Charrad","year":"2014","journal-title":"J. Stat. Softw."},{"key":"B14","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1057\/palgrave.ivs.9500076","article-title":"Vista: Validating and refining clusters via visualization","volume":"3","author":"Chen","year":"2004","journal-title":"Inf. Vis."},{"key":"B15","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1145\/1216295.1216367","article-title":"Interactive visual clustering","author":"desJardins","year":"2007","journal-title":"Proc. 12th Int. Conf. Intelligent User Interfaces"},{"key":"B16","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1111\/cgf.12895","article-title":"GEMSe: Visualization-guided exploration of multi-channel segmentation algorithms","volume":"35","author":"Fr\u00f6hler","year":"2016","journal-title":"Comput. Graph. Forum"},{"volume-title":"Deep Learning","year":"2016","author":"Goodfellow","key":"B17"},{"key":"B18","unstructured":"Neural network evolution playground with backprop NEAT\n            HaD.\n          2016"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v091.i01","article-title":"dbscan: Density based clustering of applications with noise (DBSCAN) and related algorithms","volume":"1","author":"Hahsler","year":"2018","journal-title":"R. package version"},{"key":"B20","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1109\/mcg.2014.2","article-title":"Visual exploration of parameter influence on phylogenetic trees","volume":"34","author":"Hess","year":"2014","journal-title":"IEEE Comput. Graph. Appl."},{"key":"B21","unstructured":"A statistical analysis of the work of Bob Ross\n            HickeyW.\n          2014"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2843369","article-title":"Visual analytics in deep learning: An interrogative survey for the next frontiers","author":"Hohman","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B23","unstructured":"RNA-seq differential expression analysis\n            HussM.\n          2014"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v011.i09","article-title":"Kernlab \u2013 an S4 package for kernel methods in R","volume":"11","author":"Karatzoglou","year":"2004","journal-title":"J. Stat. Softw."},{"key":"B25","doi-asserted-by":"publisher","first-page":"406","DOI":"10.1186\/s12859-017-1813-7","article-title":"Interactive visual exploration and refinement of cluster assignments","volume":"18","author":"Kern","year":"2017","journal-title":"BMC Bioinforma."},{"key":"B26","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1109\/tvcg.2017.2745085","article-title":"Clustervision: Visual supervision of unsupervised clustering","volume":"24","author":"Kwon","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1155","DOI":"10.1111\/j.1467-8659.2012.03108.x","article-title":"iVisClustering: An interactive visual document clustering via topic modeling","volume":"31","author":"Lee","year":"2012","journal-title":"Comput. Graph. Forum."},{"key":"B28","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1016\/j.visinf.2017.01.006","article-title":"Towards better analysis of machine learning models: A visual analytics perspective","volume":"1","author":"Liu","year":"2017","journal-title":"Vis. Inf."},{"key":"B29","doi-asserted-by":"publisher","first-page":"550","DOI":"10.1186\/s13059-014-0550-8","article-title":"Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2","volume":"15","author":"Love","year":"2014","journal-title":"Genome Biol."},{"key":"B30","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/j.cag.2013.09.004","article-title":"Supporting the integrated visual analysis of input parameters and simulation trajectories","volume":"39","author":"Luboschik","year":"2014","journal-title":"Comput. Graph."},{"key":"B31","doi-asserted-by":"publisher","first-page":"S5","DOI":"10.1186\/1471-2105-16-s11-s5","article-title":"XCluSim: A visual analytics tool for interactively comparing multiple clustering results of bioinformatics data","volume":"16","author":"L\u2019Yi","year":"2015","journal-title":"BMC Bioinforma."},{"key":"B32","article-title":"Clusterix: A visual analytics approach to clustering","volume-title":"Symposium on visualization in data science at IEEE VIS","author":"Maguire","year":"2016"},{"key":"B33","first-page":"4073","article-title":"iVoLVER: Interactive visual language for visualization extraction and reconstruction","author":"M\u00e9ndez","year":"2016"},{"key":"B34","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1109\/tvcg.2017.2745158","article-title":"TreePOD: Sensitivity-aware selection of Pareto-optimal decision trees","volume":"24","author":"M\u00fchlbacher","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B35","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/j.cag.2014.02.004","article-title":"Interactive exploration of parameter space in data mining: Comprehending the predictive quality of large decision tree collections","volume":"41","author":"Padua","year":"2014","journal-title":"Comput. Graph."},{"key":"B36","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1109\/tvcg.2016.2598589","article-title":"WeightLifter: Visual weight space exploration for multi-criteria decision making","volume":"23","author":"Pajer","year":"2017","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B37","first-page":"233","article-title":"Ensemble-vis: A framework for the statistical visualization of ensemble data","volume":"2009","author":"Potter","year":"2009","journal-title":"ICDM Work."},{"key":"B38","doi-asserted-by":"publisher","first-page":"S9","DOI":"10.1186\/1471-2105-16-s11-s9","article-title":"Visual parameter optimisation for biomedical image processing","volume":"16","author":"Pretorius","year":"2015","journal-title":"BMC Bioinforma."},{"volume-title":"R: A language and environment for statistical computing","year":"","key":"B39"},{"key":"B40","unstructured":"Rdocumentation: Invoke a system command"},{"key":"B41","doi-asserted-by":"publisher","first-page":"R95","DOI":"10.1186\/gb-2013-14-9-r95","article-title":"Comprehensive evaluation of differential gene expression analysis methods for RNA-seq data","volume":"14","author":"Rapaport","year":"2013","journal-title":"Genome Biol."},{"key":"B42","doi-asserted-by":"publisher","first-page":"2092","DOI":"10.1109\/tvcg.2014.2346291","article-title":"iVisDesigner: Expressive interactive design of information visualizations","volume":"20","author":"Ren","year":"2014","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B43","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1093\/bioinformatics\/btp616","article-title":"edgeR: a Bioconductor package for differential expression analysis of digital gene expression data","volume":"26","author":"Robinson","year":"2010","journal-title":"Bioinformatics"},{"key":"B44","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1111\/cgf.12391","article-title":"Lyra: An interactive visualization design environment","volume":"33","author":"Satyanarayan","year":"2014","journal-title":"Comput. Graph. Forum."},{"key":"B45","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1109\/tvcg.2016.2599030","article-title":"Vega-lite: A grammar of interactive graphics","volume":"23","author":"Satyanarayan","year":"2017","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B46","first-page":"669","article-title":"Declarative interaction design for data visualization","author":"Satyanarayan","year":"2014"},{"key":"B47","doi-asserted-by":"publisher","first-page":"2161","DOI":"10.1109\/tvcg.2014.2346321","article-title":"Visual parameter space analysis: A conceptual framework","volume":"20","author":"Sedlmair","year":"2014","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B48","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1109\/mc.2002.1016905","article-title":"Interactively exploring hierarchical clustering results [gene identification]","volume":"35","author":"Seo","year":"2002","journal-title":"IEEE Comput."},{"key":"B49","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1177\/1473871615612883","article-title":"VisExpress: Visual exploration of differential gene expression data","volume":"16","author":"Simon","year":"2017","journal-title":"Inf. Vis."},{"article-title":"Direct-manipulation visualization of deep networks","year":"2017","author":"Smilkov","key":"B50"},{"key":"B51","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1007\/0-387-29362-0_23","article-title":"limma: Linear models for microarray data","author":"Smyth","year":"2005","journal-title":"Bioinforma. Comput. Biol. Solutions Using R Bioconductor"},{"key":"B52","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1109\/2945.981851","article-title":"Polaris: A system for query, analysis, and visualization of multidimensional relational databases","volume":"8","author":"Stolte","year":"2002","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B53","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1093\/bioinformatics\/18.1.207","article-title":"Genesis: Cluster analysis of microarray data","volume":"18","author":"Sturn","year":"2002","journal-title":"Bioinformatics"},{"key":"B54","doi-asserted-by":"publisher","first-page":"1064","DOI":"10.1093\/bioinformatics\/btw749","article-title":"Algorithm sensitivity analysis and parameter tuning for tissue image segmentation pipelines","volume":"33","author":"Teodoro","year":"2017","journal-title":"Bioinformatics"},{"key":"B55","doi-asserted-by":"publisher","first-page":"1892","DOI":"10.1109\/tvcg.2011.248","article-title":"Tuner: Principled parameter finding for image segmentation algorithms using visual response surface exploration","volume":"17","author":"Torsney-Weir","year":"2011","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B56","first-page":"77","article-title":"Integrating cluster formation and cluster evaluation in interactive visual analysis","author":"Turkay","year":"2011"},{"key":"B57","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2312\/cgvc.20161288","article-title":"MLCut: Exploring multi-level cuts in dendrograms for biological data","volume":"2016","author":"Vogogias","year":"2016","journal-title":"Comput. Graph. Vis. Comput. CGVC"},{"key":"B58","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1109\/tvcg.2016.2598830","article-title":"Multi-resolution climate ensemble parameter analysis with nested parallel coordinates plots","volume":"23","author":"Wang","year":"2017","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B59","doi-asserted-by":"publisher","first-page":"1458","DOI":"10.1109\/tvcg.2010.223","article-title":"World lines","volume":"16","author":"Waser","year":"2010","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B60","unstructured":"Receiver operating characteristic \u2014 Wikipedia, the free encyclopedia2018"},{"key":"B61","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1109\/INFVIS.2005.1532142","article-title":"Graph-theoretic scagnostics","author":"Wilkinson","year":"2005","journal-title":"IEEE Symposium Inf. Vis."},{"volume-title":"The grammar of graphics","year":"2006","author":"Wilkinson","key":"B62"},{"key":"B63","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1109\/tvcg.2015.2467191","article-title":"Voyager: Exploratory analysis via faceted browsing of visualization recommendations","volume":"22","author":"Wongsuphasawat","year":"2016","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B64","first-page":"2648","article-title":"Voyager 2: Augmenting visual analysis with partial view specifications","author":"Wongsuphasawat","year":"2017"},{"key":"B65","doi-asserted-by":"publisher","first-page":"2339","DOI":"10.1109\/tvcg.2017.2723393","article-title":"Keshif: Rapid and expressive tabular data exploration for novices","volume":"24","author":"Yal\u00e7\u0131n","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B66","unstructured":"Making R packages accessible to non-programmers using the VisRseq framework\n            YounesyH.\n          2016"},{"key":"B67","doi-asserted-by":"publisher","first-page":"S2","DOI":"10.1186\/1471-2105-16-s11-s2","article-title":"VisRseq: R-Based visual framework for analysis of sequencing data","volume":"16","author":"Younesy","year":"2015","journal-title":"BMC Bioinforma."}],"container-title":["Frontiers in Bioinformatics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fbinf.2023.1153800\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,25]],"date-time":"2023-05-25T10:46:05Z","timestamp":1685011565000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fbinf.2023.1153800\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,25]]},"references-count":67,"alternative-id":["10.3389\/fbinf.2023.1153800"],"URL":"https:\/\/doi.org\/10.3389\/fbinf.2023.1153800","relation":{},"ISSN":["2673-7647"],"issn-type":[{"type":"electronic","value":"2673-7647"}],"subject":[],"published":{"date-parts":[[2023,5,25]]},"article-number":"1153800"}}