{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,5]],"date-time":"2026-05-05T03:34:39Z","timestamp":1777952079105,"version":"3.51.4"},"reference-count":24,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2017,1,6]],"date-time":"2017-01-06T00:00:00Z","timestamp":1483660800000},"content-version":"vor","delay-in-days":7,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Sensitivity analysis and parameter tuning are important processes in large-scale image analysis. They are very costly because the image analysis workflows are required to be executed several times to systematically correlate output variations with parameter changes or to tune parameters. An integrated solution with minimum user interaction that uses effective methodologies and high performance computing is required to scale these studies to large imaging datasets and expensive analysis workflows.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The experiments with two segmentation workflows show that the proposed approach can (i) quickly identify and prune parameters that are non-influential; (ii) search a small fraction (about 100 points) of the parameter search space with billions to trillions of points and improve the quality of segmentation results (Dice and Jaccard metrics) by as much as 1.42\u00d7\u2009compared to the results from the default parameters; (iii) attain good scalability on a high performance cluster with several effective optimizations.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Our work demonstrates the feasibility of performing sensitivity analyses, parameter studies and auto-tuning with large datasets. The proposed framework can enable the quantification of error estimations and output variations in image segmentation pipelines.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and Implementation<\/jats:title>\n                  <jats:p>Source code: https:\/\/github.com\/SBU-BMI\/region-templates\/.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btw749","type":"journal-article","created":{"date-parts":[[2016,12,9]],"date-time":"2016-12-09T20:06:38Z","timestamp":1481313998000},"page":"1064-1072","source":"Crossref","is-referenced-by-count":20,"title":["Algorithm sensitivity analysis and parameter tuning for tissue image segmentation pipelines"],"prefix":"10.1093","volume":"33","author":[{"given":"George","family":"Teodoro","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Bras\u00edlia, Bras\u00edlia, Brazil"},{"name":"Biomedical Informatics Department, Stony Brook University, Stony Brook, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tahsin M","family":"Kur\u00e7","sequence":"additional","affiliation":[{"name":"Biomedical Informatics Department, Stony Brook University, Stony Brook, NY, USA"},{"name":"Scientific Data Group, Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lu\u00eds F R","family":"Taveira","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Bras\u00edlia, Bras\u00edlia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alba C M A","family":"Melo","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Bras\u00edlia, Bras\u00edlia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yi","family":"Gao","sequence":"additional","affiliation":[{"name":"Biomedical Informatics Department, Stony Brook University, Stony Brook, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jun","family":"Kong","sequence":"additional","affiliation":[{"name":"Biomedical Informatics Department, Emory University, Atlanta, GA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joel H","family":"Saltz","sequence":"additional","affiliation":[{"name":"Biomedical Informatics Department, Stony Brook University, Stony Brook, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2016,12,30]]},"reference":[{"key":"2023020205010394800_btw749-B1","doi-asserted-by":"crossref","first-page":"1509","DOI":"10.1016\/j.envsoft.2006.10.004","article-title":"An effective screening design for sensitivity analysis of large models","volume":"22","author":"Campolongo","year":"2007","journal-title":"Environ. Modell. Softw"},{"key":"2023020205010394800_btw749-B2","doi-asserted-by":"crossref","first-page":"297","DOI":"10.2307\/1932409","article-title":"Measures of the amount of ecologic association between species","volume":"26","author":"Dice","year":"1945","journal-title":"Ecology"},{"key":"2023020205010394800_btw749-B3","doi-asserted-by":"crossref","first-page":"979117","DOI":"10.1117\/12.2217029","article-title":"Hierarchical nucleus segmentation in digital pathology images","volume":"9791","author":"Gao","year":"2016","journal-title":"Proc. SPIE"},{"key":"2023020205010394800_btw749-B4","author":"Han","year":"2000"},{"key":"2023020205010394800_btw749-B5","first-page":"101","volume-title":"Uncertainty Management in Simulation-Optimization of Complex Systems, Volume 59 of Operations Research\/Computer Science Interfaces Series","author":"Iooss","year":"2015"},{"key":"2023020205010394800_btw749-B6","first-page":"431","volume-title":"Direct Global Optimization Algorithm, Encyclopedia of Optimization.","author":"Jones","year":"2001"},{"key":"2023020205010394800_btw749-B7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pone.0081049","article-title":"Machine-based morphologic analysis of glioblastoma using whole-slide pathology images uncovers clinically relevant molecular correlates","volume":"8","author":"Kong","year":"2013","journal-title":"PLoS ONE"},{"key":"2023020205010394800_btw749-B8","author":"Kumar","year":"2003"},{"key":"2023020205010394800_btw749-B9","first-page":"239","article-title":"A comparison of three methods for selecting values of input variables in the analysis of output from a computer code","volume":"21","author":"McKay","year":"1979","journal-title":"Technometrics"},{"key":"2023020205010394800_btw749-B10","author":"McIntosh","year":"2007"},{"key":"2023020205010394800_btw749-B11","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1080\/00401706.1991.10484804","article-title":"Factorial sampling plans for preliminary computational experiments","volume":"33","author":"Morris","year":"1991","journal-title":"Technometrics"},{"key":"2023020205010394800_btw749-B12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1471-2105-16-S11-S9","article-title":"Visual parameter optimisation for biomedical image processing","volume":"16","author":"Pretorius","year":"2015","journal-title":"BMC Bioinformatics"},{"key":"2023020205010394800_btw749-B13","doi-asserted-by":"crossref","first-page":"1247","DOI":"10.1007\/s10898-012-9951-y","article-title":"Derivative-free optimization: a review of algorithms and comparison of software implementations","volume":"56","author":"Rios","year":"2013","journal-title":"J. Global Optim"},{"key":"2023020205010394800_btw749-B14","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1016\/S0010-4655(02)00280-1","article-title":"Making best use of model evaluations to compute sensitivity indices","volume":"145","author":"Saltelli","year":"2002","journal-title":"Comput. Phys. Commun"},{"key":"2023020205010394800_btw749-B15","volume-title":"Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models","author":"Saltelli","year":"2004"},{"key":"2023020205010394800_btw749-B16","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1109\/4235.735432","article-title":"Fitness sharing and niching methods revisited","volume":"2","author":"Sareni","year":"1998","journal-title":"IEEE Trans. Evol. Comput"},{"key":"2023020205010394800_btw749-B17","doi-asserted-by":"crossref","first-page":"2100","DOI":"10.1109\/TVCG.2013.181","article-title":"Open-box spectral clustering: applications to medical image analysis","volume":"19","author":"Schultz","year":"2013","journal-title":"IEEE Trans. Vis. Comput. Graph"},{"key":"2023020205010394800_btw749-B18","first-page":"2951","volume-title":"Advances in Neural Information Processing Systems 25","author":"Snoek","year":"2012"},{"key":"2023020205010394800_btw749-B19","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1016\/S0378-4754(00)00270-6","article-title":"Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates","volume":"55","author":"Sobol","year":"2001","journal-title":"Math. Comput. Simul"},{"key":"2023020205010394800_btw749-B20","author":"Szummer","year":"2008"},{"key":"2023020205010394800_btw749-B21","author":"Tabatabaee","year":"2005"},{"key":"2023020205010394800_btw749-B22","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1016\/j.parco.2014.09.003","article-title":"Region templates: Data representation and management for high-throughput image analysis","volume":"40","author":"Teodoro","year":"2014","journal-title":"Parallel Comput"},{"key":"2023020205010394800_btw749-B23","doi-asserted-by":"crossref","first-page":"1892","DOI":"10.1109\/TVCG.2011.248","article-title":"Tuner: principled parameter finding for image segmentation algorithms using visual response surface exploration","volume":"17","author":"Torsney-Weir","year":"2011","journal-title":"IEEE Trans. Vis. Comput. Graph"},{"key":"2023020205010394800_btw749-B24","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.ress.2011.12.008","article-title":"Sensitivity analysis techniques applied to a system of hyperbolic conservation laws","volume":"107","author":"Weirs","year":"2012","journal-title":"Reliab. Eng. Syst. Saf"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/33\/7\/1064\/49038562\/bioinformatics_33_7_1064.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/33\/7\/1064\/49038562\/bioinformatics_33_7_1064.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,2]],"date-time":"2023-02-02T05:03:42Z","timestamp":1675314222000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/33\/7\/1064\/2843894"}},"subtitle":[],"editor":[{"given":"Robert","family":"Murphy","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2016,12,30]]},"references-count":24,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2017,4,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btw749","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2017,4,1]]},"published":{"date-parts":[[2016,12,30]]}}}