{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T12:47:27Z","timestamp":1775134047793,"version":"3.50.1"},"reference-count":40,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"name":"National Science Foundation","award":["CMMI-2206972 and CMMI-2206973"],"award-info":[{"award-number":["CMMI-2206972 and CMMI-2206973"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Model. Comput. Simul."],"published-print":{"date-parts":[[2026,4,30]]},"abstract":"<jats:p>\n                    We introduce a framework for constructing confidence intervals for the performance of a system as a function of a parameter, decision variable or system state, even when the system is not simulated at the particular parameter, decision variable or state. The proposed methods leverage observations from some other simulated model instances and known functional properties of the performance function being evaluated. The intervals, termed\n                    <jats:italic toggle=\"yes\">p<\/jats:italic>\n                    lausible intervals, deliver a desired coverage probability uniformly over\n                    <jats:italic toggle=\"yes\">all<\/jats:italic>\n                    model instances as the minimum sample size at the simulated model instances increases, and they attain the strongest possible consistency from simulating a finite number of model instances. We illustrate the versatility and effectiveness of plausible intervals through two numerical experiments.\n                  <\/jats:p>","DOI":"10.1145\/3786594","type":"journal-article","created":{"date-parts":[[2025,12,26]],"date-time":"2025-12-26T11:43:21Z","timestamp":1766749401000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Plausible Intervals: Global Inference from Limited Simulation of Structured Problems"],"prefix":"10.1145","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-5124-8124","authenticated-orcid":false,"given":"Tianqi","family":"Qiao","sequence":"first","affiliation":[{"name":"Industrial and Systems Engineering, Texas A&M University","place":["College Station, United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6473-6434","authenticated-orcid":false,"given":"David J.","family":"Eckman","sequence":"additional","affiliation":[{"name":"Industrial and Systems Engineering, Texas A&M University","place":["College Station, United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1325-2624","authenticated-orcid":false,"given":"Barry L.","family":"Nelson","sequence":"additional","affiliation":[{"name":"Industrial Engineering and Management Sciences, Northwestern University","place":["Evanston, United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,4,2]]},"reference":[{"key":"e_1_3_3_2_2","article-title":"Improved algorithms for linear stochastic bandits","volume":"24","author":"Abbasi-Yadkori Yasin","year":"2011","unstructured":"Yasin Abbasi-Yadkori, D\u00e1vid P\u00e1l, and Csaba Szepesv\u00e1ri. 2011. Improved algorithms for linear stochastic bandits. Advances in Neural Information Processing Systems 24 (2011). https:\/\/papers.nips.cc\/paper_files\/paper\/2011\/hash\/e1d5be1c7f2f456670de3d53c7b54f4a-Abstract.html","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"2","key":"e_1_3_3_3_2","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1287\/opre.1090.0754","article-title":"Stochastic kriging for simulation metamodeling","volume":"58","author":"Ankenman Bruce","year":"2010","unstructured":"Bruce Ankenman, Barry L. Nelson, and Jeremy Staum. 2010. Stochastic kriging for simulation metamodeling. Operations Research 58, 2 (2010), 371\u2013382.","journal-title":"Operations Research"},{"key":"e_1_3_3_4_2","first-page":"397","article-title":"Using confidence bounds for exploitation-exploration trade-offs","volume":"3","author":"Auer Peter","year":"2002","unstructured":"Peter Auer. 2002. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research 3, Nov (2002), 397\u2013422.","journal-title":"Journal of Machine Learning Research"},{"issue":"1","key":"e_1_3_3_5_2","first-page":"64","article-title":"Analyzing stochastic computer models: A review with opportunities","volume":"37","author":"Baker Evan","year":"2022","unstructured":"Evan Baker, Pierre Barbillon, Arindam Fadikar, Robert B. Gramacy, Radu Herbei, David Higdon, Jiangeng Huang, Leah R. Johnson, Pulong Ma, Anirban Mondal, et\u00a0al. 2022. Analyzing stochastic computer models: A review with opportunities. Statistical Science 37, 1 (2022), 64\u201389.","journal-title":"Statistical Science"},{"issue":"337","key":"e_1_3_3_6_2","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1080\/01621459.1972.10481216","article-title":"The isotonic regression problem and its dual","volume":"67","author":"Barlow Richard E.","year":"1972","unstructured":"Richard E. Barlow and Hugh D. Brunk. 1972. The isotonic regression problem and its dual. Journal of the American Statistical Association 67, 337 (1972), 140\u2013147.","journal-title":"Journal of the American Statistical Association"},{"key":"e_1_3_3_7_2","doi-asserted-by":"crossref","first-page":"1635","DOI":"10.1109\/WSC.2018.8632335","volume-title":"Proceedings of the 2018 Winter Simulation Conference","author":"Barton Russell R.","year":"2018","unstructured":"Russell R. Barton, Henry Lam, and Eunhye Song. 2018. Revisiting direct bootstrap resampling for input model uncertainty. In Proceedings of the 2018 Winter Simulation Conference. IEEE, 1635\u20131645."},{"issue":"1","key":"e_1_3_3_8_2","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s10107-017-1174-z","article-title":"Robust sample average approximation","volume":"171","author":"Bertsimas Dimitris","year":"2018","unstructured":"Dimitris Bertsimas, Vishal Gupta, and Nathan Kallus. 2018. Robust sample average approximation. Mathematical Programming 171, 1 (2018), 217\u2013282.","journal-title":"Mathematical Programming"},{"issue":"1","key":"e_1_3_3_9_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000024","article-title":"Regret analysis of stochastic and nonstochastic multi-armed bandit problems","volume":"5","author":"Bubeck S\u00e9bastien","year":"2012","unstructured":"S\u00e9bastien Bubeck, Nicolo Cesa-Bianchi, et\u00a0al. 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends\u00ae in Machine Learning 5, 1 (2012), 1\u2013122.","journal-title":"Foundations and Trends\u00ae in Machine Learning"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1214\/12-AOS1068"},{"issue":"1","key":"e_1_3_3_11_2","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s10107-018-1338-5","article-title":"Concave regression: Value-constrained estimation and likelihood ratio-based inference","volume":"174","author":"Doss Charles R","year":"2019","unstructured":"Charles R Doss. 2019. Concave regression: Value-constrained estimation and likelihood ratio-based inference. Mathematical Programming 174, 1 (2019), 5\u201339.","journal-title":"Mathematical Programming"},{"issue":"3","key":"e_1_3_3_12_2","first-page":"423","article-title":"Optimal confidence bands for shape-restricted curves","volume":"9","author":"D\u00fcmbgen Lutz","year":"2003","unstructured":"Lutz D\u00fcmbgen. 2003. Optimal confidence bands for shape-restricted curves. Bernoulli 9, 3 (2003), 423\u2013449.","journal-title":"Bernoulli"},{"issue":"6","key":"e_1_3_3_13_2","doi-asserted-by":"crossref","first-page":"3473","DOI":"10.1287\/opre.2021.2206","article-title":"Plausible screening using functional properties for simulations with large solution spaces","volume":"70","author":"Eckman David J.","year":"2022","unstructured":"David J. Eckman, Matthew Plumlee, and Barry L. Nelson. 2022. Plausible screening using functional properties for simulations with large solution spaces. Operations Research 70, 6 (2022), 3473\u20133489.","journal-title":"Operations Research"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1214\/18-STS657"},{"issue":"1","key":"e_1_3_3_15_2","first-page":"3261","article-title":"Multivariate convex regression with adaptive partitioning","volume":"14","author":"Hannah Lauren A.","year":"2013","unstructured":"Lauren A. Hannah and David B. Dunson. 2013. Multivariate convex regression with adaptive partitioning. The Journal of Machine Learning Research 14, 1 (2013), 3261\u20133294.","journal-title":"The Journal of Machine Learning Research"},{"issue":"2","key":"e_1_3_3_16_2","first-page":"525","article-title":"Finite-sample confidence envelopes for shape-restricted densities","volume":"23","author":"Hengartner Nicolas W.","year":"1995","unstructured":"Nicolas W. Hengartner and Philip B. Stark. 1995. Finite-sample confidence envelopes for shape-restricted densities. The Annals of Statistics 23, 2 (1995), 525\u2013550.","journal-title":"The Annals of Statistics"},{"key":"e_1_3_3_17_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11222-014-9520-y","article-title":"A unified framework of constrained regression","volume":"26","author":"Hofner Benjamin","year":"2016","unstructured":"Benjamin Hofner, Thomas Kneib, and Torsten Hothorn. 2016. A unified framework of constrained regression. Statistics and Computing 26, 1\u20132 (2016), 1\u201314. https:\/\/link.springer.com\/journal\/11222\/volumes-and-issues\/26-1","journal-title":"Statistics and Computing"},{"key":"e_1_3_3_18_2","doi-asserted-by":"crossref","first-page":"1780","DOI":"10.1109\/WSC.2015.7408295","volume-title":"Proceedings of the 2015 Winter Simulation Conference","author":"Jian Nanjing","year":"2015","unstructured":"Nanjing Jian and Shane G. Henderson. 2015. An introduction to simulation optimization. In Proceedings of the 2015 Winter Simulation Conference. IEEE, 1780\u20131794."},{"issue":"2","key":"e_1_3_3_19_2","first-page":"356","article-title":"Online risk monitoring using offline simulation","volume":"32","author":"Jiang Guangxin","year":"2020","unstructured":"Guangxin Jiang, L Jeff Hong, and Barry L Nelson. 2020. Online risk monitoring using offline simulation. INFORMS Journal on Computing 32, 2 (2020), 356\u2013375.","journal-title":"INFORMS Journal on Computing"},{"key":"e_1_3_3_20_2","volume-title":"Proceedings of the 2024 Winter Simulation Conference","author":"Keslin Gregory","year":"2024","unstructured":"Gregory Keslin, Daniel W. Apley, and Barry L. Nelson. 2024. Plausible inference with a plausible Lipschitz constant. In Proceedings of the 2024 Winter Simulation Conference. IEEE, 3554\u20133565."},{"key":"e_1_3_3_21_2","doi-asserted-by":"crossref","unstructured":"Graham Laidler Lucy E. Morgan Nicos G. Pavlidis Barry L. Nelson. 2025. Stochastic neighbourhood components analysis. INFORMS Journal on Data Science 4 3 (2025) 248\u2013264.","DOI":"10.1287\/ijds.2023.0018"},{"issue":"4","key":"e_1_3_3_22_2","first-page":"1090","article-title":"Recovering best statistical guarantees via the empirical divergence-based distributionally robust optimization","volume":"67","author":"Lam Henry","year":"2019","unstructured":"Henry Lam. 2019. Recovering best statistical guarantees via the empirical divergence-based distributionally robust optimization. Operations Research 67, 4 (2019), 1090\u20131105.","journal-title":"Operations Research"},{"key":"e_1_3_3_23_2","unstructured":"Armin Lederer Jonas Umlauft and Sandra Hirche. 2021. Uniform error and posterior variance bounds for gaussian process regression with application to safe control. Advances in Neural Information Processing Systems 32 (2019). https:\/\/papers.nips.cc\/paper_files\/paper\/2019\/hash\/fe73f687e5bc5280214e0486b273a5f9-Abstract.html"},{"issue":"1","key":"e_1_3_3_24_2","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1287\/opre.1110.1007","article-title":"Consistency of multidimensional convex regression","volume":"60","author":"Lim Eunji","year":"2012","unstructured":"Eunji Lim and Peter W Glynn. 2012. Consistency of multidimensional convex regression. Operations Research 60, 1 (2012), 196\u2013208.","journal-title":"Operations Research"},{"issue":"10","key":"e_1_3_3_25_2","doi-asserted-by":"crossref","first-page":"923","DOI":"10.1080\/24725854.2021.2018739","article-title":"Unifying offline and online simulation for online decision-making","volume":"54","author":"Liu Haitao","year":"2022","unstructured":"Haitao Liu, Jinpeng Liang, Loo Hay Lee, and Ek Peng Chew. 2022. Unifying offline and online simulation for online decision-making. IISE Transactions 54, 10 (2022), 923\u2013935.","journal-title":"IISE Transactions"},{"issue":"3","key":"e_1_3_3_26_2","doi-asserted-by":"crossref","first-page":"879","DOI":"10.1111\/sjos.12217","article-title":"Simultaneous confidence tubes in multivariate linear regression","volume":"43","author":"Liu Wei","year":"2016","unstructured":"Wei Liu, Yang Han, Fang Wan, Frank Bretz, and Anthony J Hayter. 2016. Simultaneous confidence tubes in multivariate linear regression. Scandinavian Journal of Statistics 43, 3 (2016), 879\u2013885.","journal-title":"Scandinavian Journal of Statistics"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1198\/016214506000001301"},{"issue":"1","key":"e_1_3_3_28_2","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1111\/j.1751-5823.2007.00027.x","article-title":"Construction of exact simultaneous confidence bands for a simple linear regression model","volume":"76","author":"Liu Wei","year":"2008","unstructured":"Wei Liu, Shan Lin, and Walter W. Piegorsch. 2008. Construction of exact simultaneous confidence bands for a simple linear regression model. International Statistical Review 76, 1 (2008), 39\u201357.","journal-title":"International Statistical Review"},{"issue":"1","key":"e_1_3_3_29_2","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1007\/BF02680565","article-title":"Discrete convex analysis","volume":"83","author":"Murota Kazuo","year":"1998","unstructured":"Kazuo Murota. 1998. Discrete convex analysis. Mathematical Programming 83, 1-3 (1998), 313\u2013371.","journal-title":"Mathematical Programming"},{"key":"e_1_3_3_30_2","unstructured":"Hamed Rahimian and Sanjay Mehrotra. 2019. Distributionally robust optimization: A review. https:\/\/optimization-online.org\/wp-content\/uploads\/2019\/08\/7332.pdf"},{"issue":"3","key":"e_1_3_3_31_2","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1080\/00401706.1966.10490374","article-title":"A comparison of some control chart procedures","volume":"8","author":"Roberts SW","year":"1966","unstructured":"SW Roberts. 1966. A comparison of some control chart procedures. Technometrics 8, 3 (1966), 411\u2013430.","journal-title":"Technometrics"},{"key":"e_1_3_3_32_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-3799-8","volume-title":"The Design and Analysis of Computer Experiments","author":"Santner Thomas J.","year":"2003","unstructured":"Thomas J. Santner, Brian J. Williams, William I. Notz, and Brain J. Williams. 2003. The Design and Analysis of Computer Experiments. Springer, New York."},{"issue":"4","key":"e_1_3_3_33_2","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1287\/mksc.2016.1023","article-title":"Customer acquisition via display advertising using multi-armed bandit experiments","volume":"36","author":"Schwartz Eric M.","year":"2017","unstructured":"Eric M. Schwartz, Eric T. Bradlow, and Peter S. Fader. 2017. Customer acquisition via display advertising using multi-armed bandit experiments. Marketing Science 36, 4 (2017), 500\u2013522.","journal-title":"Marketing Science"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3426359"},{"issue":"2","key":"e_1_3_3_35_2","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1615\/JMachLearnModelComput.2020035155","article-title":"A survey of constrained Gaussian process regression: Approaches and implementation challenges","volume":"1","author":"Swiler Laura P.","year":"2020","unstructured":"Laura P. Swiler, Mamikon Gulian, Ari L. Frankel, Cosmin Safta, and John D. Jakeman. 2020. A survey of constrained Gaussian process regression: Approaches and implementation challenges. Journal of Machine Learning for Modeling and Computing 1, 2 (2020), 119\u2013156.","journal-title":"Journal of Machine Learning for Modeling and Computing"},{"issue":"1","key":"e_1_3_3_36_2","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1198\/TECH.2010.10111","article-title":"Nearly-isotonic regression","volume":"53","author":"Tibshirani Ryan J.","year":"2011","unstructured":"Ryan J. Tibshirani, Holger Hoefling, and Robert Tibshirani. 2011. Nearly-isotonic regression. Technometrics 53, 1 (2011), 54\u201361.","journal-title":"Technometrics"},{"issue":"2","key":"e_1_3_3_37_2","first-page":"199","article-title":"Multi-armed bandit models for the optimal design of clinical trials: Benefits and challenges","volume":"30","author":"Villar Sof\u00eda S.","year":"2015","unstructured":"Sof\u00eda S. Villar, Jack Bowden, and James Wason. 2015. Multi-armed bandit models for the optimal design of clinical trials: Benefits and challenges. Statistical Science 30, 2 (2015), 199\u2013215.","journal-title":"Statistical Science"},{"issue":"2","key":"e_1_3_3_38_2","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/s10287-015-0240-3","article-title":"Likelihood robust optimization for data-driven problems","volume":"13","author":"Wang Zizhuo","year":"2016","unstructured":"Zizhuo Wang, Peter W. Glynn, and Yinyu Ye. 2016. Likelihood robust optimization for data-driven problems. Computational Management Science 13, 2 (2016), 241\u2013261.","journal-title":"Computational Management Science"},{"key":"e_1_3_3_39_2","volume-title":"Gaussian Processes for Machine Learning","author":"Williams Christopher KI","year":"2006","unstructured":"Christopher KI Williams and Carl Edward Rasmussen. 2006. Gaussian Processes for Machine Learning. Vol. 2. MIT Press, Cambridge, MA."},{"issue":"165","key":"e_1_3_3_40_2","doi-asserted-by":"crossref","first-page":"73","DOI":"10.2307\/2277011","article-title":"Applications of the theory of error to the interpretation of trends","volume":"24","author":"Working Holbrook","year":"1929","unstructured":"Holbrook Working and Harold Hotelling. 1929. Applications of the theory of error to the interpretation of trends. Journal of the American Statistical Association 24, 165 (1929), 73\u201385. http:\/\/www.jstor.org\/stable\/2277011","journal-title":"Journal of the American Statistical Association"},{"issue":"2","key":"e_1_3_3_41_2","first-page":"816","article-title":"Batch-sequential design and heteroskedastic surrogate modeling for delta smelt conservation","volume":"16","author":"Zhang Boya","year":"2022","unstructured":"Boya Zhang, Robert B. Gramacy, Leah R. Johnson, Kenneth A. Rose, and Eric Smith. 2022. Batch-sequential design and heteroskedastic surrogate modeling for delta smelt conservation. The Annals of Applied Statistics 16, 2 (2022), 816\u2013842.","journal-title":"The Annals of Applied Statistics"}],"container-title":["ACM Transactions on Modeling and Computer Simulation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3786594","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T11:46:12Z","timestamp":1775130372000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3786594"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,2]]},"references-count":40,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,4,30]]}},"alternative-id":["10.1145\/3786594"],"URL":"https:\/\/doi.org\/10.1145\/3786594","relation":{},"ISSN":["1049-3301","1558-1195"],"issn-type":[{"value":"1049-3301","type":"print"},{"value":"1558-1195","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,2]]},"assertion":[{"value":"2024-08-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-17","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}