{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T17:06:22Z","timestamp":1783962382849,"version":"3.55.0"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T00:00:00Z","timestamp":1780531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T00:00:00Z","timestamp":1780531200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100008131","name":"Rheinische Friedrich-Wilhelms-Universit\u00e4t Bonn","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100008131","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Behav Res"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Experiment-based calibration is an emerging approach for measurement validation in the behavioural sciences. It allows comparing multiple measurement methods by how well they reproduce a known experimental manipulation, providing insight into their measurement accuracy. Calibration entails questions unparalleled in classical validation approaches. The first is about inference: when should we conclude that one measurement method is truly more accurate than another? The second is about decisions: when should we decide that a method merits the investment of changing a measurement system? In this note, we review these questions in the context of the statistical challenges that arise in a calibration process: a potentially large and a priori unknown number of measurement methods; a requirement to integrate evidence across multiple calibration samples; and a possibility that some methods may not be available for all samples. We show that Bayesian meta-analytic model comparison is a suitable framework for inference in calibration, and propose a decision-theoretic approach to calculate immediate economic gain garnered through reduced sample sizes. In order to overcome the practical hurdles associated with the analysis of calibration experiments, we present\n                    <jats:italic>CalibR<\/jats:italic>\n                    , an R package for calibration inference.\n                  <\/jats:p>","DOI":"10.3758\/s13428-026-03042-9","type":"journal-article","created":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T13:54:06Z","timestamp":1780581246000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Experiment-based calibration: Inference and decision-making"],"prefix":"10.3758","volume":"58","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4606-5876","authenticated-orcid":false,"given":"Federico","family":"Mancinelli","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dominik R.","family":"Bach","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,6,4]]},"reference":[{"key":"3042_CR1","unstructured":"AERA, on the Standards for Educational, J.C., of the American Educational Research Association, P.T., the American Psychological Association, the National Council on Measurement in Education. (2014). The Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association."},{"key":"3042_CR2","doi-asserted-by":"publisher","unstructured":"Bach, D.R. (2023). Psychometrics in experimental psychology: A case for calibration. Psychonomic Bulletin & Review. https:\/\/doi.org\/10.3758\/s13423-023-02421-z.","DOI":"10.3758\/s13423-023-02421-z"},{"key":"3042_CR3","doi-asserted-by":"publisher","DOI":"10.1016\/j.jmp.2025.102950","volume":"127","author":"DR Bach","year":"2025","unstructured":"Bach, D. R. (2025). Experiment-based calibration in psychology: foundational and data-generating model. Journal of Mathematical Psychology, 127, Article 102950.","journal-title":"Journal of Mathematical Psychology"},{"key":"3042_CR4","doi-asserted-by":"publisher","unstructured":"Bach, D.R., & Friston, K.J. (2013). Model-based analysis of skin conductance responses: Towards causal models in psychophysiology. Psychophysiology,\u00a050, 15\u201322. http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/23094650, https:\/\/doi.org\/10.1111\/j.1469-8986.2012.01483.x.","DOI":"10.1111\/j.1469-8986.2012.01483.x"},{"key":"3042_CR5","doi-asserted-by":"publisher","unstructured":"Bach, D.R., & Melinscak, F. (2020). Psychophysiological modelling and the measurement of fear conditioning. Behavioral Research and Therapy,\u00a0127, 103576. https:\/\/www.ncbi.nlm.nih.gov\/pubmed\/32087391, https:\/\/doi.org\/10.1016\/j.brat.2020.103576.","DOI":"10.1016\/j.brat.2020.103576"},{"key":"3042_CR6","doi-asserted-by":"publisher","unstructured":"Bach, D.R., Castegnetti, G., Korn, C.W., Gerster, S., Melinscak, F., & Moser, T. (2018). Psychophysiological modeling: Current state and future directions. Psychophysiology, e13214. https:\/\/www.ncbi.nlm.nih.gov\/pubmed\/30175471, https:\/\/doi.org\/10.1111\/psyp.13209.","DOI":"10.1111\/psyp.13209"},{"key":"3042_CR7","doi-asserted-by":"publisher","unstructured":"Bach, D.R., Melinscak, F., Fleming, S.M., & Voelkle, M.C. (2020). Calibrating the experimental measurement of psychological attributes. Nature Human Behaviour,\u00a04, 1229\u20131235. https:\/\/www.ncbi.nlm.nih.gov\/pubmed\/33199857, https:\/\/doi.org\/10.1038\/s41562-020-00976-8.","DOI":"10.1038\/s41562-020-00976-8"},{"key":"3042_CR8","doi-asserted-by":"publisher","unstructured":"Bach, D.R., Sporrer, J., Abend, R., Beckers, T., Dunsmoor, J.E., Fullana, M.A., . . . Schiller, D. (2023). Consensus design of a calibration experiment for human fear conditioning. Neuroscience and Biobehavioral Reviews,\u00a0148, 105146. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S014976342300115X, https:\/\/doi.org\/10.1016\/j.neubiorev.2023.105146.","DOI":"10.1016\/j.neubiorev.2023.105146"},{"key":"3042_CR9","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbusres.2025.115352","volume":"193","author":"DR Bach","year":"2025","unstructured":"Bach, D. R., Rigdon, E. E., & Sarstedt, M. (2025). Calibration experiments: An alternative to multi-method approaches for measurement validation in consumer research. Journal of Business Research, 193, Article 115352.","journal-title":"Journal of Business Research"},{"key":"3042_CR10","doi-asserted-by":"publisher","DOI":"10.1002\/9781118619179","volume-title":"Structural equations with latent variables","author":"K Bollen","year":"1989","unstructured":"Bollen, K. (1989). Structural equations with latent variables. New York: Wiley."},{"key":"3042_CR11","doi-asserted-by":"crossref","unstructured":"Botvinik-Nezer, R., Holzmeister, F., Camerer, C. F., Dreber, A., Huber, J., Johannesson, M., . . . Rieck, J. R. (2020). Variability in the analysis of a single neuroimaging dataset by many teams. Nature,582, 84\u201388.","DOI":"10.1038\/s41586-020-2314-9"},{"key":"3042_CR12","doi-asserted-by":"publisher","unstructured":"Castegnetti, G., Tzovara, A., Staib, M., Gerster, S., & Bach, D.R. (2017). Assessing fear learning via conditioned respiratory amplitude responses. Psychophysiology,\u00a054, 215\u2013223. http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27933608, https:\/\/doi.org\/10.1111\/psyp.12778.","DOI":"10.1111\/psyp.12778"},{"key":"3042_CR13","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1037\/h0043943","volume":"12","author":"LJ Cronbach","year":"1957","unstructured":"Cronbach, L. J. (1957). The two disciplines of scientific psychology. American Psychologist, 12, 671.","journal-title":"American Psychologist"},{"key":"3042_CR14","doi-asserted-by":"publisher","unstructured":"Hedge, C., Powell, G., & Sumner, P. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods,\u00a050, 1166\u20131186. https:\/\/www.ncbi.nlm.nih.gov\/pubmed\/28726177, https:\/\/doi.org\/10.3758\/s13428-017-0935-1.","DOI":"10.3758\/s13428-017-0935-1"},{"key":"3042_CR15","doi-asserted-by":"publisher","unstructured":"Khemka, S., Tzovara, A., Gerster, S., Quednow, B.B., & Bach, D.R. (2017). Modeling startle eyeblink electromyogram to assess fear learning. Psychophysiology,\u00a054, 204\u2013214. http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27753123, https:\/\/doi.org\/10.1111\/psyp.12775.","DOI":"10.1111\/psyp.12775"},{"key":"3042_CR16","doi-asserted-by":"publisher","unstructured":"Korn, C.W., Staib, M., Tzovara, A., Castegnetti, G., & Bach, D.R. (2017). A pupil size response model to assess fear learning. Psychophysiology,\u00a054, 330\u2013343. http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27925650, https:\/\/doi.org\/10.1111\/psyp.12801.","DOI":"10.1111\/psyp.12801"},{"key":"3042_CR17","doi-asserted-by":"publisher","first-page":"6119","DOI":"10.3758\/s13428-024-02341-3","volume":"56","author":"F Mancinelli","year":"2024","unstructured":"Mancinelli, F., Sporrer, J. K., Myrov, V., Melinscak, F., Zimmermann, J., Liu, H., & Bach, D. R. (2024). Dimensionality and optimal combination of autonomic fear-conditioning measures in humans. Behavior Research Methods, 56, 6119\u20136129. https:\/\/doi.org\/10.3758\/s13428-024-02341-3","journal-title":"Behavior Research Methods"},{"key":"3042_CR18","doi-asserted-by":"crossref","unstructured":"McDonald, R.P. (2013). Test theory: A unified treatment. psychology press.","DOI":"10.4324\/9781410601087"},{"key":"3042_CR19","unstructured":"Nikolakopoulos, S., & Ntzoufras, I. (2021). Meta analysis of bayes factors. arXiv preprint arXiv:2103.13236."},{"key":"3042_CR20","doi-asserted-by":"publisher","unstructured":"Paulus, P.C., Castegnetti, G., & Bach, D.R. (2016). Modeling event-related heart period responses. Psychophysiology,\u00a053, 837\u201346. http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26849101, https:\/\/doi.org\/10.1111\/psyp.12622.","DOI":"10.1111\/psyp.12622"},{"key":"3042_CR21","doi-asserted-by":"publisher","first-page":"682","DOI":"10.3758\/s13423-011-0088-7","volume":"18","author":"JN Rouder","year":"2011","unstructured":"Rouder, J. N., & Morey, R. D. (2011). A Bayes factor meta-analysis of Bem\u2019s ESP claim. Psychonomic Bulletin & Review, 18, 682\u2013689.","journal-title":"Psychonomic Bulletin & Review"},{"key":"3042_CR22","doi-asserted-by":"publisher","unstructured":"Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., . . . Nosek, B. A. (2018). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science,\u00a01, 337\u2013356. https:\/\/doi.org\/10.1177\/2515245917747646.","DOI":"10.1177\/2515245917747646"},{"key":"3042_CR23","doi-asserted-by":"publisher","first-page":"1359","DOI":"10.1177\/0956797611417632","volume":"22","author":"JP Simmons","year":"2011","unstructured":"Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359\u20131366. https:\/\/doi.org\/10.1177\/0956797611417632","journal-title":"Psychological Science"},{"key":"3042_CR24","doi-asserted-by":"publisher","unstructured":"Staib, M., Castegnetti, G., & Bach, D.R. (2015). Optimising a model-based approach to inferring fear learning from skin conductance responses. Journal of Neuroscience Methods,\u00a0255, 131\u20138. http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/26291885, https:\/\/doi.org\/10.1016\/j.jneumeth.2015.08.009.","DOI":"10.1016\/j.jneumeth.2015.08.009"},{"key":"3042_CR25","doi-asserted-by":"publisher","unstructured":"Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science,\u00a011, 702\u2013712. https:\/\/www.ncbi.nlm.nih.gov\/pubmed\/27694465, https:\/\/doi.org\/10.1177\/1745691616658637.","DOI":"10.1177\/1745691616658637"},{"key":"3042_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/014662168500900101","volume":"9","author":"KF Widaman","year":"1985","unstructured":"Widaman, K. F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1\u201326. https:\/\/doi.org\/10.1177\/014662168500900101","journal-title":"Applied Psychological Measurement"},{"key":"3042_CR27","doi-asserted-by":"crossref","unstructured":"Zumbo, B.D., & Chan, E.K. (2014). Reflections on validation practices in the social, behavioral, and health sciences. In Validity and validation in social, behavioral, and health sciences. Springer, pp. 321\u2013327.","DOI":"10.1007\/978-3-319-07794-9_19"}],"container-title":["Behavior Research Methods"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.3758\/s13428-026-03042-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.3758\/s13428-026-03042-9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.3758\/s13428-026-03042-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T16:17:50Z","timestamp":1783959470000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.3758\/s13428-026-03042-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,4]]},"references-count":27,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["3042"],"URL":"https:\/\/doi.org\/10.3758\/s13428-026-03042-9","relation":{},"ISSN":["1554-3528"],"issn-type":[{"value":"1554-3528","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,4]]},"assertion":[{"value":"19 September 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 April 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 June 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"None.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest\/Competing interests"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"All authors read and approved the final manuscript.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}],"article-number":"186"}}