{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T15:43:46Z","timestamp":1772811826557,"version":"3.50.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1011004","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2023,5,8]],"date-time":"2023-05-08T00:00:00Z","timestamp":1683504000000}}],"reference-count":48,"publisher":"Public Library of Science (PLoS)","issue":"4","license":[{"start":{"date-parts":[[2023,4,26]],"date-time":"2023-04-26T00:00:00Z","timestamp":1682467200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/publicdomain\/zero\/1.0\/"}],"funder":[{"DOI":"10.13039\/100000052","name":"NIH Office of the Director","doi-asserted-by":"publisher","award":["T32- GM139800"],"award-info":[{"award-number":["T32- GM139800"]}],"id":[{"id":"10.13039\/100000052","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CAREER Award MCB 1942255"],"award-info":[{"award-number":["CAREER Award MCB 1942255"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000052","name":"NIH Office of the Director","doi-asserted-by":"publisher","award":["U54-CA217450"],"award-info":[{"award-number":["U54-CA217450"]}],"id":[{"id":"10.13039\/100000052","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000052","name":"NIH Office of the Director","doi-asserted-by":"publisher","award":["U01-CA215845"],"award-info":[{"award-number":["U01-CA215845"]}],"id":[{"id":"10.13039\/100000052","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Mathematical models are often used to explore network-driven cellular processes from a systems perspective. However, a dearth of quantitative data suitable for model calibration leads to models with parameter unidentifiability and questionable predictive power. Here we introduce a combined Bayesian and Machine Learning Measurement Model approach to explore how quantitative and non-quantitative data constrain models of apoptosis execution within a missing data context. We find model prediction accuracy and certainty strongly depend on rigorous data-driven formulations of the measurement, and the size and make-up of the datasets. For instance, two orders of magnitude more ordinal (e.g., immunoblot) data are necessary to achieve accuracy comparable to quantitative (e.g., fluorescence) data for calibration of an apoptosis execution model. Notably, ordinal and nominal (e.g., cell fate observations) non-quantitative data synergize to reduce model uncertainty and improve accuracy. Finally, we demonstrate the potential of a data-driven Measurement Model approach to identify model features that could lead to informative experimental measurements and improve model predictive power.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1011004","type":"journal-article","created":{"date-parts":[[2023,4,26]],"date-time":"2023-04-26T17:47:49Z","timestamp":1682531269000},"page":"e1011004","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["Model certainty in cellular network-driven processes with missing data"],"prefix":"10.1371","volume":"19","author":[{"given":"Michael W.","family":"Irvin","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arvind","family":"Ramanathan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3668-7468","authenticated-orcid":true,"given":"Carlos F.","family":"Lopez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2023,4,26]]},"reference":[{"key":"pcbi.1011004.ref001","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1007\/s10822-011-9490-5","article-title":"If we designed airplanes like we design drugs\u2026","volume":"26","author":"WS Woltosz","year":"2012","journal-title":"Journal of computer-aided molecular design"},{"issue":"6","key":"pcbi.1011004.ref002","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1038\/nrd1754","article-title":"Can cell systems biology rescue drug discovery?","volume":"4","author":"EC Butcher","year":"2005","journal-title":"Nature Reviews Drug Discovery"},{"issue":"8","key":"pcbi.1011004.ref003","doi-asserted-by":"crossref","first-page":"a028894","DOI":"10.1101\/cshperspect.a028894","article-title":"Will systems biology deliver its promise and contribute to the development of new or improved vaccines? From data to understanding through systems biology","volume":"10","author":"T Hagan","year":"2018","journal-title":"Cold Spring Harbor perspectives in biology"},{"key":"pcbi.1011004.ref004","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/3087.001.0001","volume-title":"Foundations of systems biology","author":"H. Kitano","year":"2001"},{"issue":"6912","key":"pcbi.1011004.ref005","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1038\/nature01254","article-title":"Computational systems biology","volume":"420","author":"H. Kitano","year":"2002","journal-title":"Nature"},{"issue":"5","key":"pcbi.1011004.ref006","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1016\/j.cell.2019.04.024","article-title":"The coming decade of cell death research: five riddles","volume":"177","author":"DR Green","year":"2019","journal-title":"Cell"},{"issue":"91","key":"pcbi.1011004.ref007","doi-asserted-by":"crossref","first-page":"20130505","DOI":"10.1098\/rsif.2013.0505","article-title":"Reverse engineering and identification in systems biology: strategies, perspectives and challenges","volume":"11","author":"AF Villaverde","year":"2014","journal-title":"Journal of the Royal Society Interface"},{"key":"pcbi.1011004.ref008","article-title":"Observability and structural identifiability of nonlinear biological systems","author":"AF Villaverde","year":"2019","journal-title":"Complexity."},{"issue":"10","key":"pcbi.1011004.ref009","doi-asserted-by":"crossref","first-page":"giz127","DOI":"10.1093\/gigascience\/giz127","article-title":"Key challenges facing data-driven multicellular systems biology","volume":"8","author":"P. Macklin","year":"2019","journal-title":"Gigascience"},{"issue":"3","key":"pcbi.1011004.ref010","doi-asserted-by":"crossref","first-page":"e1003498","DOI":"10.1371\/journal.pcbi.1003498","article-title":"Model-based analysis for qualitative data: an application in drosophila germline stem cell regulation","volume":"10","author":"M Pargett","year":"2014","journal-title":"PLoS computational biology"},{"key":"pcbi.1011004.ref011","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1007\/s00285-020-01522-w","article-title":"Parameterization of mechanistic models from qualitative data using an efficient optimal scaling approach","volume":"81","author":"L Schmiester","year":"2020","journal-title":"Journal of Mathematical Biology"},{"issue":"10","key":"pcbi.1011004.ref012","doi-asserted-by":"crossref","first-page":"3177","DOI":"10.1093\/bioinformatics\/btaa084","article-title":"Bayesian inference using qualitative observations of underlying continuous variables","volume":"36","author":"ED Mitra","year":"2020","journal-title":"Bioinformatics"},{"issue":"1","key":"pcbi.1011004.ref013","doi-asserted-by":"crossref","first-page":"3901","DOI":"10.1038\/s41467-018-06439-z","article-title":"Using both qualitative and quantitative data in parameter identification for systems biology models","volume":"9","author":"ED Mitra","year":"2018","journal-title":"Nature communications"},{"issue":"1","key":"pcbi.1011004.ref014","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1091\/mbc.11.1.369","article-title":"Kinetic analysis of a molecular model of the budding yeast cell cycle","volume":"11","author":"KC Chen","year":"2000","journal-title":"Molecular biology of the cell"},{"issue":"8","key":"pcbi.1011004.ref015","doi-asserted-by":"crossref","first-page":"3841","DOI":"10.1091\/mbc.e03-11-0794","article-title":"Integrative analysis of cell cycle control in budding yeast","volume":"15","author":"KC Chen","year":"2004","journal-title":"Molecular biology of the cell"},{"issue":"2","key":"pcbi.1011004.ref016","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1093\/bioinformatics\/btz581","article-title":"Efficient parameterization of large-scale dynamic models based on relative measurements","volume":"36","author":"L Schmiester","year":"2020","journal-title":"Bioinformatics"},{"key":"pcbi.1011004.ref017","volume-title":"Model selection and multi-model inference","author":"D Anderson","year":"2004","edition":"2"},{"issue":"1","key":"pcbi.1011004.ref018","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1086\/658408","article-title":"Measurement and meaning in biology","volume":"86","author":"D Houle","year":"2011","journal-title":"The quarterly review of biology"},{"issue":"3","key":"pcbi.1011004.ref019","doi-asserted-by":"crossref","first-page":"54","DOI":"10.3390\/medsci6030054","article-title":"Apoptosis: activation and inhibition in health and disease","volume":"6","author":"SG Solano-G\u00e1lvez","year":"2018","journal-title":"Medical Sciences"},{"issue":"5","key":"pcbi.1011004.ref020","doi-asserted-by":"crossref","first-page":"803","DOI":"10.15252\/msb.20145584","article-title":"Fractional killing arises from cell-to-cell variability in overcoming a caspase activity threshold","volume":"11","author":"J Roux","year":"2015","journal-title":"Molecular systems biology"},{"issue":"1","key":"pcbi.1011004.ref021","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.molcel.2008.02.012","article-title":"Quantitative analysis of pathways controlling extrinsic apoptosis in single cells","volume":"30","author":"JG Albeck","year":"2008","journal-title":"Molecular cell"},{"issue":"26","key":"pcbi.1011004.ref022","doi-asserted-by":"crossref","first-page":"16589","DOI":"10.1074\/jbc.273.26.16589","article-title":"Apoptosis induction by caspase-8 is amplified through the mitochondrial release of cytochrome c","volume":"273","author":"T Kuwana","year":"1998","journal-title":"Journal of Biological Chemistry"},{"key":"pcbi.1011004.ref023","first-page":"2021","article-title":"Probability-based mechanisms in biological networks with parameter uncertainty","author":"OO Ortega","year":"2021","journal-title":"bioRxiv"},{"issue":"1","key":"pcbi.1011004.ref024","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1111\/j.1749-6632.2009.04905.x","article-title":"TRAIL-induced apoptosis: between tumor therapy and immunopathology","volume":"1171","author":"N Corazza","year":"2009","journal-title":"Annals of the New York Academy of Sciences"},{"issue":"4","key":"pcbi.1011004.ref025","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1080\/01926230701320337","article-title":"Apoptosis: a review of programmed cell death","volume":"35","author":"S. Elmore","year":"2007","journal-title":"Toxicologic pathology"},{"issue":"4","key":"pcbi.1011004.ref026","doi-asserted-by":"crossref","first-page":"558","DOI":"10.1016\/j.bbamcr.2011.01.026","article-title":"Caspase-8 and bid: caught in the act between death receptors and mitochondria","volume":"1813","author":"C Kantari","year":"2011","journal-title":"Biochimica et Biophysica Acta (BBA)-Molecular Cell Research"},{"issue":"6","key":"pcbi.1011004.ref027","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1038\/sj.neo.7900270","article-title":"Defining characteristics of Types I and II apoptotic cells in response to TRAIL","volume":"4","author":"N \u00d6z\u00f6ren","year":"2002","journal-title":"Neoplasia"},{"issue":"17","key":"pcbi.1011004.ref028","first-page":"3976","article-title":"Specific proteolytic cleavage of poly (ADP-ribose) polymerase: an early marker of chemotherapy-induced apoptosis","volume":"53","author":"SH Kaufmann","year":"1993","journal-title":"Cancer research"},{"issue":"2","key":"pcbi.1011004.ref029","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/S0960-9822(98)70042-9","article-title":"Dominant-negative FADD inhibits TNFR60-, Fas\/Apo1-and TRAIL-R\/Apo2-mediated cell death but not gene induction","volume":"8","author":"H Wajant","year":"1998","journal-title":"Current biology"},{"issue":"1","key":"pcbi.1011004.ref030","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1471-2407-14-771","article-title":"Controlled delivery of BID protein fused with TAT peptide sensitizes cancer cells to apoptosis","volume":"14","author":"EJ Orzechowska","year":"2014","journal-title":"BMC cancer"},{"key":"pcbi.1011004.ref031","doi-asserted-by":"crossref","first-page":"104518","DOI":"10.1016\/j.envsoft.2019.104518","article-title":"Equifinality, sloppiness, and emergent structures of mechanistic soil biogeochemical models","volume":"122","author":"GL Marschmann","year":"2019","journal-title":"Environmental Modelling & Software"},{"issue":"8","key":"pcbi.1011004.ref032","doi-asserted-by":"crossref","first-page":"3186","DOI":"10.1016\/j.jbusres.2015.12.007","article-title":"Auxiliary theories as translation mechanisms for measurement model specification","volume":"69","author":"L Sajtos","year":"2016","journal-title":"Journal of Business Research"},{"issue":"2","key":"pcbi.1011004.ref033","doi-asserted-by":"crossref","first-page":"04020013","DOI":"10.1061\/(ASCE)CF.1943-5509.0001406","article-title":"Hybrid performance-measurement model of elevators","volume":"34","author":"MU Zubair","year":"2020","journal-title":"Journal of Performance of Constructed Facilities"},{"key":"pcbi.1011004.ref034","doi-asserted-by":"crossref","first-page":"107853","DOI":"10.1016\/j.ijpe.2020.107853","article-title":"A performance measurement system for industry 4.0 enabled smart manufacturing system in SMMEs-A review and empirical investigation","volume":"229","author":"SS Kamble","year":"2020","journal-title":"International journal of production economics"},{"key":"pcbi.1011004.ref035","doi-asserted-by":"crossref","unstructured":"Xia Y, Wang P, Berntorp K, Boufounos P, Orlik P, Svensson L, Granstr\u00f6m K. Extended object tracking with automotive radar using learned structural measurement model. In2020 IEEE Radar Conference (RadarConf20) 2020 Sep 21 (pp. 1\u20136). IEEE.","DOI":"10.1109\/RadarConf2043947.2020.9266598"},{"issue":"1","key":"pcbi.1011004.ref036","doi-asserted-by":"crossref","first-page":"646","DOI":"10.1038\/msb.2013.1","article-title":"Programming biological models in Python using PySB","volume":"9","author":"CF Lopez","year":"2013","journal-title":"Molecular systems biology"},{"key":"pcbi.1011004.ref037","unstructured":"Hindmarsh AC, Petzold LR. LSODA, ordinary differential equation solver for stiff or non-stiff system."},{"issue":"1","key":"pcbi.1011004.ref038","doi-asserted-by":"crossref","first-page":"644","DOI":"10.1038\/msb.2012.69","article-title":"Properties of cell death models calibrated and compared using Bayesian approaches","volume":"9","author":"H Eydgahi","year":"2013","journal-title":"Molecular systems biology"},{"key":"pcbi.1011004.ref039","unstructured":"Antoniuk K, Franc V, Hlav\u00e1\u010d V. Mord: Multi-class classifier for ordinal regression. InMachine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2013, Prague, Czech Republic, September 23\u201327, 2013, Proceedings, Part III 13 2013 (pp. 96\u2013111). Springer Berlin Heidelberg."},{"issue":"1","key":"pcbi.1011004.ref040","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1177\/2515245918823199","article-title":"Ordinal regression models in psychology: A tutorial","volume":"2","author":"PC B\u00fcrkner","year":"2019","journal-title":"Advances in Methods and Practices in Psychological Science"},{"issue":"1","key":"pcbi.1011004.ref041","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1348\/000711008X401314","article-title":"Analysis of ordinal categorical data with misclassification","volume":"63","author":"WY Poon","year":"2010","journal-title":"British Journal of Mathematical and Statistical Psychology"},{"issue":"2","key":"pcbi.1011004.ref042","doi-asserted-by":"crossref","first-page":"e1545","DOI":"10.1002\/wics.1545","article-title":"Ordinal regression: A review and a taxonomy of models","volume":"14","author":"G. Tutz","year":"2022","journal-title":"Wiley Interdisciplinary Reviews: Computational Statistics"},{"key":"pcbi.1011004.ref043","volume-title":"Mastering Machine Learning with scikit-learn","author":"G. Hackeling","year":"2017"},{"issue":"12","key":"pcbi.1011004.ref044","doi-asserted-by":"crossref","first-page":"e9677","DOI":"10.15252\/msb.20209677","article-title":"An incoherent feedforward loop interprets NF\u03baB\/RelA dynamics to determine TNF-induced necroptosis decisions","volume":"16","author":"M Oliver Metzig","year":"2020","journal-title":"Molecular systems biology"},{"key":"pcbi.1011004.ref045","volume-title":"Machine learning: a probabilistic perspective","author":"KP Murphy","year":"2012"},{"key":"pcbi.1011004.ref046","first-page":"13","article-title":"Numerical differentiation\u2014finite differences","volume":"160","author":"P. Moin","year":"2012","journal-title":"Fundam Eng Numer Anal"},{"key":"pcbi.1011004.ref047","volume-title":"Signals and systems: continuous and discrete","author":"RE Ziemer","year":"1993"},{"issue":"4","key":"pcbi.1011004.ref048","doi-asserted-by":"crossref","first-page":"695","DOI":"10.1093\/bioinformatics\/btx626","article-title":"PyDREAM: high-dimensional parameter inference for biological models in python","volume":"34","author":"EM Shockley","year":"2018","journal-title":"Bioinformatics"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1011004","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2023,5,8]],"date-time":"2023-05-08T00:00:00Z","timestamp":1683504000000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011004","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,8]],"date-time":"2023-05-08T18:45:23Z","timestamp":1683571523000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011004"}},"subtitle":[],"editor":[{"given":"Martin","family":"Meier-Schellersheim","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2023,4,26]]},"references-count":48,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,4,26]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1011004","relation":{"new_version":[{"id-type":"doi","id":"10.1371\/journal.pcbi.1011004","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,26]]}}}