{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,6]],"date-time":"2026-07-06T22:24:09Z","timestamp":1783376649656,"version":"3.54.6"},"reference-count":52,"publisher":"Institute for Operations Research and the Management Sciences (INFORMS)","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["INFORMS Journal on Computing"],"published-print":{"date-parts":[[2026,5]]},"abstract":"<jats:p>We propose a new form of predictive models, Partially Interpretable Estimators (PIE), which jointly train an interpretable model and a black-box model to achieve partial model transparency while maintaining high predictive performance. Our design is motivated by prior research showing that interpretability does not require exposing all model details. Therefore, our objective is to explain the main components of the prediction, withholding complicated calculations that may not be necessary for users. PIE is designed to attribute a prediction to the contribution from individual features via a sparse linear additive model to achieve interpretability while complementing the prediction with a black-box model to boost the predictive performance. As such, the linear additive model captures the primary feature contributions, while the black-box component augments PIE\u2019s predictive power by capturing the \u201cnuances\u201d of feature interactions as a refinement. Moreover, we include a sparsity constraint, allowing users to adjust the model to meet domain-specific needs of interpretability. To optimize predictive performance, we propose a coordinated training algorithm that jointly trains the two components of PIE. Experimental results show that PIE achieves accuracy comparable to state-of-the-art black-box models, with human assessments confirming that its interpretability is nearly equivalent to linear models.<\/jats:p>\n                  <jats:p>History: Accepted by Ram Ramesh, Area Editor for Data Science &amp; Machine Learning.<\/jats:p>\n                  <jats:p>Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https:\/\/pubsonline.informs.org\/doi\/suppl\/10.1287\/ijoc.2022.0098 ) as well as from the IJOC GitHub software repository ( https:\/\/github.com\/INFORMSJoC\/2022.0098 ). The complete IJOC Software and Data Repository is available at https:\/\/informsjoc.github.io\/ .<\/jats:p>","DOI":"10.1287\/ijoc.2022.0098","type":"journal-article","created":{"date-parts":[[2025,7,7]],"date-time":"2025-07-07T10:28:20Z","timestamp":1751884100000},"page":"766-782","source":"Crossref","is-referenced-by-count":1,"title":["PIE\u2014Partially Interpretable Estimators with Refinement"],"prefix":"10.1287","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8687-4208","authenticated-orcid":false,"given":"Tong","family":"Wang","sequence":"first","affiliation":[{"name":"Yale School of Management, Yale University, New Haven, Connecticut 06511"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-7809-3097","authenticated-orcid":false,"given":"Jingyi","family":"Yang","sequence":"additional","affiliation":[{"name":"Leonard N. Stern School of Business, Tisch Hall, New York University, New York, New York 10012"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8522-7207","authenticated-orcid":false,"given":"Yunyi","family":"Li","sequence":"additional","affiliation":[{"name":"McCombs School of Business, The University of Texas at Austin, Austin, Texas 78705;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4187-552X","authenticated-orcid":false,"given":"Boxiang","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Statistics and Actuarial Science, The University of Iowa, Iowa City, Iowa 52242"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"109","reference":[{"key":"B1","volume":"31","author":"Adebayo J","year":"2018","journal-title":"Adv. Neural Inform. Processing Systems"},{"key":"B2","unstructured":"Agarwal R, Frosst N, Zhang X, Caruana R, Hinton GE (2020) Neural additive models: Interpretable machine learning with neural nets. Preprint, submitted April 29, https:\/\/arxiv.org\/abs\/2004.13912."},{"key":"B3","unstructured":"A\u00efvodji U, Arai H, Fortineau O, Gambs S, Hara S, Tapp A (2019) Fairwashing: The risk of rationalization. Preprint, submitted January 28, https:\/\/arxiv.org\/abs\/1901.09749."},{"key":"B4","doi-asserted-by":"crossref","unstructured":"Caruana R, Lou Y, Gehrke J, Koch P, Sturm M, Elhadad N (2015) Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission.\n                      Proc. 21st ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining\n                      (ACM, New York), 1721\u20131730.","DOI":"10.1145\/2783258.2788613"},{"key":"B5","doi-asserted-by":"crossref","unstructured":"Chen T, Guestrin C (2016) XGBoost: A scalable tree boosting system.\n                      Proc. 22nd ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining\n                      (ACM, New York), 785\u2013794.","DOI":"10.1145\/2939672.2939785"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1111\/j.0006-341X.2001.00539.x"},{"key":"B7","unstructured":"Dhurandhar A, Shanmugam K, Luss R (2020) Enhancing simple models by exploiting what they already know.\n                      Internat. Conf. Machine Learning\n                      (PMLR, New York), 2525\u20132534."},{"key":"B8","unstructured":"Doshi-Velez F, Kim B (2017) Towards a rigorous science of interpretable machine learning. Preprint, submitted February 28, https:\/\/arxiv.org\/abs\/1702.08608."},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1016218223"},{"key":"B10","unstructured":"Garcez A, Besold TR, De Raedt L, F\u00f6ldiak P, Hitzler P, Icard T, K\u00fchnberger KU, Lamb LC, Miikkulainen R, Silver DL (2015) Neural-symbolic learning and reasoning: Contributions and challenges.\n                      Proc. AAAI Spring Sympos. Knowledge Representation Reasoning Integrating Symbolic Neural Approaches, Stanford\n                      ."},{"key":"B11","doi-asserted-by":"crossref","unstructured":"Gilpin LH, Bau D, Yuan BZ, Bajwa A, Specter M, Kagal L (2018) Explaining explanations: An overview of interpretability of machine learning.\n                      IEEE 5th Internat. Conf. Data Sci. Advanced Analytics (DSAA)\n                      (IEEE, Piscataway, NJ), 80\u201389.","DOI":"10.1109\/DSAA.2018.00018"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.52202\/068431-0037"},{"key":"B13","unstructured":"Guo M, Zhang Q, Liao X, Chen Y (2019) An interpretable machine learning framework for modeling human decision behavior. Preprint, submitted June 4, https:\/\/arxiv.org\/abs\/1906.01233."},{"key":"B14","volume-title":"Generalized Additive Models","volume":"43","author":"Hastie TJ","year":"1990"},{"key":"B15","unstructured":"Higgins I, Matthey L, Pal A, Burgess C, Glorot X, Botvinick M, Mohamed S, Lerchner A (2017) beta-VAE: Learning basic visual concepts with a constrained variational framework.\n                      Internat. Conf. Learn. Representations\n                      (OpenReview.net)."},{"key":"B16","unstructured":"Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. Preprint, submitted March 9, https:\/\/arxiv.org\/abs\/1503.02531."},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1016\/j.amc.2006.01.064"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.159"},{"key":"B19","unstructured":"Kelly M, Longjohn R, Nottingham K (2025) UCI machine learning repository. http:\/\/archive.ics.uci.edu\/ml."},{"key":"B20","unstructured":"Kohavi R (1996) Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid.\n                      Proc. 2nd Internat. Conf. Knowledge Discovery Data Mining\n                      (AAAI Press, Palo Alto, CA), 202\u2013207."},{"key":"B21","unstructured":"Lakkaraju H, Kamar E, Caruana R, Leskovec J (2017) Interpretable & explorable approximations of black box models. Preprint, submitted July 4, https:\/\/arxiv.org\/abs\/1707.01154."},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.abk1942"},{"key":"B23","doi-asserted-by":"crossref","unstructured":"Lou Y, Caruana R, Gehrke J, Hooker G (2013) Accurate intelligible models with pairwise interactions.\n                      Proc. 19th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining\n                      (ACM, New York), 623\u2013631.","DOI":"10.1145\/2487575.2487579"},{"key":"B24","unstructured":"Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions.\n                      Proc. 31st Internat. Conf. Neural Inform. Processing Systems\n                      (Curran Associates Inc., Red Hook, NY), 4768\u20134777."},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-9473(98)00033-4"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1017\/dap.2021.3"},{"key":"B27","doi-asserted-by":"crossref","unstructured":"Neumeier M, Botsch M, Tollk\u00fchn A, Berberich T (2021) Variational autoencoder-based vehicle trajectory prediction with an interpretable latent space.\n                      IEEE Internat. Intelligent Transportation Systems Conf. (ITSC)\n                      (IEEE, Piscataway, NJ), 820\u2013827.","DOI":"10.1109\/ITSC48978.2021.9565120"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2009.191"},{"key":"B29","unstructured":"Pan D, Wang T, Hara S (2020) Interpretable companions for black-box models.\n                      Internat. Conf. Artificial Intelligence Statistics\n                      (PMLR, New York), 2444\u20132454."},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1561\/2400000003"},{"key":"B31","unstructured":"Parliament of the European Union Council (2016) General Data Protection Regulation. https:\/\/gdpr-info.eu\/."},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1007\/s00439-018-1919-7"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.3390\/a13010017"},{"key":"B34","doi-asserted-by":"crossref","unstructured":"Ragodos R, Wang T, Feng L (2024) From model explanation to data misinterpretation: Uncovering the pitfalls of post hoc explainers in business research. Preprint, submitted August 30, https:\/\/arxiv.org\/pdf\/2408.16987?","DOI":"10.2139\/ssrn.4915307"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2021.110600"},{"key":"B36","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C (2016) Why should I trust you? Explaining the predictions of any classifier.\n                      Proc. 22nd ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining\n                      (ACM, New York), 1135\u20131144.","DOI":"10.1145\/2939672.2939778"},{"key":"B37","unstructured":"Richards J, Huser R (2022) A unifying partially-interpretable framework for neural network-based extreme quantile regression. Preprint, submitted August 16, https:\/\/arxiv.org\/abs\/2208.07581."},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40167-1"},{"key":"B39","doi-asserted-by":"crossref","unstructured":"Ross AS, Hughes MC, Doshi-Velez F (2017) Right for the right reasons: Training differentiable models by constraining their explanations. Preprint, submitted March 10, https:\/\/arxiv.org\/abs\/1703.03717.","DOI":"10.24963\/ijcai.2017\/371"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0048-x"},{"key":"B41","unstructured":"Samek W, Wiegand T, M\u00fcller KR (2017) Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. Preprint, submitted August 28, https:\/\/arxiv.org\/abs\/1708.08296."},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.11.011"},{"key":"B43","unstructured":"Simonyan K, Vedaldi A, Zisserman A (2013) Deep inside convolutional networks: Visualising image classification models and saliency maps. Preprint, submitted December 20, https:\/\/arxiv.org\/abs\/1312.6034."},{"key":"B44","doi-asserted-by":"crossref","unstructured":"Slack D, Hilgard S, Jia E, Singh S, Lakkaraju H (2019) Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods. Preprint, submitted November 6, https:\/\/arxiv.org\/abs\/1911.02508.","DOI":"10.1145\/3375627.3375830"},{"key":"B45","first-page":"841","volume":"31","author":"Wachter S","year":"2017","journal-title":"Harvard J. Law Tech."},{"issue":"137","key":"B46","first-page":"1","volume":"22","author":"Wang T","year":"2021","journal-title":"J. Machine Learn. Res."},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1002\/sta4.582"},{"key":"B48","doi-asserted-by":"crossref","unstructured":"Wang T, Yang J, Li Y, Wang B (2025) PIE - Partially interpretable estimators with refinement. https:\/\/doi.org\/10.1287\/ijoc.2022.0098.cd, https:\/\/github.com\/INFORMSJoC\/2022.0098.","DOI":"10.1287\/ijoc.2022.0098.cd"},{"issue":"1","key":"B49","first-page":"2357","volume":"18","author":"Wang T","year":"2017","journal-title":"J. Machine Learn. Res."},{"key":"B50","doi-asserted-by":"crossref","unstructured":"Wu J, Tenenbaum JB, Kohli P (2017) Neural scene de-rendering.\n                      Proc. IEEE Conf. Comput. Vision Pattern Recognition (Honolulu)\n                      , 699\u2013707.","DOI":"10.1109\/CVPR.2017.744"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2005.00532.x"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1111\/rssa.12227"}],"container-title":["INFORMS Journal on Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubsonline.informs.org\/doi\/pdf\/10.1287\/ijoc.2022.0098","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T08:52:46Z","timestamp":1781513566000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubsonline.informs.org\/doi\/10.1287\/ijoc.2022.0098"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5]]},"references-count":52,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,5]]}},"alternative-id":["10.1287\/ijoc.2022.0098"],"URL":"https:\/\/doi.org\/10.1287\/ijoc.2022.0098","relation":{},"ISSN":["1091-9856","1526-5528"],"issn-type":[{"value":"1091-9856","type":"print"},{"value":"1526-5528","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5]]}}}