{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T22:24:13Z","timestamp":1759962253832},"reference-count":61,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2022,9,21]],"date-time":"2022-09-21T00:00:00Z","timestamp":1663718400000},"content-version":"vor","delay-in-days":263,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,9,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>This paper presents Diff-Explainer, the first hybrid framework for explainable multi-hop inference that integrates explicit constraints with neural architectures through differentiable convex optimization. Specifically, Diff- Explainer allows for the fine-tuning of neural representations within a constrained optimization framework to answer and explain multi-hop questions in natural language. To demonstrate the efficacy of the hybrid framework, we combine existing ILP-based solvers for multi-hop Question Answering (QA) with Transformer-based representations. An extensive empirical evaluation on scientific and commonsense QA tasks demonstrates that the integration of explicit constraints in a end-to-end differentiable framework can significantly improve the performance of non- differentiable ILP solvers (8.91%\u201313.3%). Moreover, additional analysis reveals that Diff-Explainer is able to achieve strong performance when compared to standalone Transformers and previous multi-hop approaches while still providing structured explanations in support of its predictions.<\/jats:p>","DOI":"10.1162\/tacl_a_00508","type":"journal-article","created":{"date-parts":[[2022,9,21]],"date-time":"2022-09-21T18:04:11Z","timestamp":1663783451000},"page":"1103-1119","update-policy":"http:\/\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":2,"title":["Diff-Explainer: Differentiable Convex Optimization for Explainable Multi-hop Inference"],"prefix":"10.1162","volume":"10","author":[{"given":"Mokanarangan","family":"Thayaparan","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Manchester, United Kingdom. mokanarangan.thayaparan@manchester.ac.uk"},{"name":"Idiap Research Institute, Switzerland"}]},{"given":"Marco","family":"Valentino","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Manchester, United Kingdom. marco.valentino@manchester.ac.uk"},{"name":"Idiap Research Institute, Switzerland"}]},{"given":"Deborah","family":"Ferreira","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Manchester, United Kingdom. deborah.ferreira@manchester.ac.uk"}]},{"given":"Julia","family":"Rozanova","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Manchester, United Kingdom. julia.rozanova@manchester.ac.uk"}]},{"given":"Andr\u00e9","family":"Freitas","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Manchester, United Kingdom. andre.freitas@manchester.ac.uk"},{"name":"Idiap Research Institute, Switzerland"}]}],"member":"281","published-online":{"date-parts":[[2022,9,19]]},"reference":[{"issue":"2","key":"2022092118021274700_bib1","doi-asserted-by":"publisher","first-page":"107","DOI":"10.23952\/jano.1.2019.2.02","article-title":"Differentiating through a cone program","volume":"1","author":"Agrawal","year":"2019","journal-title":"Journal of Applied and Numerical Optimization"},{"key":"2022092118021274700_bib2","article-title":"Differentiable convex optimization layers","volume-title":"Advances in Neural Information Processing Systems","author":"Agrawal","year":"2019"},{"key":"2022092118021274700_bib3","first-page":"136","article-title":"Optnet: Differentiable optimization as a layer in neural networks","volume-title":"International Conference on Machine Learning","author":"Amos","year":"2017"},{"issue":"3","key":"2022092118021274700_bib4","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1007\/s10589-019-00122-9","article-title":"Solution refinement at regular points of conic problems","volume":"74","author":"Busseti","year":"2019","journal-title":"Computational Optimization and Applications"},{"key":"2022092118021274700_bib5","doi-asserted-by":"publisher","first-page":"818","DOI":"10.3115\/v1\/P14-1077","article-title":"Encoding relation requirements for relation extraction via joint inference","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Chen","year":"2014"},{"key":"2022092118021274700_bib6","doi-asserted-by":"publisher","first-page":"431","DOI":"10.3115\/1610075.1610136","article-title":"Joint extraction of entities and relations for opinion recognition","volume-title":"Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing","author":"Choi","year":"2006"},{"key":"2022092118021274700_bib7","doi-asserted-by":"publisher","first-page":"590","DOI":"10.3115\/1699571.1699590","article-title":"Adapting a polarity lexicon using integer linear programming for domain-specific sentiment classification","volume-title":"Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing","author":"Choi","year":"2009"},{"key":"2022092118021274700_bib8","article-title":"Think you have solved question answering? Try arc, the AI2 reasoning challenge","author":"Clark","year":"2018","journal-title":"arXiv preprint arXiv: 1803.05457"},{"key":"2022092118021274700_bib9","doi-asserted-by":"publisher","first-page":"2580","DOI":"10.1609\/aaai.v30i1.10325","article-title":"Combining retrieval, statistics, and inference to answer elementary science questions.","volume-title":"AAAI","author":"Clark","year":"2016"},{"key":"2022092118021274700_bib10","doi-asserted-by":"publisher","first-page":"3882","DOI":"10.24963\/ijcai.2020\/537","article-title":"Transformers as soft reasoners over language","volume-title":"Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence","author":"Clark","year":"2021"},{"key":"2022092118021274700_bib11","volume-title":"Aspects of Semidefinite Programming: Interior Point Algorithms and Selected Applications","author":"De Klerk","year":"2006"},{"key":"2022092118021274700_bib12","article-title":"Transforming question answering datasets into natural language inference datasets","author":"Demszky","year":"2018","journal-title":"arXiv preprint arXiv:1809.02922"},{"key":"2022092118021274700_bib13","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2022092118021274700_bib14","first-page":"1013","article-title":"Differentiable learning of submodular models","volume":"30","author":"Djolonga","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2022092118021274700_bib15","article-title":"Task-based end-to-end model learning in stochastic optimization","author":"Donti","year":"2017","journal-title":"arXiv preprint arXiv: 1703.04529"},{"key":"2022092118021274700_bib16","first-page":"22231","article-title":"Measuring systematic generalization in neural proof generation with transformers","volume":"33","author":"Gontier","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"5","key":"2022092118021274700_bib17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3236009","article-title":"A survey of methods for explaining black box models","volume":"51","author":"Guidotti","year":"2018","journal-title":"ACM Computing Surveys (CSUR)"},{"key":"2022092118021274700_bib18","doi-asserted-by":"publisher","first-page":"109","DOI":"10.18653\/v1\/2020.textgraphs-1.13","article-title":"Explanation regeneration via multi-hop ILP inference over knowledge base","volume-title":"Proceedings of the Graph-based Methods for Natural Language Processing (TextGraphs)","author":"Gupta","year":"2020"},{"key":"2022092118021274700_bib19","volume-title":"Semidefinite Programming for Combinatorial Optimization","author":"Helmberg","year":"2000"},{"key":"2022092118021274700_bib20","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-1703","article-title":"Multi-hop inference for sentence-level textgraphs: How challenging is meaningfully combining information for science question answering?","author":"Jansen","year":"2018","journal-title":"arXiv preprint arXiv:1805.11267"},{"key":"2022092118021274700_bib21","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.textgraphs-1.17","article-title":"Textgraphs 2021 shared task on multi-hop inference for explanation regeneration","volume-title":"Proceedings of the Fifteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-15)","author":"Jansen","year":"2021"},{"key":"2022092118021274700_bib22","article-title":"Worldtree: A corpus of explanation graphs for elementary science questions supporting multi- hop inference","volume-title":"Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)","author":"Jansen","year":"2018"},{"key":"2022092118021274700_bib23","article-title":"Billion-scale similarity search with GPUs","author":"Johnson","year":"2017","journal-title":"CoRR"},{"key":"2022092118021274700_bib24","doi-asserted-by":"publisher","first-page":"302","DOI":"10.1145\/800057.808695","article-title":"A new polynomial- time algorithm for linear programming","volume-title":"Proceedings of the Sixteenth Annual ACM Symposium on Theory of Computing","author":"Karmarkar","year":"1984"},{"key":"2022092118021274700_bib25","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1007\/978-1-4684-2001-2_9","article-title":"Reducibility among combinatorial problems","volume-title":"Complexity of computer computations","author":"Karp","year":"1972"},{"key":"2022092118021274700_bib26","first-page":"1145","article-title":"Question answering via integer programming over semi-structured knowledge","volume-title":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence","author":"Khashabi","year":"2016"},{"key":"2022092118021274700_bib27","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11574","article-title":"Question answering as global reasoning over semantic abstractions","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Khashabi","year":"2018"},{"key":"2022092118021274700_bib28","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.171","article-title":"UnifiedQA: Crossing format boundaries with a single QA system","author":"Khashabi","year":"2020","journal-title":"arXiv preprint arXiv:2005.00700"},{"key":"2022092118021274700_bib29","doi-asserted-by":"publisher","first-page":"8082","DOI":"10.1609\/aaai.v34i05.6319","article-title":"Qasc: A dataset for question answering via sentence composition","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Khot","year":"2020"},{"key":"2022092118021274700_bib30","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-2049","article-title":"Answering complex questions using open information extraction","author":"Khot","year":"2017","journal-title":"arXiv preprint arXiv:1704.05572"},{"key":"2022092118021274700_bib31","doi-asserted-by":"publisher","first-page":"181","DOI":"10.18653\/v1\/P17-2049","article-title":"Generalized inference with multiple semantic role labeling systems","volume-title":"Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005)","author":"Koomen","year":"2005"},{"key":"2022092118021274700_bib32","doi-asserted-by":"publisher","first-page":"2737","DOI":"10.18653\/v1\/P19-1263","article-title":"Exploiting explicit paths for multi-hop reading comprehension","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Kundu","year":"2019"},{"key":"2022092118021274700_bib33","doi-asserted-by":"publisher","first-page":"168","DOI":"10.1016\/j.neucom.2020.08.011","article-title":"Explaining the black-box model: A survey of local interpretation methods for deep neural networks","volume":"419","author":"Liang","year":"2021","journal-title":"Neurocomputing"},{"key":"2022092118021274700_bib34","doi-asserted-by":"publisher","first-page":"166","DOI":"10.1137\/0801013","article-title":"Cones of matrices and set-functions and 0-1 optimization","volume":"1","author":"Lov\u00e1sz","year":"1991","journal-title":"SIAM Journal on Optimization"},{"key":"2022092118021274700_bib35","first-page":"335","article-title":"Learning to attend on essential terms: An enhanced retriever-reader model for open-domain question answering","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Ni","year":"2019"},{"key":"2022092118021274700_bib36","doi-asserted-by":"publisher","first-page":"1999","DOI":"10.1137\/20M1366307","article-title":"Operator splitting for a homogeneous embedding of the linear complementarity problem","volume":"31","author":"O\u2019Donoghue","year":"2021","journal-title":"SIAM Journal on Optimization"},{"key":"2022092118021274700_bib37","article-title":"Comboptnet: Fit the right np-hard problem by learning integer programming constraints","author":"Paulus","year":"2021","journal-title":"CoRR"},{"key":"2022092118021274700_bib38","doi-asserted-by":"publisher","first-page":"1346","DOI":"10.3115\/1220355.1220552","article-title":"Semantic role labeling via integer linear programming inference","volume-title":"COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics","author":"Punyakanok","year":"2004"},{"key":"2022092118021274700_bib39","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1410","article-title":"Sentence-bert: Sentence embeddings using siamese bert-networks","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing","author":"Reimers","year":"2019"},{"key":"2022092118021274700_bib40","first-page":"1","article-title":"A linear programming formulation for global inference in natural language tasks","volume-title":"Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004","author":"Roth","year":"2004"},{"issue":"5","key":"2022092118021274700_bib41","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nature Machine Intelligence"},{"key":"2022092118021274700_bib42","doi-asserted-by":"crossref","first-page":"122","DOI":"10.18653\/v1\/2020.emnlp-main.9","article-title":"Prover: Proof generation for interpretable reasoning over rules","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Saha","year":"2020"},{"key":"2022092118021274700_bib43","volume-title":"Theory of Linear and Integer Programming","author":"Schrijver","year":"1998"},{"key":"2022092118021274700_bib44","article-title":"Mpnet: Masked and permuted pre-training for language understanding","author":"Song","year":"2020","journal-title":"CoRR"},{"key":"2022092118021274700_bib45","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.11164","article-title":"Conceptnet 5.5: An open multilingual graph of general knowledge","author":"Speer","year":"2016","journal-title":"CoRR"},{"key":"2022092118021274700_bib46","doi-asserted-by":"publisher","first-page":"885","DOI":"10.18653\/v1\/N18-1081","article-title":"Supervised open information extraction","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Stanovsky","year":"2018"},{"key":"2022092118021274700_bib47","doi-asserted-by":"publisher","first-page":"3621","DOI":"10.18653\/v1\/2021.findings-acl.317","article-title":"Proofwriter: Generating implications, proofs, and abductive statements over natural language","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Tafjord","year":"2021"},{"key":"2022092118021274700_bib48","article-title":"A survey on explainability in machine reading comprehension","author":"Thayaparan","year":"2020","journal-title":"arXiv preprint arXiv:2010.00389"},{"key":"2022092118021274700_bib49","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/2021.findings-acl.1","article-title":"Explainable inference over grounding-abstract chains for science questions","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Thayaparan","year":"2021"},{"key":"2022092118021274700_bib50","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/379","article-title":"Differentiable submodular maximization","author":"Tschiatschek","year":"2018","journal-title":"arXiv preprint arXiv: 1803.01785"},{"key":"2022092118021274700_bib51","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i10.21392","article-title":"Hybrid autoregressive inference for scalable multi-hop explanation regeneration","volume-title":"36th AAAI Conference on Artificial Intelligence","author":"Valentino","year":"2021"},{"issue":"1","key":"2022092118021274700_bib52","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1137\/1038003","article-title":"Semidefinite programming","volume":"38","author":"Vandenberghe","year":"1996","journal-title":"SIAM review"},{"key":"2022092118021274700_bib53","first-page":"6545","article-title":"Satnet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver","volume-title":"International Conference on Machine Learning","author":"Wang","year":"2019"},{"key":"2022092118021274700_bib54","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6","article-title":"Huggingface\u2019s transformers: State-of-the-art natural language processing","author":"Wolf","year":"2019","journal-title":"CoRR"},{"key":"2022092118021274700_bib55","doi-asserted-by":"publisher","DOI":"10.1002\/9781119606475","volume-title":"Integer Programming","author":"Wolsey","year":"2020"},{"key":"2022092118021274700_bib56","first-page":"5456","article-title":"Worldtree v2: A corpus of science-domain structured explanations and inference patterns supporting multi-hop inference","volume-title":"Proceedings of The 12th Language Resources and Evaluation Conference","author":"Xie","year":"2020"},{"key":"2022092118021274700_bib57","first-page":"1044","article-title":"Dynamic semantic graph construction and reasoning for explainable multi-hop science question answering","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Weiwen","year":"2021"},{"key":"2022092118021274700_bib58","doi-asserted-by":"publisher","first-page":"2681","DOI":"10.18653\/v1\/N19-1274","article-title":"Alignment over heterogeneous embeddings for question answering","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Yadav","year":"2019"},{"key":"2022092118021274700_bib59","doi-asserted-by":"publisher","first-page":"2578","DOI":"10.18653\/v1\/D19-1260","article-title":"Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Yadav","year":"2019"},{"key":"2022092118021274700_bib60","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.414","article-title":"Unsupervised alignment- based iterative evidence retrieval for multi-hop question answering","author":"Yadav","year":"2020","journal-title":"arXiv preprint arXiv:2005 .01218"},{"key":"2022092118021274700_bib61","article-title":"Kg 2: Learning to reason science exam questions with contextual knowledge graph embeddings","author":"Zhang","year":"2018","journal-title":"arXiv preprint arXiv :1805.12393"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00508\/2043744\/tacl_a_00508.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00508\/2043744\/tacl_a_00508.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,9,21]],"date-time":"2022-09-21T18:04:55Z","timestamp":1663783495000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00508\/113021\/Diff-Explainer-Differentiable-Convex-Optimization"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022]]},"references-count":61,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00508","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022]]},"published":{"date-parts":[[2022]]}}}