{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T23:59:09Z","timestamp":1740182349299,"version":"3.37.3"},"reference-count":41,"publisher":"IOP Publishing","issue":"4","license":[{"start":{"date-parts":[[2021,9,17]],"date-time":"2021-09-17T00:00:00Z","timestamp":1631836800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,9,17]],"date-time":"2021-09-17T00:00:00Z","timestamp":1631836800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"Marie Sklodowska-Curie Grant Agreement","award":["801110"],"award-info":[{"award-number":["801110"]}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2021,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>In the setting of regression, the standard formulation of gradient boosting generates a sequence of improvements to a constant model. In this paper, we reformulate gradient boosting such that it is able to generate a sequence of improvements to a nonconstant model, which may contain prior knowledge or physical insight about the data generating process. Moreover, we introduce a simple variant of multi-target stacking that extends our approach to the setting of multi-target regression. An experiment on a real-world superconducting quantum device calibration dataset demonstrates that our approach outperforms the state-of-the-art calibration model even though it only receives a paucity of training examples. Further, it significantly outperforms a well-known gradient boosting algorithm, known as LightGBM, as well as an entirely data-driven reimplementation of the calibration model, which suggests the viability of our approach.<\/jats:p>","DOI":"10.1088\/2632-2153\/ac1ee9","type":"journal-article","created":{"date-parts":[[2021,8,18]],"date-time":"2021-08-18T22:27:13Z","timestamp":1629325633000},"page":"045022","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["A new formulation of gradient boosting"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2867-1175","authenticated-orcid":false,"given":"Alex","family":"Wozniakowski","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3746-244X","authenticated-orcid":false,"given":"Jayne","family":"Thompson","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5459-4313","authenticated-orcid":false,"given":"Mile","family":"Gu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4483-5643","authenticated-orcid":false,"given":"Felix C","family":"Binder","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2021,9,17]]},"reference":[{"year":"1977","author":"Tukey","key":"mlstac1ee9bib1"},{"article-title":"Learning boolean formulae or finite automata is as hard as factoring","year":"1988","author":"Kearns","key":"mlstac1ee9bib2"},{"key":"mlstac1ee9bib3","doi-asserted-by":"publisher","first-page":"3397","DOI":"10.1109\/78.258082","article-title":"Matching pursuits with time-frequency dictionaries","volume":"41","author":"Mallat","year":"1993","journal-title":"IEEE Trans. Signal Process."},{"key":"mlstac1ee9bib4","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1006\/jcss.1997.1504","article-title":"A decision-theoretic generalization of on-line learning and an application to boosting","volume":"55","author":"Freund","year":"1997","journal-title":"J. Comput. Syst. Sci."},{"article-title":"Arcing the edge","year":"1997","author":"Breiman","key":"mlstac1ee9bib5"},{"key":"mlstac1ee9bib6","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1214\/aos\/1016218223","article-title":"Additive logistic regression: a statistical view of boosting","volume":"28","author":"Friedman","year":"2000","journal-title":"Ann. Stat."},{"key":"mlstac1ee9bib7","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: a gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Stat."},{"key":"mlstac1ee9bib8","first-page":"pp 538","article-title":"Incorporating prior knowledge into boosting","author":"Schapire","year":"2002"},{"key":"mlstac1ee9bib9","doi-asserted-by":"publisher","first-page":"477","DOI":"10.1214\/07-STS242","article-title":"Boosting algorithms: regularization, prediction and model fitting","volume":"22","author":"B\u00fchlmann","year":"2007","journal-title":"Stat. Sci."},{"key":"mlstac1ee9bib10","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1198\/016214503000125","article-title":"Boosting with the l 2 loss","volume":"98","author":"B\u00fchlmann","year":"2011","journal-title":"J. Am. Stat. Assoc."},{"year":"2009","author":"Hastie","key":"mlstac1ee9bib11"},{"key":"mlstac1ee9bib12","first-page":"pp 361","article-title":"Estimation with quadratic loss","volume":"vol 4.1","author":"James","year":"1961"},{"key":"mlstac1ee9bib13","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked generalization","volume":"5","author":"Wolpert","year":"1992","journal-title":"Neural Netw."},{"key":"mlstac1ee9bib14","doi-asserted-by":"publisher","first-page":"1691","DOI":"10.1016\/0005-1098(95)00120-8","article-title":"Nonlinear black-box modeling in system identification: a unified overview","volume":"31","author":"Sj\u00f6berg","year":"1995","journal-title":"Automatica"},{"key":"mlstac1ee9bib15","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1007\/BF00117832","article-title":"Stacked regressions","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn."},{"key":"mlstac1ee9bib16","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1111\/1467-9868.00054","article-title":"Predicting multivariate responses in multiple linear regression","volume":"59","author":"Breiman","year":"1997","journal-title":"R. Stat. Soc. Ser. B"},{"key":"mlstac1ee9bib17","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask learning","volume":"28","author":"Caruana","year":"1997","journal-title":"Mach. Learn."},{"key":"mlstac1ee9bib18","first-page":"625","article-title":"Why does unsupervised pre-training help deep learning?","volume":"11","author":"Erhan","year":"2010","journal-title":"J. Mach. Learn. Res."},{"article-title":"Towards an automated classification of transient events in synoptic sky surveys","year":"2011","author":"Djorgovski","key":"mlstac1ee9bib19"},{"key":"mlstac1ee9bib20","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1017\/S1743921312001056","article-title":"Real time classification of transient events in synoptic sky surveys","volume":"7","author":"Mahabal","year":"2011","journal-title":"Proc. Int. Astron. Union"},{"key":"mlstac1ee9bib21","first-page":"387","article-title":"Discovery, classification and scientific exploration of transient events from the Catalina Real-time Transient Survey","volume":"39","author":"Mahabal","year":"2011","journal-title":"Bull. Astron. Soc. India"},{"key":"mlstac1ee9bib22","doi-asserted-by":"publisher","first-page":"216","DOI":"10.1002\/widm.1157","article-title":"A survey on multi-output regression","volume":"5","author":"Borchani","year":"2015","journal-title":"Data Mining and Knowledge Discovery"},{"key":"mlstac1ee9bib23","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1007\/s10994-016-5546-z","article-title":"Multi-target regression via input space expansion: treating targets as inputs","volume":"104","author":"Spyromitros-Xioufis","year":"2016","journal-title":"Mach. Learn."},{"key":"mlstac1ee9bib24","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1007\/s10618-018-0595-5","article-title":"Multi-target prediction: a unifying view on problems and methods","volume":"33","author":"Waegeman","year":"2019","journal-title":"Data Min. Knowl. Discovery"},{"key":"mlstac1ee9bib25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1088\/1361-6420\/ab7d2c","article-title":"Non-unique games over compact groups and orientation estimation in cryo-EM","volume":"36","author":"Chen","year":"2020","journal-title":"Inverse Problems"},{"key":"mlstac1ee9bib26","doi-asserted-by":"publisher","first-page":"1175","DOI":"10.1126\/science.aao1401","article-title":"Spectroscopic signatures of localization with interacting photons in superconducting qubits","volume":"358","author":"Roushan","year":"2017","journal-title":"Science"},{"key":"mlstac1ee9bib27","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1126\/science.aao4309","article-title":"A blueprint for demonstrating quantum supremacy with superconducting qubits","volume":"360","author":"Neill","year":"2018","journal-title":"Science"},{"article-title":"Growth and preservation of entanglement in a many-body localized system","year":"2019","author":"Chiaro","key":"mlstac1ee9bib28"},{"key":"mlstac1ee9bib29","first-page":"3149","article-title":"LightGBM: a highly efficient gradient boosting decision tree","author":"Guolin","year":"2017"},{"key":"mlstac1ee9bib30","first-page":"4765","article-title":"A unified approach to interpreting model predictions","author":"Lundberg","year":"2017"},{"key":"mlstac1ee9bib31","first-page":"pp 1697","article-title":"A general boosting method and its application to learning ranking functions for web search","author":"Zheng","year":"2007"},{"key":"mlstac1ee9bib32","first-page":"pp 1209","article-title":"Generalized boosting algorithms for convex optimization","author":"Grubb","year":"2011"},{"key":"mlstac1ee9bib33","doi-asserted-by":"publisher","first-page":"2087","DOI":"10.1021\/acs.jctc.5b00099","article-title":"Big data meets quantum chemistry approximations: the \u0394-machine learning approach","volume":"11","author":"Ramakrishnan","year":"2015","journal-title":"J. Chem. Theory Computat."},{"key":"mlstac1ee9bib34","first-page":"2825","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"mlstac1ee9bib35","first-page":"pp 512","article-title":"Boosting algorithms as gradient descent","author":"Mason","year":"1999"},{"key":"mlstac1ee9bib36","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1023\/A:1013955821559","article-title":"Kernel matching pursuit","volume":"48","author":"Vincent","year":"2002","journal-title":"Mach. Learn."},{"key":"mlstac1ee9bib37","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1109\/MSP.2011.2179329","article-title":"A tour of modern image filtering: new insights and methods, both practical and theoretical","volume":"30","author":"Milanfar","year":"2013","journal-title":"IEEE Signal Process. Mag."},{"key":"mlstac1ee9bib38","doi-asserted-by":"publisher","first-page":"1187","DOI":"10.1137\/140990978","article-title":"Boosting of image denoising algorithms","volume":"8","author":"Romano","year":"2015","journal-title":"SIAM J. Imaging Sci."},{"article-title":"Gradient and newton boosting for classification and regression","year":"2018","author":"Sigrist","key":"mlstac1ee9bib39"},{"article-title":"Scikit-physlearn","year":"2020","author":"Wozniakowski","key":"mlstac1ee9bib40"},{"article-title":"Scikit-physlearn manual","year":"2020","author":"Wozniakowski","key":"mlstac1ee9bib41"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac1ee9","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac1ee9\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac1ee9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac1ee9\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac1ee9\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac1ee9\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac1ee9\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac1ee9\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,12]],"date-time":"2021-12-12T17:48:34Z","timestamp":1639331314000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac1ee9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,17]]},"references-count":41,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,9,17]]},"published-print":{"date-parts":[[2021,12,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ac1ee9","relation":{},"ISSN":["2632-2153"],"issn-type":[{"type":"electronic","value":"2632-2153"}],"subject":[],"published":{"date-parts":[[2021,9,17]]},"assertion":[{"value":"A new formulation of gradient boosting","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2021 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2021-08-04","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-08-18","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-09-17","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}