{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T08:59:11Z","timestamp":1782377951621,"version":"3.54.5"},"reference-count":53,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T00:00:00Z","timestamp":1699833600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"publisher","award":["FA9550-19-1-039"],"award-info":[{"award-number":["FA9550-19-1-039"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>We consider the problem of learning with sensitive features under the privileged information setting where the goal is to learn a classifier that uses features not available (or too sensitive to collect) at test\/deployment time to learn a better model at training time. We focus on tree-based learners, specifically gradient-boosted decision trees for learning with privileged information. Our methods use privileged features as knowledge to guide the algorithm when learning from fully observed (usable) features. We derive the theory, empirically validate the effectiveness of our algorithms, and verify them on standard fairness metrics.<\/jats:p>","DOI":"10.3389\/frai.2023.1260583","type":"journal-article","created":{"date-parts":[[2023,11,14]],"date-time":"2023-11-14T02:21:12Z","timestamp":1699928472000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Learning with privileged and sensitive information: a gradient-boosting approach"],"prefix":"10.3389","volume":"6","author":[{"given":"Siwen","family":"Yan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Phillip","family":"Odom","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rahul","family":"Pasunuri","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristian","family":"Kersting","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sriraam","family":"Natarajan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,11,13]]},"reference":[{"key":"B1","first-page":"18","article-title":"\u201cLearning from sparse data by exploiting monotonicity constraints,\u201d","volume-title":"UAI'05: Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence","author":"Altendorf","year":"2005"},{"key":"B2","first-page":"254","article-title":"\u201cMachine bias,\u201d","volume-title":"Ethics of Data and Analytics","author":"Angwin","year":"2016"},{"key":"B3","unstructured":"\u201cA POMDP formulation of preference elicitation problems,\u201d239246\n            BoutilierC.\n          \n            DechterR.\n            KearnsM. J.\n            SuttonR. S.\n          Edmonton, ABAAAI Press; The MIT PressProceedings of the Eighteenth National Conference on Artificial Intelligence and Fourteenth Conference on Innovative Applications of Artificial Intelligence2002"},{"key":"B4","first-page":"2685","article-title":"\u201cIntegrating deep learning with first-order logic programmed constraints for zero-day phishing attack detection,\u201d","volume-title":"IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP","author":"Bu","year":"2021"},{"key":"B5","first-page":"563","article-title":"\u201cBoosting with side information,\u201d","volume-title":"11th Asian Conference on Computer Vision","author":"Chen","year":"2012"},{"key":"B6","doi-asserted-by":"crossref","first-page":"719","DOI":"10.1145\/3133956.3134092","article-title":"\u201cFairness in an unfair world: fair multiparty computation from public bulletin boards,\u201d","volume-title":"Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security","author":"Choudhuri","year":"2017"},{"key":"B7","first-page":"134","article-title":"\u201cA case study of algorithm-assisted decision making in child maltreatment hotline screening decisions,\u201d","volume-title":"Conference on Fairness, Accountability and Transparency, FAT 2018","author":"Chouldechova","year":"2018"},{"key":"B8","author":"Cortez","year":"2008","journal-title":"Using Data Mining to Predict Secondary School Student Performance"},{"key":"B9","first-page":"110","article-title":"\u201cHuman-guided learning of column networks: knowledge injection for relational deep learning,\u201d","volume-title":"CODS-COMAD '21: Proceedings of the 3rd ACM India Joint International Conference on Data Science & Management of Data (8th ACM IKDD CODS & 26th COMAD)","author":"Das","year":"2021"},{"key":"B10","unstructured":"DheeruD.\n            TaniskidouE.\n          The UCI Machine Learning Repository2017"},{"key":"B11","article-title":"Gradient tree boosting for training conditional random fields","author":"Dietterich","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"B12","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1080\/21642583.2018.1482477","article-title":"Prior knowledge-based deep learning method for indoor object recognition and application","volume":"6","author":"Ding","year":"2018","journal-title":"Syst. Sci. Control"},{"key":"B13","first-page":"645","article-title":"\u201cPreference elicitation and interview minimization in stable matchings,\u201d","volume-title":"Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence","author":"Drummond","year":"2014"},{"key":"B14","first-page":"214","article-title":"\u201cFairness through awareness,\u201d","volume-title":"Innovations in Theoretical Computer Science 2012","author":"Dwork","year":"2012"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: a gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Stat"},{"key":"B16","first-page":"521","article-title":"\u201cKnowledge-Based support vector machine classifiers,\u201d","volume-title":"Advances in Neural Information Processing Systems 15 (NIPS 2002)","author":"Fung","year":"2002"},{"key":"B17","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1145\/3303772.3303791","article-title":"\u201cEvaluating the fairness of predictive student models through slicing analysis,\u201d","author":"Gardner","year":"2019","journal-title":"LAK19: Proceedings of the 9th International Conference on Learning Analytics"},{"key":"B18","doi-asserted-by":"publisher","first-page":"539.e1","DOI":"10.1016\/j.ajog.2015.01.019","article-title":"A description of the methods of the nulliparous pregnancy outcomes study: monitoring mothers-to-be (numom2b)","volume":"212","author":"Haas","year":"2015","journal-title":"Am. J. Obstet. Gynecol"},{"key":"B19","first-page":"3315","article-title":"\u201cEquality of opportunity in supervised learning,\u201d","volume-title":"Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016","author":"Hardt","year":"2016"},{"key":"B20","first-page":"837","article-title":"\u201cMind the nuisance: Gaussian process classification using privileged noise,\u201d","volume-title":"Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014","author":"Hern\u00e1ndez-Lobato","year":"2014"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1503.02531","article-title":"Distilling the knowledge in a neural network","author":"Hinton","year":"2015","journal-title":"arXiv:1503.02531"},{"key":"B22","first-page":"200","article-title":"\u201cTransductive inference for text classification using support vector machines,\u201d","volume-title":"Proceedings of the Sixteenth International Conference on Machine Learning (ICML 1999)","author":"Joachims","year":"1999"},{"key":"B23","first-page":"35","article-title":"\u201cFairness-aware classifier with prejudice remover regularizer,\u201d","volume-title":"Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2012","author":"Kamishima","year":"2012"},{"key":"B24","first-page":"2635","article-title":"\u201cBlind justice: fairness with encrypted sensitive attributes,\u201d","volume-title":"Proceedings of the 35th International Conference on Machine Learning, ICML 2018","author":"Kilbertus","year":"2018"},{"key":"B25","unstructured":"\u201cScaling up the accuracy of naive-Bayes classifiers: a decision-tree hybrid,\u201d202207\n            KohaviR.\n          \n            SimoudisE.\n            HanJ.\n            FayyadU. M.\n          Portland, ORAAAI PressProceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96)1996"},{"key":"B26","doi-asserted-by":"publisher","first-page":"4460","DOI":"10.1609\/aaai.v34i04.5873","article-title":"A unified framework for knowledge intensive gradient boosting: leveraging human experts for noisy sparse domains","volume":"34","author":"Kokel","year":"2020","journal-title":"Proc. AAAI Conf. Artif. Intell"},{"key":"B27","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1145\/3178876.3186133","article-title":"\u201cAdaptive sensitive reweighting to mitigate bias in fairness-aware classification,\u201d","volume-title":"WWW '18: Proceedings of the 2018 World Wide Web Conference","author":"Krasanakis","year":"2018"},{"key":"B28","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1109\/ICDM.2013.79","article-title":"\u201cGuiding autonomous agents to better behaviors through human advice,\u201d","volume-title":"2013 IEEE 13th International Conference on Data Mining","author":"Kunapuli","year":"2013"},{"key":"B29","doi-asserted-by":"publisher","first-page":"170171","DOI":"10.1038\/sdata.2017.171","article-title":"Open university learning analytics dataset","volume":"4","author":"Kuzilek","year":"2017","journal-title":"Sci. Data"},{"key":"B30","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1016\/j.neunet.2014.02.002","article-title":"Learning using privileged information: SV M+ and weighted SVM","volume":"53","author":"Lapin","year":"2014","journal-title":"Neural Netw"},{"key":"B31","doi-asserted-by":"publisher","first-page":"766","DOI":"10.1016\/j.neunet.2009.06.030","article-title":"Predictive learning with structured (grouped) data","volume":"22","author":"Liang","year":"2009","journal-title":"Neural Netw"},{"key":"B32","unstructured":"\u201cUnifying distillation and privileged information,\u201d\n            Lopez-PazD.\n            BottouL.\n            Sch\u00f6lkopfB.\n            VapnikV.\n          \n            BengioY.\n            LeCunY.\n          4th International Conference on Learning Representations, ICLR 20162016"},{"key":"B33","first-page":"130","article-title":"\u201cIdentifying rare diseases from behavioural data: a machine learning approach,\u201d","volume-title":"Proceedings of the First IEEE International Conference on Connected Health: Applications, Systems and Engineering Technologies, CHASE, 2016","author":"MacLeod","year":"2016"},{"key":"B34","unstructured":"\u201cGiving advice about preferred actions to reinforcement learners via knowledge-based kernel regression,\u201d819824\n            MaclinR.\n            ShavlikJ.\n            TorreyL.\n            WalkerT.\n            WildE.\n          \n            VelosoM. M.\n            KambhampatiS.\n          Proceedings, the Twentieth National Conference on Artificial Intelligence and the Seventeenth Innovative Applications of Artificial Intelligence Conference2005"},{"key":"B35","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1016\/j.dss.2014.03.001","article-title":"A data-driven approach to predict the success of bank telemarketing","volume":"62","author":"Moro","year":"2014","journal-title":"Decis. Support Syst"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-13644-8","author":"Natarajan","year":"2015","journal-title":"Boosted Statistical Relational Learners: From Benchmarks to Data-Driven Medicine."},{"key":"B37","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1007\/s10994-011-5244-9","article-title":"Gradient-based boosting for statistical relational learning: the relational dependency network case","volume":"86","author":"Natarajan","year":"2012","journal-title":"Mach. Learn"},{"key":"B38","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1016\/j.cmpb.2018.02.003","article-title":"A novel biomedical image indexing and retrieval system via deep preference learning","volume":"158","author":"Pang","year":"2018","journal-title":"Comput. Methods Prog. Biomed"},{"key":"B39","first-page":"1894","article-title":"\u201cOn the theory of learning with privileged information,\u201d","volume-title":"Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010","author":"Pechyony","year":"2010"},{"key":"B40","first-page":"677","article-title":"\u201cRecycling privileged learning and distribution matching for fairness,\u201d","volume-title":"Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017","author":"Quadrianto","year":"2017"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-01560-1","article-title":"Active Learning","author":"Settles","year":"2012","journal-title":"Synthesis Lectures on Artificial Intelligence and Machine Learning."},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.107","article-title":"\u201cLearning to rank using privileged information,\u201d","author":"Sharmanska","year":"2013","journal-title":"CVPR"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1155\/2014\/781670","article-title":"Impact of hba1c measurement on hospital readmission rates: analysis of 70,000 clinical database patient records","author":"Strack","year":"2014","journal-title":"BioMed Res. Int."},{"key":"B44","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/0004-3702(94)90105-8","article-title":"Knowledge-based artificial neural networks","volume":"70","year":"1994","journal-title":"Artif. Intell."},{"key":"B45","article-title":"\u201cThe 2001 census in the Netherlands,\u201d","author":"Van der Laan","year":"2000","journal-title":"Conference the Census of Population"},{"key":"B46","doi-asserted-by":"publisher","first-page":"544","DOI":"10.1016\/j.neunet.2009.06.042","article-title":"A new learning paradigm: learning using privileged information","volume":"22","author":"Vapnik","year":"2009","journal-title":"Neural Netw"},{"key":"B47","first-page":"199","article-title":"\u201cFair classification under strict unawareness,\u201d","volume-title":"Proceedings of the 2021 SIAM International Conference on Data Mining, SDM 2021","author":"Wang","year":"2021"},{"key":"B48","first-page":"9225","article-title":"\u201cIntegrating deep learning with logic fusion for information extraction,\u201d","volume-title":"The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020","author":"Wang","year":"2020"},{"key":"B49","first-page":"6786","article-title":"\u201cFairness risk measures,\u201d","volume-title":"Proceedings of the 36th International Conference on Machine Learning, ICML 2019","author":"Williamson","year":"2019"},{"key":"B50","first-page":"580","article-title":"\u201cKnowledge intensive learning: combining qualitative constraints with causal independence for parameter learning in probabilistic models,\u201d","volume-title":"Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2013","author":"Yang","year":"2013"},{"key":"B51","doi-asserted-by":"publisher","first-page":"2473","DOI":"10.1016\/j.eswa.2007.12.020","article-title":"The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients","volume":"36","author":"Yeh","year":"2009","journal-title":"Expert Syst. Appl"},{"key":"B52","first-page":"962","article-title":"\u201cFairness constraints: mechanisms for fair classification,\u201d","volume-title":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017","author":"Zafar","year":"2017"},{"key":"B53","doi-asserted-by":"publisher","first-page":"1060","DOI":"10.1007\/s10618-017-0506-1","article-title":"Measuring discrimination in algorithmic decision making","volume":"31","author":"\u017dliobait\u0117","year":"2017","journal-title":"Data Mining Knowl. Discov"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1260583\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,14]],"date-time":"2023-11-14T02:21:30Z","timestamp":1699928490000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1260583\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,13]]},"references-count":53,"alternative-id":["10.3389\/frai.2023.1260583"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1260583","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,13]]},"article-number":"1260583"}}