{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,6]],"date-time":"2025-10-06T05:49:51Z","timestamp":1759729791869,"version":"3.41.0"},"reference-count":86,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA2","license":[{"start":{"date-parts":[[2023,10,16]],"date-time":"2023-10-16T00:00:00Z","timestamp":1697414400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2023,10,16]]},"abstract":"<jats:p>Due to the under-specified interfaces, developers face challenges in correctly integrating machine learning (ML) APIs in software. Even when the ML API and the software are well designed on their own, the resulting application misbehaves when the API output is incompatible with the software. It is desirable to have an adapter that converts ML API output at runtime to better fit the software need and prevent integration failures.  \nIn this paper, we conduct an empirical study to understand ML API integration problems in real-world applications. Guided by this study, we present SmartGear, a tool that automatically detects and converts mismatching or incorrect ML API output at run time, serving as a middle layer between ML API and software. Our evaluation on a variety of open-source applications shows that SmartGear detects 70% incompatible API outputs and prevents 67% potential integration failures, outperforming alternative solutions.<\/jats:p>","DOI":"10.1145\/3622806","type":"journal-article","created":{"date-parts":[[2023,10,16]],"date-time":"2023-10-16T15:41:29Z","timestamp":1697470889000},"page":"264-291","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Run-Time Prevention of Software Integration Failures of Machine Learning APIs"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9162-9688","authenticated-orcid":false,"given":"Chengcheng","family":"Wan","sequence":"first","affiliation":[{"name":"East China Normal University, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-5957-5071","authenticated-orcid":false,"given":"Yuhan","family":"Liu","sequence":"additional","affiliation":[{"name":"University of Chicago, Chicago, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3964-4079","authenticated-orcid":false,"given":"Kuntai","family":"Du","sequence":"additional","affiliation":[{"name":"University of Chicago, Chicago, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0816-8150","authenticated-orcid":false,"given":"Henry","family":"Hoffmann","sequence":"additional","affiliation":[{"name":"University of Chicago, Chicago, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6877-1683","authenticated-orcid":false,"given":"Junchen","family":"Jiang","sequence":"additional","affiliation":[{"name":"University of Chicago, Chicago, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9778-6673","authenticated-orcid":false,"given":"Michael","family":"Maire","sequence":"additional","affiliation":[{"name":"University of Chicago, Chicago, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0757-4600","authenticated-orcid":false,"given":"Shan","family":"Lu","sequence":"additional","affiliation":[{"name":"Microsoft, Redmond, USA \/ University of Chicago, Chicago, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,10,16]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Aander-ETL. 2017. A smart album application. Online document. https:\/\/github.com\/Grusinator\/Aander-ETL \t\t\t\t  Aander-ETL. 2017. A smart album application. Online document. https:\/\/github.com\/Grusinator\/Aander-ETL"},{"key":"e_1_2_1_2_1","unstructured":"Amazon. 2022. Amazon artificial intelligence service. Online document. https:\/\/aws.amazon.com\/machine-learning\/ai-services \t\t\t\t  Amazon. 2022. Amazon artificial intelligence service. Online document. https:\/\/aws.amazon.com\/machine-learning\/ai-services"},{"key":"e_1_2_1_3_1","unstructured":"Amazon. 2022. Amazon Rekognition Image. Online document. https:\/\/aws.amazon.com\/rekognition\/image-features\/ \t\t\t\t  Amazon. 2022. Amazon Rekognition Image. Online document. https:\/\/aws.amazon.com\/rekognition\/image-features\/"},{"key":"e_1_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Saleema Amershi Andrew Begel Christian Bird Robert DeLine Harald Gall Ece Kamar Nachiappan Nagappan Besmira Nushi and Thomas Zimmermann. 2019. Software engineering for machine learning: A case study. In ICSE-SEIP. 291\u2013300. https:\/\/doi.org\/0.1109\/ICSE-SEIP.2019.00042 \t\t\t\t  Saleema Amershi Andrew Begel Christian Bird Robert DeLine Harald Gall Ece Kamar Nachiappan Nagappan Besmira Nushi and Thomas Zimmermann. 2019. Software engineering for machine learning: A case study. In ICSE-SEIP. 291\u2013300. https:\/\/doi.org\/0.1109\/ICSE-SEIP.2019.00042","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183519.3183537"},{"key":"e_1_2_1_6_1","volume-title":"International Conference on Machine Learning. 566\u2013576","author":"Bai Yu","year":"2021","unstructured":"Yu Bai , Song Mei , Huan Wang , and Caiming Xiong . 2021 . Don\u2019t just blame over-parametrization for over-confidence: Theoretical analysis of calibration in binary classification . In International Conference on Machine Learning. 566\u2013576 . https:\/\/doi.org\/10.48550\/arXiv.2102.07856 10.48550\/arXiv.2102.07856 Yu Bai, Song Mei, Huan Wang, and Caiming Xiong. 2021. Don\u2019t just blame over-parametrization for over-confidence: Theoretical analysis of calibration in binary classification. In International Conference on Machine Learning. 566\u2013576. https:\/\/doi.org\/10.48550\/arXiv.2102.07856"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-22110-1_14"},{"key":"e_1_2_1_8_1","unstructured":"Johannes Buchner. 2022. ImageHash: An image hashing library written in Python. Online document https:\/\/pypi.org\/project\/ImageHash\/. \t\t\t\t  Johannes Buchner. 2022. ImageHash: An image hashing library written in Python. Online document https:\/\/pypi.org\/project\/ImageHash\/."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00039"},{"key":"#cr-split#-e_1_2_1_10_1.1","unstructured":"Lingjiao Chen Tracy Cai Matei Zaharia and James Zou. 2022. Did the model change? Efficiently assessing machine learning API shifts. In ICLR Poster. https:\/\/doi.org\/10.48550\/arXiv.2107.14203 10.48550\/arXiv.2107.14203"},{"key":"#cr-split#-e_1_2_1_10_1.2","unstructured":"Lingjiao Chen Tracy Cai Matei Zaharia and James Zou. 2022. Did the model change? Efficiently assessing machine learning API shifts. In ICLR Poster. https:\/\/doi.org\/10.48550\/arXiv.2107.14203"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3300078"},{"key":"#cr-split#-e_1_2_1_12_1.1","unstructured":"Lingjiao Chen Matei Zaharia and James Zou. 2022. FrugalMCT: Efficient Online ML API Selection for Multi-Label Classification Tasks. In PMLR. https:\/\/doi.org\/10.48550\/arXiv.2102.09127 10.48550\/arXiv.2102.09127"},{"key":"#cr-split#-e_1_2_1_12_1.2","unstructured":"Lingjiao Chen Matei Zaharia and James Zou. 2022. FrugalMCT: Efficient Online ML API Selection for Multi-Label Classification Tasks. In PMLR. https:\/\/doi.org\/10.48550\/arXiv.2102.09127"},{"key":"e_1_2_1_13_1","volume-title":"Frugalml: How to use ml prediction apis more accurately and cheaply. In Advances in neural information processing systems. 33, 10685\u201310696. https:\/\/doi.org\/10.48550\/arXiv:2006.07512","author":"Chen Lingjiao","year":"2020","unstructured":"Lingjiao Chen , Matei Zaharia , and James Y Zou . 2020 . Frugalml: How to use ml prediction apis more accurately and cheaply. In Advances in neural information processing systems. 33, 10685\u201310696. https:\/\/doi.org\/10.48550\/arXiv:2006.07512 10.48550\/arXiv:2006.07512 Lingjiao Chen, Matei Zaharia, and James Y Zou. 2020. Frugalml: How to use ml prediction apis more accurately and cheaply. In Advances in neural information processing systems. 33, 10685\u201310696. https:\/\/doi.org\/10.48550\/arXiv:2006.07512"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2019.112948"},{"key":"e_1_2_1_15_1","volume-title":"International Symposium on Automated Technology for Verification and Analysis. https:\/\/doi.org\/10","author":"Cheng Chih-Hong","year":"2018","unstructured":"Chih-Hong Cheng , Chung-Hao Huang , and Hirotoshi Yasuoka . 2018 . Quantitative projection coverage for testing ml-enabled autonomous systems . In International Symposium on Automated Technology for Verification and Analysis. https:\/\/doi.org\/10 .48550\/arXiv.1805.04333 10.48550\/arXiv.1805.04333 Chih-Hong Cheng, Chung-Hao Huang, and Hirotoshi Yasuoka. 2018. Quantitative projection coverage for testing ml-enabled autonomous systems. In International Symposium on Automated Technology for Verification and Analysis. https:\/\/doi.org\/10.48550\/arXiv.1805.04333"},{"key":"e_1_2_1_16_1","first-page":"1301","article-title":"A survey on machine learning: concept, algorithms and applications","volume":"5","author":"Das Kajaree","year":"2017","unstructured":"Kajaree Das and Rabi Narayan Behera . 2017 . A survey on machine learning: concept, algorithms and applications . International Journal of Innovative Research in Computer and Communication Engineering , 5 , 2 (2017), 1301 \u2013 1309 . Kajaree Das and Rabi Narayan Behera. 2017. A survey on machine learning: concept, algorithms and applications. International Journal of Innovative Research in Computer and Communication Engineering, 5, 2 (2017), 1301\u20131309.","journal-title":"International Journal of Innovative Research in Computer and Communication Engineering"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/IDEA49133.2020.9170675"},{"key":"e_1_2_1_18_1","unstructured":"FESMKMITL. 2021. A smart album application. Online document. https:\/\/github.com\/matthewjmc\/FESMKMITL \t\t\t\t  FESMKMITL. 2021. A smart album application. Online document. https:\/\/github.com\/matthewjmc\/FESMKMITL"},{"key":"e_1_2_1_19_1","unstructured":"Flood-Depths. 2021. A flood detection application. Online document. https:\/\/github.com\/nlonberg\/flood-depths \t\t\t\t  Flood-Depths. 2021. A flood detection application. Online document. https:\/\/github.com\/nlonberg\/flood-depths"},{"key":"e_1_2_1_20_1","unstructured":"FortniteTracker. 2019. A real time game tracker application. Online document. https:\/\/github.com\/Godsinred\/FortniteKillfeed \t\t\t\t  FortniteTracker. 2019. A real time game tracker application. Online document. https:\/\/github.com\/Godsinred\/FortniteKillfeed"},{"key":"e_1_2_1_21_1","unstructured":"Google. 2022. Google Cloud AI. Online document. https:\/\/cloud.google.com\/products\/ai \t\t\t\t  Google. 2022. Google Cloud AI. Online document. https:\/\/cloud.google.com\/products\/ai"},{"key":"e_1_2_1_22_1","unstructured":"Google. 2022. Google Cloud Natural Language. Online document. https:\/\/cloud.google.com\/natural-language\/docs\/categories \t\t\t\t  Google. 2022. Google Cloud Natural Language. Online document. https:\/\/cloud.google.com\/natural-language\/docs\/categories"},{"key":"#cr-split#-e_1_2_1_23_1.1","unstructured":"Margherita Grandini Enrico Bagli and Giorgio Visani. 2020. Metrics for multi-class classification: an overview. arXiv preprint arXiv:2008.05756 https:\/\/doi.org\/10.48550\/arXiv.2008.05756 10.48550\/arXiv.2008.05756"},{"key":"#cr-split#-e_1_2_1_23_1.2","unstructured":"Margherita Grandini Enrico Bagli and Giorgio Visani. 2020. Metrics for multi-class classification: an overview. arXiv preprint arXiv:2008.05756 https:\/\/doi.org\/10.48550\/arXiv.2008.05756"},{"key":"e_1_2_1_24_1","volume-title":"2021 IEEE\/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 71\u201380","author":"Grent Henk","year":"2021","unstructured":"Henk Grent , Aleksei Akimov , and Maur\u00edcio Aniche . 2021 . Automatically identifying parameter constraints in complex Web APIs: A case study at Adyen . In 2021 IEEE\/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 71\u201380 . https:\/\/doi.org\/10.48550\/arXiv.2102.00871 10.48550\/arXiv.2102.00871 Henk Grent, Aleksei Akimov, and Maur\u00edcio Aniche. 2021. Automatically identifying parameter constraints in complex Web APIs: A case study at Adyen. In 2021 IEEE\/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 71\u201380. https:\/\/doi.org\/10.48550\/arXiv.2102.00871"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-06981-9_2"},{"key":"e_1_2_1_26_1","unstructured":"Dave Halter. 2022. Jedi: an awesome auto-completion static analysis and refactoring library for Python. Online document. https:\/\/jedi.readthedocs.io \t\t\t\t  Dave Halter. 2022. Jedi: an awesome auto-completion static analysis and refactoring library for Python. Online document. https:\/\/jedi.readthedocs.io"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.2334-5837.2016.00179.x"},{"key":"e_1_2_1_28_1","volume-title":"Using self-supervised learning can improve model robustness and uncertainty. Advances in neural information processing systems, 32","author":"Hendrycks Dan","year":"2019","unstructured":"Dan Hendrycks , Mantas Mazeika , Saurav Kadavath , and Dawn Song . 2019. Using self-supervised learning can improve model robustness and uncertainty. Advances in neural information processing systems, 32 ( 2019 ), https:\/\/doi.org\/doi\/10.5555\/3454287.3455690 Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. 2019. Using self-supervised learning can improve model robustness and uncertainty. Advances in neural information processing systems, 32 (2019), https:\/\/doi.org\/doi\/10.5555\/3454287.3455690"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2016.7739680"},{"key":"e_1_2_1_30_1","volume-title":"2017 16th IEEE international conference on machine learning and applications (ICMLA). 101\u2013105","author":"Hosseini Hossein","year":"2017","unstructured":"Hossein Hosseini , Baicen Xiao , and Radha Poovendran . 2017 . Google\u2019s cloud vision api is not robust to noise . In 2017 16th IEEE international conference on machine learning and applications (ICMLA). 101\u2013105 . https:\/\/doi.org\/10.48550\/arXiv.1704.05051 10.48550\/arXiv.1704.05051 Hossein Hosseini, Baicen Xiao, and Radha Poovendran. 2017. Google\u2019s cloud vision api is not robust to noise. In 2017 16th IEEE international conference on machine learning and applications (ICMLA). 101\u2013105. https:\/\/doi.org\/10.48550\/arXiv.1704.05051"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3326285.3329042"},{"key":"e_1_2_1_32_1","unstructured":"IBM. 2022. IBM Watson. Online document. https:\/\/www.ibm.com\/watson \t\t\t\t  IBM. 2022. IBM Watson. Online document. https:\/\/www.ibm.com\/watson"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.3233\/978-1-61499-495-4-26"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICAI55435.2022.9773725"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884783"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3182515"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126543"},{"key":"#cr-split#-e_1_2_1_38_1.1","unstructured":"Alina Kuznetsova Hassan Rom Neil Alldrin Jasper Uijlings Ivan Krasin Jordi Pont-Tuset Shahab Kamali Stefan Popov Matteo Malloci and Alexander Kolesnikov. 2020. The open images dataset v4. International Journal of Computer Vision 1-26. https:\/\/doi.org\/10.48550\/arXiv.1811.00982 10.48550\/arXiv.1811.00982"},{"key":"#cr-split#-e_1_2_1_38_1.2","doi-asserted-by":"crossref","unstructured":"Alina Kuznetsova Hassan Rom Neil Alldrin Jasper Uijlings Ivan Krasin Jordi Pont-Tuset Shahab Kamali Stefan Popov Matteo Malloci and Alexander Kolesnikov. 2020. The open images dataset v4. International Journal of Computer Vision 1-26. https:\/\/doi.org\/10.48550\/arXiv.1811.00982","DOI":"10.1007\/s11263-020-01316-z"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1810295.1810352"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/MET.2017.6"},{"volume-title":"The Future of Software Quality Assurance","author":"Linz Tilo","key":"e_1_2_1_41_1","unstructured":"Tilo Linz . 2020. Testing Autonomous Systems . In The Future of Software Quality Assurance . Springer , Cham , 61\u201375. Tilo Linz. 2020. Testing Autonomous Systems. In The Future of Software Quality Assurance. Springer, Cham, 61\u201375."},{"key":"e_1_2_1_42_1","unstructured":"Lisa-Assistant. 2021. A voice assistant application. Online document. https:\/\/github.com\/AlexNguyen27\/lisa-assistant-gcp \t\t\t\t  Lisa-Assistant. 2021. A voice assistant application. Online document. https:\/\/github.com\/AlexNguyen27\/lisa-assistant-gcp"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2594291.2594337"},{"key":"e_1_2_1_44_1","unstructured":"David Marby and Nijiko Yonskai. 2021. Pyan3: Offline call graph generator for Python 3. Online document. https:\/\/github.com\/davidfraser\/pyan \t\t\t\t  David Marby and Nijiko Yonskai. 2021. Pyan3: Offline call graph generator for Python 3. Online document. https:\/\/github.com\/davidfraser\/pyan"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDAR.2015.7333746"},{"key":"e_1_2_1_46_1","unstructured":"Microsoft. 2022. Microsoft Azure Cognitive Services. Online document. https:\/\/azure.microsoft.com\/en-us\/services\/cognitive-services \t\t\t\t  Microsoft. 2022. Microsoft Azure Cognitive Services. Online document. https:\/\/azure.microsoft.com\/en-us\/services\/cognitive-services"},{"key":"e_1_2_1_47_1","unstructured":"Microsoft. 2022. Microsoft Azure Image Tagging. Online document. https:\/\/docs.microsoft.com\/en-us\/azure\/cognitive-services\/computer-vision\/concept-tagging-images \t\t\t\t  Microsoft. 2022. Microsoft Azure Image Tagging. Online document. https:\/\/docs.microsoft.com\/en-us\/azure\/cognitive-services\/computer-vision\/concept-tagging-images"},{"key":"e_1_2_1_48_1","unstructured":"Hong Minhee. 2021. Python interface for Wikidata. Online document. https:\/\/pypi.org\/project\/Wikidata\/ \t\t\t\t  Hong Minhee. 2021. Python interface for Wikidata. Online document. https:\/\/pypi.org\/project\/Wikidata\/"},{"key":"#cr-split#-e_1_2_1_49_1.1","unstructured":"Chaithanya Kumar Mummadi Robin Hutmacher Kilian Rambach Evgeny Levinkov Thomas Brox and Jan Hendrik Metzen. 2021. Test-time adaptation to distribution shift by confidence maximization and input transformation. arXiv preprint arXiv:2106.14999 https:\/\/doi.org\/10.48550\/arxiv.2106.14999 10.48550\/arxiv.2106.14999"},{"key":"#cr-split#-e_1_2_1_49_1.2","unstructured":"Chaithanya Kumar Mummadi Robin Hutmacher Kilian Rambach Evgeny Levinkov Thomas Brox and Jan Hendrik Metzen. 2021. Test-time adaptation to distribution shift by confidence maximization and input transformation. arXiv preprint arXiv:2106.14999 https:\/\/doi.org\/10.48550\/arxiv.2106.14999"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510209"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP51992.2021.00023"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.23919\/FRUCT49677.2020.9211020"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1629575.1629585"},{"key":"e_1_2_1_54_1","unstructured":"Plant-Watcher. 2018. A plant management application. Online document. https:\/\/github.com\/siwasu17\/plant-watcher \t\t\t\t  Plant-Watcher. 2018. A plant management application. Online document. https:\/\/github.com\/siwasu17\/plant-watcher"},{"key":"e_1_2_1_55_1","unstructured":"ProjectSyn. 2020. A voice-activated light application. Online document. https:\/\/github.com\/mochiliu\/projectsyn \t\t\t\t  ProjectSyn. 2020. A voice-activated light application. Online document. https:\/\/github.com\/mochiliu\/projectsyn"},{"key":"e_1_2_1_56_1","unstructured":"RoomR. 2020. A property management application. Online document. https:\/\/github.com\/rodrigoHM\/RoomR-Server \t\t\t\t  RoomR. 2020. A property management application. Online document. https:\/\/github.com\/rodrigoHM\/RoomR-Server"},{"key":"e_1_2_1_57_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1401\u20131409","author":"Royer Amelie","year":"2015","unstructured":"Amelie Royer and Christoph H Lampert . 2015 . Classifier adaptation at prediction time . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1401\u20131409 . Amelie Royer and Christoph H Lampert. 2015. Classifier adaptation at prediction time. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1401\u20131409."},{"key":"e_1_2_1_58_1","unstructured":"SeeFarBeyond. 2022. A coin finder application. Online document. https:\/\/github.com\/arosloff\/SeeFarBeyond \t\t\t\t  SeeFarBeyond. 2022. A coin finder application. Online document. https:\/\/github.com\/arosloff\/SeeFarBeyond"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.2478\/jaiscr-2019-0006"},{"key":"e_1_2_1_60_1","unstructured":"SmartCan. 2019. An garbage classification application. Online document. https:\/\/github.com\/ertheosiswadi\/smart_can \t\t\t\t  SmartCan. 2019. An garbage classification application. Online document. https:\/\/github.com\/ertheosiswadi\/smart_can"},{"key":"e_1_2_1_61_1","unstructured":"Kevin D. Smith Jim J. Jewett Skip Montanaro and Anthony Baxter. 2022. PEP 318 \u2013 Decorators for Functions and Methods. Online document. https:\/\/peps.python.org\/pep-0318\/ \t\t\t\t  Kevin D. Smith Jim J. Jewett Skip Montanaro and Anthony Baxter. 2022. PEP 318 \u2013 Decorators for Functions and Methods. Online document. https:\/\/peps.python.org\/pep-0318\/"},{"key":"e_1_2_1_62_1","volume-title":"Bytecode: Python module to generate and modify bytecode. Online document. https:\/\/pypi.org\/project\/bytecode\/","author":"Stinner Victor","year":"2021","unstructured":"Victor Stinner . 2021 . Bytecode: Python module to generate and modify bytecode. Online document. https:\/\/pypi.org\/project\/bytecode\/ Victor Stinner. 2021. Bytecode: Python module to generate and modify bytecode. Online document. https:\/\/pypi.org\/project\/bytecode\/"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/CRV.2019.00010"},{"key":"#cr-split#-e_1_2_1_64_1.1","doi-asserted-by":"crossref","unstructured":"Paul Voigtlaender and Bastian Leibe. 2017. Online adaptation of convolutional neural networks for video object segmentation. In BMVC. https:\/\/doi.org\/10.48550\/arXiv.1706.09364 10.48550\/arXiv.1706.09364","DOI":"10.5244\/C.31.116"},{"key":"#cr-split#-e_1_2_1_64_1.2","doi-asserted-by":"crossref","unstructured":"Paul Voigtlaender and Bastian Leibe. 2017. Online adaptation of convolutional neural networks for video object segmentation. In BMVC. https:\/\/doi.org\/10.48550\/arXiv.1706.09364","DOI":"10.5244\/C.31.116"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00024"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510068"},{"key":"e_1_2_1_67_1","unstructured":"Chengcheng Wan Yuhan Liu Kuntai Du Henry Hoffmann Junchen Jiang Michael Maire and Shan Lu. 2023. Artifact for \"Run-Time Prevention of Software Integration Failures of Machine Learning APIs\". Online document. https:\/\/zenodo.org\/record\/8106520 \t\t\t\t  Chengcheng Wan Yuhan Liu Kuntai Du Henry Hoffmann Junchen Jiang Michael Maire and Shan Lu. 2023. Artifact for \"Run-Time Prevention of Software Integration Failures of Machine Learning APIs\". Online document. https:\/\/zenodo.org\/record\/8106520"},{"key":"e_1_2_1_68_1","unstructured":"Chengcheng Wan Yuhan Liu Kuntai Du Henry Hoffmann Junchen Jiang Michael Maire and Shan Lu. 2023. SmartGear. Online document. https:\/\/github.com\/mlapistudy\/SmartGear \t\t\t\t  Chengcheng Wan Yuhan Liu Kuntai Du Henry Hoffmann Junchen Jiang Michael Maire and Shan Lu. 2023. SmartGear. Online document. https:\/\/github.com\/mlapistudy\/SmartGear"},{"key":"e_1_2_1_69_1","volume-title":"Tent: Fully Test-time Adaptation by Entropy Minimization. In ICLR. https:\/\/doi.org\/10.48550\/arXiv.2006.10726","author":"Wang Dequan","year":"2021","unstructured":"Dequan Wang , Evan Shelhamer , Shaoteng Liu , Bruno Olshausen , and Trevor Darrell . 2021 . Tent: Fully Test-time Adaptation by Entropy Minimization. In ICLR. https:\/\/doi.org\/10.48550\/arXiv.2006.10726 10.48550\/arXiv.2006.10726 Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. 2021. Tent: Fully Test-time Adaptation by Entropy Minimization. In ICLR. https:\/\/doi.org\/10.48550\/arXiv.2006.10726"},{"key":"e_1_2_1_70_1","first-page":"11809","article-title":"Rethinking calibration of deep neural networks: Do not be afraid of overconfidence","volume":"34","author":"Wang Deng-Bao","year":"2021","unstructured":"Deng-Bao Wang , Lei Feng , and Min-Ling Zhang . 2021 . Rethinking calibration of deep neural networks: Do not be afraid of overconfidence . Advances in Neural Information Processing Systems , 34 (2021), 11809 \u2013 11820 . Deng-Bao Wang, Lei Feng, and Min-Ling Zhang. 2021. Rethinking calibration of deep neural networks: Do not be afraid of overconfidence. Advances in Neural Information Processing Systems, 34 (2021), 11809\u201311820.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.610"},{"key":"e_1_2_1_72_1","unstructured":"Wikidata. 2022. A free and open knowledge base. Online document. https:\/\/www.wikidata.org\/ \t\t\t\t  Wikidata. 2022. A free and open knowledge base. Online document. https:\/\/www.wikidata.org\/"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.30"},{"key":"e_1_2_1_74_1","first-page":"11745","article-title":"Fixes that fail: Self-defeating improvements in machine-learning systems","volume":"34","author":"Wu Ruihan","year":"2021","unstructured":"Ruihan Wu , Chuan Guo , Awni Hannun , and Laurens van der Maaten . 2021 . Fixes that fail: Self-defeating improvements in machine-learning systems . Advances in Neural Information Processing Systems , 34 (2021), 11745 \u2013 11756 . https:\/\/doi.org\/10.48550\/arXiv.2103.11766 10.48550\/arXiv.2103.11766 Ruihan Wu, Chuan Guo, Awni Hannun, and Laurens van der Maaten. 2021. Fixes that fail: Self-defeating improvements in machine-learning systems. Advances in Neural Information Processing Systems, 34 (2021), 11745\u201311756. https:\/\/doi.org\/10.48550\/arXiv.2103.11766","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM48880.2022.9796701"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICMEW.2015.7169816"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3131365.3131372"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238187"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-11713-8_21"},{"key":"e_1_2_1_80_1","volume-title":"International Conference on Machine Learning. 7693\u20137702","author":"Zintgraf Luisa","year":"2019","unstructured":"Luisa Zintgraf , Kyriacos Shiarli , Vitaly Kurin , Katja Hofmann , and Shimon Whiteson . 2019 . Fast context adaptation via meta-learning . In International Conference on Machine Learning. 7693\u20137702 . https:\/\/doi.org\/10.48550\/arXiv.1810.03642 10.48550\/arXiv.1810.03642 Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. 2019. Fast context adaptation via meta-learning. In International Conference on Machine Learning. 7693\u20137702. https:\/\/doi.org\/10.48550\/arXiv.1810.03642"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3622806","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3622806","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:04Z","timestamp":1750178224000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3622806"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,16]]},"references-count":86,"journal-issue":{"issue":"OOPSLA2","published-print":{"date-parts":[[2023,10,16]]}},"alternative-id":["10.1145\/3622806"],"URL":"https:\/\/doi.org\/10.1145\/3622806","relation":{},"ISSN":["2475-1421"],"issn-type":[{"type":"electronic","value":"2475-1421"}],"subject":[],"published":{"date-parts":[[2023,10,16]]},"assertion":[{"value":"2023-10-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}