{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T15:03:57Z","timestamp":1781708637702,"version":"3.54.5"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T00:00:00Z","timestamp":1781654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62302304"],"award-info":[{"award-number":["62302304"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Shanghai Sailing Program","award":["23YF1427500"],"award-info":[{"award-number":["23YF1427500"]}]},{"name":"ShanghaiTech Startup Funding, and the National Cryptologic Science Foundation of China","award":["2025NCSF01012"],"award-info":[{"award-number":["2025NCSF01012"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,7,31]]},"abstract":"<jats:p>Dynamic languages like Python are highly valued for the flexibility they offer, primarily due to their dynamic typing system. However, this flexibility often comes at a cost: type-related errors typically manifest only at runtime, which can result in elusive bugs. This scenario highlights the urgent need for effective automatic type inference strategies. Recently, there has been significant progress in the development of deep learning-based methods targeting the issues. Despite their potential, these methods face considerable limitations in the context of rare types (predominantly user-defined types in Python), which are often underrepresented in training datasets. To tackle the inherent unpredictability of dynamic characteristics in Python, we present an innovative framework named RunTyper. RunTyper uniquely integrates dynamic analysis with a sophisticated framework that combines dynamic instrumentation and static rules. This hybrid approach is utilized to validate and refine predictions made by deep neural networks. When compared with Type4Py, a deep learning-based type inference tool, the experiment results indicate that RunTyper achieves a significantly higher accuracy rate of 53.7%, whereas Type4Py attains only 41.9%. Additionally, a series of experiments demonstrate that RunTyper significantly enhances performance in user-defined types. When compared to HiTyper, which similarly seeks to refine deep type inference but is limited to static rules, RunTyper exhibits a notable improvement, increasing accuracy from 17.0% to 22.9%, an enhancement rate of 35%. Moreover, the experiments indicate that RunTyper improves the accuracy of GPT-4 in processing user-defined types by a remarkable 17%. These findings emphasize RunTyper\u2019s proficiency in refining and enhancing the results produced by deep prediction models as well as its ability to accurately infer rare types that are traditionally difficult to predict.<\/jats:p>","DOI":"10.1145\/3771544","type":"journal-article","created":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:50:12Z","timestamp":1760104212000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["RunTyper: Enhancing Deep Type Inference Using Dynamic Analysis for Python"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-1516-8060","authenticated-orcid":false,"given":"Guowei","family":"Yang","sequence":"first","affiliation":[{"name":"ShanghaiTech University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8595-5388","authenticated-orcid":false,"given":"Shilin","family":"He","sequence":"additional","affiliation":[{"name":"Microsoft Corporation, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0581-2679","authenticated-orcid":false,"given":"Fu","family":"Song","sequence":"additional","affiliation":[{"name":"Key Laboratory of System Software (Chinese Academy of Sciences) and State Key Laboratory of Computer Science, Institute of Software Chinese Academy of Sciences, Beijing, China, State Key Laboratory of Cryptology, University of the Chinese Academy of Sciences, Beijing, China and Nanjing Institute of Software Technology, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2988-6012","authenticated-orcid":false,"given":"Yuqi","family":"Chen","sequence":"additional","affiliation":[{"name":"ShanghaiTech University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,17]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3385412.3385997"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/1925844.1926437"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/1297081.1297091"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/11531142_19"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133872"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-89051-3_3"},{"key":"e_1_3_2_8_2","unstructured":"Dropbox. 2024. PyAnnotate: Auto-generate PEP-484 Annotations. Retrieved from https:\/\/github.com\/dropbox\/pyannotate\/"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549126"},{"key":"e_1_3_2_10_2","unstructured":"Facebook. 2023. Pyre: A Performant Type-Checker for Python 3. Retrieved from https:\/\/pyre-check.org\/"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/1529282.1529700"},{"key":"e_1_3_2_12_2","article-title":"PEP 526","author":"Gonzalez R.","year":"2016","unstructured":"R. Gonzalez, P. House, I. Levkivskyi, L. Roach, and G. Van Rossum. 2016. PEP 526\u2014Python syntax for syntax for variable annotations. Index of Python Enhancement Proposals. Retrived from https:\/\/peps.python.org\/pep-0526\/","journal-title":"Index of Python Enhancement Proposals"},{"key":"e_1_3_2_13_2","unstructured":"Google. 2023. PyType. Retrieved from https:\/\/github.com\/google\/pytype\/"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-96142-2_2"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236051"},{"key":"e_1_3_2_16_2","first-page":"19","volume-title":"Proceedings of the 32nd Australasian Conference on Computer Science","volume":"91","author":"Holkner Alex","year":"2009","unstructured":"Alex Holkner and James Harland. 2009. Evaluating the dynamic behaviour of Python applications. In Proceedings of the 32nd Australasian Conference on Computer Science, Vol. 91, 19\u201328."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP46214.2022.9833751"},{"key":"e_1_3_2_18_2","unstructured":"IEEE Spectrum. 2023. The Top Programming Languages. Retrieved from https:\/\/spectrum.ieee.org\/the-top-programming-languages-2023"},{"key":"e_1_3_2_19_2","unstructured":"Instagram. 2024. MonkeyType. Retrieved from https:\/\/github.com\/Instagram\/MonkeyType\/"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3340459"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-03237-0_17"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3473135"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3426422.3426985"},{"key":"e_1_3_2_24_2","unstructured":"Jukka Lehtosalo G. V Rossum Ivan Levkivskyi Michael J. Sullivan David Fisher Greg Price Michael Lee N. Seyfer R. Barton S. Ilinskiy et al. 2017. MyPy\u2014Optional static typing for Python. Retrieved July 12 2023 from http:\/\/mypy-lang.org\/"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534411"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00045"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3473932"},{"key":"e_1_3_2_28_2","article-title":"PEP 591\u2014Adding a Final Qualifier to Typing","author":"Michael L.","year":"2019","unstructured":"L. Michael and L. Ivan. 2019. PEP 591\u2014Adding a Final Qualifier to Typing. Index of Python Enhancement Proposals. Retrived from https:\/\/peps.python.org\/pep-0591\/","journal-title":"Index of Python Enhancement Proposals"},{"key":"e_1_3_2_29_2","article-title":"PEP 586\u2014Literal Types","author":"Michael L.","year":"2019","unstructured":"L. Michael, L. Ivan, and L. Jukka. 2019. PEP 586\u2014Literal Types. Index of Python Enhancement Proposals. Retrived from https:\/\/peps.python.org\/pep-0586\/","journal-title":"Index of Python Enhancement Proposals"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR52588.2021.00079"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510124"},{"key":"e_1_3_2_32_2","first-page":"1","volume-title":"Proceedings of the ACM on Programming Languages","volume":"3","author":"Miyazaki Yusuke","year":"2019","unstructured":"Yusuke Miyazaki, Taro Sekiyama, and Atsushi Igarashi. 2019. Dynamic type inference for gradual Hindley\u2013Milner typing. In Proceedings of the ACM on Programming Languages 3, POPL (2019), 1\u201329."},{"key":"e_1_3_2_33_2","volume-title":"Proceedings of the 34th European Conference on Object-Oriented Programming (ECOOP \u201920)","author":"Monat Rapha\u00ebl","year":"2020","unstructured":"Rapha\u00ebl Monat, Abdelraouf Ouadjaout, and Antoine Min\u00e9. 2020. Static type analysis by abstract interpretation of Python programs. In Proceedings of the 34th European Conference on Object-Oriented Programming (ECOOP \u201920). Schloss Dagstuhl-Leibniz-Zentrum f\u00fcr Informatik."},{"key":"e_1_3_2_34_2","unstructured":"Irene Vlassi Pandi Earl T. Barr Andrew D. Gordon and Charles Sutton. 2020. OptTyper: Probabilistic type inference by optimising logical and natural constraints. arXiv:2004.00348. Retrieved from https:\/\/arxiv.org\/abs\/2004.00348"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510038"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409715"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3426422.3426981"},{"key":"e_1_3_2_38_2","first-page":"76","volume-title":"Proceedings of the ECOOP Workshops","author":"Rapicault Pascal","year":"1998","unstructured":"Pascal Rapicault, Mireille Blay-Fornarino, St\u00e9phane Ducasse, and Anne-Marie Dery. 1998. Dynamic type inference to support object-oriented reengineering in Smalltalk. In Proceedings of the ECOOP Workshops. CiteSeer, 76\u201377."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2775051.2677009"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/2480362.2480655"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/1806596.1806598"},{"key":"e_1_3_2_42_2","volume-title":"Starkiller: A Static Type Inferencer and Compiler for Python","author":"Salib Michael","year":"2004","unstructured":"Michael Salib. 2004. Starkiller: A Static Type Inferencer and Compiler for Python. Ph.D. Dissertation. Massachusetts Institute of Technology."},{"key":"e_1_3_2_43_2","article-title":"PEP 484\u2014Type hints","author":"Van Rossum Guido","year":"2014","unstructured":"Guido Van Rossum, Jukka Lehtosalo, and Lukasz Langa. 2014. PEP 484\u2014Type hints. Index of Python Enhancement Proposals. Retrived fromhttps:\/\/peps.python.org\/pep-0484\/","journal-title":"Index of Python Enhancement Proposals"},{"key":"e_1_3_2_44_2","unstructured":"Yin Wang. 2019. PySonar2: An Advanced Semantic Indexer for Python. Retrieved from https:\/\/github.com\/yinwang0\/pysonar2"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510041"},{"key":"e_1_3_2_46_2","unstructured":"Jiayi Wei Greg Durrett and Isil Dillig. 2023. TypeT5: Seq2seq type inference using static analysis. arXiv:2303.09564. Retrieved from https:\/\/arxiv.org\/abs\/2303.09564"},{"key":"e_1_3_2_47_2","unstructured":"Jiayi Wei Maruth Goyal Greg Durrett and Isil Dillig. 2020. LambdaNet: Probabilistic type inference using graph neural networks. arXiv:2005.02161. Retrieved from https:\/\/arxiv.org\/abs\/2005.02161"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950343"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00170"},{"key":"e_1_3_2_50_2","article-title":"PEP 585\u2014Type hinting generics in standard collections","author":"Langa \u0141ukasz","year":"2019","unstructured":"\u0141ukasz Langa. 2019. PEP 585\u2014Type hinting generics in standard collections. Index of Python Enhancement Proposals. Retrived fromhttps:\/\/peps.python.org\/pep-0585\/","journal-title":"Index of Python Enhancement Proposals"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3771544","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T14:45:14Z","timestamp":1781707514000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3771544"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,17]]},"references-count":49,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2026,7,31]]}},"alternative-id":["10.1145\/3771544"],"URL":"https:\/\/doi.org\/10.1145\/3771544","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,17]]},"assertion":[{"value":"2024-04-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-22","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}