{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T02:18:09Z","timestamp":1775787489777,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":41,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,4,15]],"date-time":"2024-04-15T00:00:00Z","timestamp":1713139200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,4,15]]},"DOI":"10.1145\/3643661.3643953","type":"proceedings-article","created":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T11:51:23Z","timestamp":1723031483000},"page":"7-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":19,"title":["Assured Offline LLM-Based Software Engineering"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-4763-0396","authenticated-orcid":false,"given":"Nadia","family":"Alshahwan","sequence":"first","affiliation":[{"name":"Meta Platforms Inc, Menlo Park, California, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5864-4488","authenticated-orcid":false,"given":"Mark","family":"Harman","sequence":"additional","affiliation":[{"name":"Meta Platforms Inc, Menlo Park, California, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-9359-0949","authenticated-orcid":false,"given":"Inna","family":"Harper","sequence":"additional","affiliation":[{"name":"Meta Platforms Inc, Menlo Park, California, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-5311-762X","authenticated-orcid":false,"given":"Alexandru","family":"Marginean","sequence":"additional","affiliation":[{"name":"Meta Platforms Inc, Menlo Park, California, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-4204-5185","authenticated-orcid":false,"given":"Shubho","family":"Sengupta","sequence":"additional","affiliation":[{"name":"Meta Platforms Inc, Menlo Park, California, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8825-6986","authenticated-orcid":false,"given":"Eddy","family":"Wang","sequence":"additional","affiliation":[{"name":"Meta Platforms Inc, Menlo Park, California, USA"}]}],"member":"320","published-online":{"date-parts":[[2024,8,7]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3387940.3392089"},{"key":"e_1_3_2_1_2_1","volume-title":"Software Testing Research Challenges: An Industrial Perspective. In 2023 IEEE Conference on Software Testing, Verification and Validation (ICST","author":"Alshahwan Nadia","year":"2023","unstructured":"Nadia Alshahwan, Mark Harman, and Alexandru Marginean. 2023. Software Testing Research Challenges: An Industrial Perspective. In 2023 IEEE Conference on Software Testing, Verification and Validation (ICST 2023). IEEE, 1--10."},{"key":"e_1_3_2_1_3_1","volume-title":"Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. ACM, 59","author":"Anand Saswat","year":"2012","unstructured":"Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated concolic testing of smartphone apps. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. ACM, 59."},{"key":"e_1_3_2_1_4_1","volume-title":"Pro Business Applications with Silverlight 5","author":"Anderson Chris","unstructured":"Chris Anderson. 2012. The Model-View-Viewmodel (MVVM) design pattern. In Pro Business Applications with Silverlight 5. Springer, 461--499."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2827066"},{"key":"e_1_3_2_1_6_1","unstructured":"Rajendra T Dodhiawala NS Sridharan Peter Raulefs and Cynthia Pickering. 1989. Real-Time AI Systems: A Definition and An Architecture.. In IJCAI. Citeseer 256--264."},{"key":"e_1_3_2_1_7_1","volume-title":"ICSE Future of Software Engineering (FoSE","author":"Fan Angela","year":"2023","unstructured":"Angela Fan, Beliz Gokkaya, Mitya Lyubarskiy, Mark Harman, Shubho Sengupta, Shin Yoo, and Jie Zhang. 2023. Large Language Models for Software Engineering: Survey and Open Problems. In ICSE Future of Software Engineering (FoSE 2023). To Appear."},{"key":"e_1_3_2_1_8_1","volume-title":"Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (FSE","author":"First Emily","year":"2023","unstructured":"Emily First, Markus Rabe, Talia Ringer, and Yuriy Brun. 2023. Baldur: Whole-proof generation and repair with large language models. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (FSE 2023). 1229--1241."},{"key":"e_1_3_2_1_9_1","volume-title":"Design Patterns","author":"Gamma Erich","unstructured":"Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. 1995. Design Patterns. Addison-Wesley."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1145\/3318162","article-title":"Automated program repair","volume":"62","author":"Goues Claire Le","year":"2019","unstructured":"Claire Le Goues, Michael Pradel, and Abhik Roychoudhury. 2019. Automated program repair. Commun. ACM 62, 12 (2019), 56--65.","journal-title":"Commun. ACM"},{"key":"e_1_3_2_1_11_1","volume-title":"Why Source Code Analysis and Manipulation Will Always Be Important. In 10th IEEE International Working Conference on Source Code Analysis and Manipulation","author":"Harman Mark","year":"2010","unstructured":"Mark Harman. 2010. Why Source Code Analysis and Manipulation Will Always Be Important. In 10th IEEE International Working Conference on Source Code Analysis and Manipulation. Timisoara, Romania, 7--19."},{"key":"e_1_3_2_1_12_1","volume-title":"3rd IEEE\/ACM International Workshop on Automated Program Repair, APR@ICSE 2022","author":"Harman Mark","year":"2022","unstructured":"Mark Harman. 2022. Scaling Genetic Improvement and Automated Program Repair (keynote paper). In 3rd IEEE\/ACM International Workshop on Automated Program Repair, APR@ICSE 2022, Pittsburgh, PA, USA, May 19, 2022. IEEE, 1--7. 10.1145\/3524459.3527353"},{"key":"e_1_3_2_1_13_1","volume-title":"4th International Workshop on Emerging Trends in Software Metrics (WeTSOM","author":"Harman Mark","year":"2013","unstructured":"Mark Harman, John Clark, and Mel \u00d3 Cinn\u00e9die. 2013. Dynamic Adaptive Search Based Software Engineering Needs Fast Approximate Metrics (Keynote Paper). In 4th International Workshop on Emerging Trends in Software Metrics (WeTSOM 2013). San Francisco, USA."},{"key":"e_1_3_2_1_14_1","volume-title":"8th IEEE International Conference on Software Testing, Verification and Validation (ICST","author":"Harman Mark","year":"2015","unstructured":"Mark Harman, Yue Jia, and Yuanyuan Zhang. 2015. Achievements, open problems and challenges for search based software testing (keynote Paper). In 8th IEEE International Conference on Software Testing, Verification and Validation (ICST 2015). Graz, Austria."},{"key":"e_1_3_2_1_15_1","first-page":"14","article-title":"Search Based Software Engineering","volume":"43","author":"Harman Mark","year":"2001","unstructured":"Mark Harman and Bryan F. Jones. 2001. Search Based Software Engineering. Information and Software Technology 43, 14 (Dec. 2001), 833--839.","journal-title":"Information and Software Technology"},{"key":"e_1_3_2_1_16_1","first-page":"1","article-title":"Search Based Software Engineering","volume":"45","author":"Harman Mark","year":"2012","unstructured":"Mark Harman, Afshin Mansouri, and Yuanyuan Zhang. 2012. Search Based Software Engineering: Trends, Techniques and Applications. Comput. Surveys 45, 1 (November 2012), 11:1--11:61.","journal-title":"Trends, Techniques and Applications. Comput. Surveys"},{"key":"e_1_3_2_1_17_1","volume-title":"Encyclopedia of Information Science and Technology","author":"Hasan Osman","unstructured":"Osman Hasan and Sofiene Tahar. 2015. Formal verification methods. In Encyclopedia of Information Science and Technology, Third Edition. IGI global, 7162--7170."},{"key":"e_1_3_2_1_18_1","unstructured":"Maurice Herlihy Nir Shavit Victor Luchangco and Michael Spear. 2020. The art of multiprocessor programming. Newnes."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/j.neucom.2021.04.112","article-title":"Online learning: A comprehensive survey","volume":"459","author":"Hoi Steven CH","year":"2021","unstructured":"Steven CH Hoi, Doyen Sahoo, Jing Lu, and Peilin Zhao. 2021. Online learning: A comprehensive survey. Neurocomputing 459 (2021), 249--289.","journal-title":"Neurocomputing"},{"key":"e_1_3_2_1_20_1","volume-title":"Large Language Models for Software Engineering: A Systematic Literature Review. arXiv prnote arXiv:2308.10620","author":"Hou Xinyi","year":"2023","unstructured":"Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2023. Large Language Models for Software Engineering: A Systematic Literature Review. arXiv prnote arXiv:2308.10620 (2023)."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3067156"},{"key":"e_1_3_2_1_22_1","volume-title":"Realtime systems: Design principles for distributed embedded applications","author":"Kopetz Hermann","unstructured":"Hermann Kopetz and Wilfried Steiner. 2022. Real-Time Communication. In Realtime systems: Design principles for distributed embedded applications. Springer, 177--200."},{"key":"e_1_3_2_1_23_1","volume-title":"Genetic Programming: On the Programming of Computers by Means of Natural Selection","author":"Koza J. R.","year":"1992","unstructured":"J. R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA."},{"key":"e_1_3_2_1_24_1","first-page":"1","article-title":"Optimising Existing Software with Genetic Programming","volume":"19","author":"Langdon William B.","year":"2015","unstructured":"William B. Langdon and Mark Harman. 2015. Optimising Existing Software with Genetic Programming. IEEE Transactions on Evolutionary Computation (TEVC) 19, 1 (Feb 2015), 118--135.","journal-title":"IEEE Transactions on Evolutionary Computation (TEVC)"},{"key":"e_1_3_2_1_25_1","volume-title":"36th International Conference on Software Engineering","author":"Manotas Irene","year":"2014","unstructured":"Irene Manotas, Lori Pollock, and James Clause. 2014. SEEDS: A Software Engineer's Energy-optimization Decision Support Framework. In 36th International Conference on Software Engineering (Hyderabad, India). ACM, New York, NY, USA, 503--514."},{"key":"e_1_3_2_1_26_1","volume-title":"International Conference on Software Engineering (ICSE) Software Engineering in Practice (SEIP) track","author":"Marginean Alexandru","year":"2019","unstructured":"Alexandru Marginean, Johannes Bader, Satish Chandra, Mark Harman, Yue Jia, Ke Mao, Alexander Mols, and Andrew Scott. 2019. SapFix: Automated End-to-End Repair at Scale. In International Conference on Software Engineering (ICSE) Software Engineering in Practice (SEIP) track. Montreal, Canada."},{"key":"e_1_3_2_1_27_1","volume-title":"4th Software Quality Management Conference","author":"Milankovic-Atkinson Maja","year":"1996","unstructured":"Maja Milankovic-Atkinson and Elli Georgiadou. 1996. Metrics for reuse of object-oriented software. In 4th Software Quality Management Conference. Cambridge."},{"key":"e_1_3_2_1_28_1","volume-title":"Foundations of Software Engineering (FSE","author":"Murali Vijayaraghavan","year":"2024","unstructured":"Vijayaraghavan Murali, Chandra Maddila, Imad Ahmad, Michael Bolin, Daniel Cheng, Negar Ghorbani, Renuka Fernandez, Nachiappan Nagappan, and Peter C Rigby. 2024. CodeCompose: A Large-Scale Industrial Deployment of AI-assisted Code Authoring. In Foundations of Software Engineering (FSE 2024). Porto de Galinhas, Brazil. Earlier version available as arXiv:2305.12050."},{"key":"e_1_3_2_1_29_1","volume-title":"Proceedings of the 32nd IEEE\/ACM International Conference on Automated Software Engineering, ASE 2017","author":"Paix\u00e3o Matheus","year":"2017","unstructured":"Matheus Paix\u00e3o, Jens Krinke, DongGyun Han, Chaiyong Ragkhitwetsagul, and Mark Harman. 2017. Are developers aware of the architectural impact of their changes?. In Proceedings of the 32nd IEEE\/ACM International Conference on Automated Software Engineering, ASE 2017, Urbana, IL, USA, October 30 - November 03, 2017. 95--105."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2017.2693219"},{"key":"e_1_3_2_1_31_1","volume-title":"2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 13--24","author":"Saha Seemanta","unstructured":"Seemanta Saha, Ripon K. Saha, and Mukul R. Prasad. 2019. Harnessing evolution for multi-hunk program repair. In 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 13--24."},{"key":"e_1_3_2_1_32_1","volume-title":"2010 International Conference on Computer and Communication Technology (ICCCT). IEEE, 562--568","author":"Shashank S Phani","year":"2010","unstructured":"S Phani Shashank, Praneeth Chakka, and D Vijay Kumar. 2010. A systematic literature survey of integration testing in component-based software engineering. In 2010 International Conference on Computer and Communication Technology (ICCCT). IEEE, 562--568."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1145\/2514.2517","article-title":"Response time and display rate in human performance with computers","volume":"16","author":"Shneiderman Ben","year":"1984","unstructured":"Ben Shneiderman. 1984. Response time and display rate in human performance with computers. ACM Computing Surveys (CSUR) 16, 3 (1984), 265--285.","journal-title":"ACM Computing Surveys (CSUR)"},{"key":"e_1_3_2_1_34_1","article-title":"Genetic programming for shader simplification","volume":"30","author":"Pitchaya","year":"2011","unstructured":"Pitchaya Sitthi-amorn, Nicholas Modly, Westley Weimer, and Jason Lawrence. 2011. Genetic programming for shader simplification. ACM Transactions on Graphics 30, 6 (2011), 152:1--152:11.","journal-title":"ACM Transactions on Graphics"},{"key":"e_1_3_2_1_35_1","volume-title":"Software testing with large language model: Survey, landscape, and vision. arXiv preprint arXiv:2307.07221","author":"Wang Junjie","year":"2023","unstructured":"Junjie Wang, Yuchao Huang, Chunyang Chen, Zhe Liu, Song Wang, and Qing Wang. 2023. Software testing with large language model: Survey, landscape, and vision. arXiv preprint arXiv:2307.07221 (2023)."},{"key":"e_1_3_2_1_36_1","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824--24837.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1109\/TEVC.2010.2083669","article-title":"Evolutionary Improvement of Programs","volume":"15","author":"White David Robert","year":"2011","unstructured":"David Robert White, Andrea Arcuri, and John A. Clark. 2011. Evolutionary Improvement of Programs. IEEE Transactions on Evolutionary Computation (TEVC) 15, 4 (2011), 515--538.","journal-title":"IEEE Transactions on Evolutionary Computation (TEVC)"},{"key":"e_1_3_2_1_38_1","volume-title":"Searching for Resource-Efficient Programs: Low-Power Pseudorandom Number Generators. In 2008 Genetic and Evolutionary Computation Conference (GECCO","author":"White David R.","year":"2008","unstructured":"David R. White, John Clark, Jeremy Jacob, and Simon Poulding. 2008. Searching for Resource-Efficient Programs: Low-Power Pseudorandom Number Generators. In 2008 Genetic and Evolutionary Computation Conference (GECCO 2008). Atlanta, USA, 1775--1782."},{"key":"e_1_3_2_1_39_1","volume-title":"Genetic and evolutionary computation conference (GECCO","author":"Wu Fan","year":"2015","unstructured":"Fan Wu, Mark Harman, Yue Jia, Jens Krinke, and Westley Weimer. 2015. Deep Parameter Optimisation. In Genetic and evolutionary computation conference (GECCO 2015). Madrid, Spain, 1375--1382."},{"key":"e_1_3_2_1_40_1","unstructured":"Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629 [cs.CL]"},{"key":"e_1_3_2_1_41_1","unstructured":"Shuyin Zhao. [n. d.]. GitHub Copilot now has a better AI model and new capabilities. https:\/\/github.blog\/2023-02-14-github-copilot-now-has-a-better-ai-model-and-new-capabilities\/"}],"event":{"name":"InteNSE '24: ACM\/IEEE 2nd International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering","location":"Lisbon Portugal","acronym":"InteNSE '24","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering","IEEE CS","Faculty of Engineering of University of Porto"]},"container-title":["Proceedings of the ACM\/IEEE 2nd International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643661.3643953","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3643661.3643953","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:05:32Z","timestamp":1750291532000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643661.3643953"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,15]]},"references-count":41,"alternative-id":["10.1145\/3643661.3643953","10.1145\/3643661"],"URL":"https:\/\/doi.org\/10.1145\/3643661.3643953","relation":{},"subject":[],"published":{"date-parts":[[2024,4,15]]},"assertion":[{"value":"2024-08-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}