{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T11:58:20Z","timestamp":1791460700496,"version":"4.3.4"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:00:00Z","timestamp":1750291200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100002920","name":"Research Grants Council, University Grants Committee","doi-asserted-by":"publisher","award":["SRFS2425-4S03"],"award-info":[{"award-number":["SRFS2425-4S03"]}],"id":[{"id":"10.13039\/501100002920","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001459","name":"Ministry of Education - Singapore","doi-asserted-by":"publisher","award":["Academic Research Fund (AcRF) Tier 1 grant"],"award-info":[{"award-number":["Academic Research Fund (AcRF) Tier 1 grant"]}],"id":[{"id":"10.13039\/501100001459","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,19]]},"abstract":"<jats:p>Websites are critical in today\u2019s digital world, with over 1.11 billion currently active and approximately 252,000 new sites launched daily. Converting website layout design into functional UI code is a time-consuming yet indispensable step of website development. Manual methods of converting visual designs into functional code present significant challenges, especially for non-experts. To explore automatic design-to-code solutions, we first conduct a motivating study on GPT-4o and identify three types of issues in generating UI code: element omission, element distortion, and element misarrangement. We further reveal that a focus on smaller visual segments can help multimodal large language models (MLLMs) mitigate these failures in the generation process.<\/jats:p>\n                  <jats:p>In this paper, we propose DCGen, a divide-and-conquer-based approach to automate the translation of webpage design to UI code. DCGen starts by dividing screenshots into manageable segments, generating code for each segment, and then reassembling them into complete UI code for the entire screenshot. We conduct extensive testing with a dataset comprised of real-world websites and various MLLMs and demonstrate that DCGen achieves up to a 15% improvement in visual similarity and 8% in code similarity for large input images. Human evaluations show that DCGen can help developers implement webpages significantly faster and more similar to the UI designs. To the best of our knowledge, DCGen is the first segment-aware MLLM-based approach for generating UI code directly from screenshots.<\/jats:p>","DOI":"10.1145\/3729364","type":"journal-article","created":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T11:15:34Z","timestamp":1750331734000},"page":"2099-2122","source":"Crossref","is-referenced-by-count":21,"title":["Divide-and-Conquer: Generating UI Code from Screenshots"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-6739-4675","authenticated-orcid":false,"given":"Yuxuan","family":"Wan","sequence":"first","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3935-7328","authenticated-orcid":false,"given":"Chaozheng","family":"Wang","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-4456-072X","authenticated-orcid":false,"given":"Yi","family":"Dong","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9803-8204","authenticated-orcid":false,"given":"Wenxuan","family":"Wang","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6323-1402","authenticated-orcid":false,"given":"Shuqing","family":"Li","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-8798-5667","authenticated-orcid":false,"given":"Yintong","family":"Huo","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3666-5798","authenticated-orcid":false,"given":"Michael","family":"Lyu","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,6,19]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"2024. 17+ Surprising WordPress Statistics You Should Not Miss [2024]. WPDeveloper (2024). https:\/\/wpdeveloper.com\/wordpress-statistics-2024 Accessed: 2024-05-30."},{"key":"e_1_3_1_3_2","unstructured":"2024. Differences Between Mobile App and Web Development. KnowledgeHut (2024). https:\/\/www.knowledgehut.com\/blog\/programming\/differences-mobile-app-web-development"},{"key":"e_1_3_1_4_2","unstructured":"2024. How Many Websites Are There in 2024?(13 Latest Statistics). TechJury (2024). https:\/\/techjury.net\/blog\/how-many-websites-are-there\/ Accessed: 2024-05-30."},{"key":"e_1_3_1_5_2","unstructured":"2024. Website & Mobile App Design: Differences In 2024? Ink bot Design (2024). https:\/\/inkbotdesign.com\/differences-website-mobile-app-design\/"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1049\/iet-sen.2019.0378"},{"issue":"2022","key":"e_1_3_1_7_2","doi-asserted-by":"crossref","first-page":"23716","DOI":"10.52202\/068431-1723","article-title":"Flamingo: a visual language model for few-shot learning","volume":"35","author":"Alayrac Jean-Baptiste","year":"2022","unstructured":"Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. 2022. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems 35 (2022), 23716\u201323736.","journal-title":"Advances in neural information processing systems"},{"key":"e_1_3_1_8_2","unstructured":"Anthropic. 2024. Claude. https:\/\/www.anthropic.com\/claude Accessed: 2024-06-06."},{"key":"e_1_3_1_9_2","unstructured":"Anthropic. 2024. Vision Documentation. https:\/\/docs.anthropic.com\/en\/docs\/vision Accessed: 2024-06-06."},{"key":"e_1_3_1_10_2","doi-asserted-by":"crossref","unstructured":"Shushan Arakelyan Rocktim Jyoti Das Yi Mao and Xiang Ren. 2023. Exploring Distributional Shifts in Large Language Models for Code Analysis. In Conference on Empirical Methods in Natural Language Processing. https:\/\/api.semanticscholar.org\/CorpusID:257557735","DOI":"10.18653\/v1\/2023.emnlp-main.1013"},{"key":"e_1_3_1_11_2","doi-asserted-by":"crossref","unstructured":"Batuhan Asiroglu B\u00fc\u015fta R\u00fcmeysa Mete Eyy\u00fcp Yildiz Yagiz Nal\u00e7akan Alper Sezen Mustafa Da\u011ftekin and Tolga Ensari. 2019. Automatic HTML Code Generation from Mock-Up Images Using Machine Learning Techniques. 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT) (2019) 1\u20134. https:\/\/api.semanticscholar.org\/CorpusID:195223316","DOI":"10.1109\/EBBT.2019.8741736"},{"key":"e_1_3_1_12_2","doi-asserted-by":"crossref","unstructured":"T. Beltramelli. 2018. pix2code: Generating code from a graphical user interface screenshot. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems. 1\u20136.","DOI":"10.1145\/3220134.3220135"},{"key":"e_1_3_1_13_2","doi-asserted-by":"crossref","unstructured":"Hyungjoo Chae Yeonghyeon Kim Seungone Kim Kai Tzu iunn Ong Beong woo Kwak Moohyeon Kim Seonghwan Kim Taeyoon Kwon Jiwan Chung Youngjae Yu and Jinyoung Yeo. 2024. Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models. ArXiv abs\/2404.02575 (2024). https:\/\/api.semanticscholar.org\/CorpusID:268876471","DOI":"10.18653\/v1\/2024.emnlp-main.1253"},{"key":"e_1_3_1_14_2","doi-asserted-by":"crossref","unstructured":"C. Chen T. Su G. Meng Z. Xing and Y. Liu. 2018. From UI design image to GUI skeleton: a neural machine translator to bootstrap mobile GUI implementation. In Proceedings of the 40th International Conference on Software Engineering. 665\u2013676.","DOI":"10.1145\/3180155.3180240"},{"key":"e_1_3_1_15_2","doi-asserted-by":"crossref","unstructured":"Fuxiang Chen Fateme Moradian Fard David Lo and Timofey Bryksin. 2022. On the Transferability of Pre-trained Language Models for Low-Resource Programming Languages. 2022 IEEE\/ACM 30th International Conference on Program Comprehension (ICPC) (2022) 401\u2013412. https:\/\/api.semanticscholar.org\/CorpusID:248266381","DOI":"10.1145\/3524610.3527917"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","unstructured":"Jun Chen Han Guo Kai Yi Boyang Li and Mohamed Elhoseiny. 2022. Visualgpt: Data-efficient adaptation of pretrained language models for image captioning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 18030\u201318040.","DOI":"10.1109\/CVPR52688.2022.01750"},{"key":"e_1_3_1_17_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde Jared Kaplan Harrison Edwards Yura Burda Nicholas Joseph Greg Brockman Alex Ray Raul Puri Gretchen Krueger Michael Petrov Heidy Khlaaf Girish Sastry Pamela Mishkin Brooke Chan Scott Gray Nick Ryder Mikhail Pavlov Alethea Power Lukasz Kaiser Mohammad Bavarian Clemens Winter Philippe Tillet Felipe Petroski Such David W. Cummings Matthias Plappert Fotios Chantzis Elizabeth Barnes Ariel Herbert-Voss William H. Guss Alex Nichol Igor Babuschkin Suchir Balaji Shantanu Jain Andrew Carr Jan Leike Joshua Achiam Vedant Misra Evan Morikawa Alec Radford Matthew M. Knight Miles Brundage Mira Murati Katie Mayer Peter Welinder Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. ArXiv abs\/2107.03374 (2021). https:\/\/api.semanticscholar.org\/CorpusID:235755472"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00530-021-00804-7"},{"key":"e_1_3_1_19_2","unstructured":"Xinyun Chen Maxwell Lin Nathanael Sch\u00e4rli and Denny Zhou. 2023. Teaching Large Language Models to Self-Debug. ArXiv abs\/2304.05128 (2023). https:\/\/api.semanticscholar.org\/CorpusID:258059885"},{"key":"e_1_3_1_20_2","unstructured":"CSS Chopper. 2024. How to Convert PSD to HTML : A Complete Guide. https:\/\/www.csschopper.com\/blog\/how-to-convert-psd-to-html-a-complete-guide\/ Accessed: 31-05-2024."},{"issue":"70","key":"e_1_3_1_21_2","first-page":"1","article-title":"Scaling instruction-finetuned language models","volume":"25","author":"Chung Hyung Won","year":"2024","unstructured":"Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research 25, 70 (2024), 1\u201353.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_1_22_2","doi-asserted-by":"crossref","unstructured":"A. A. J. Cizotto R. C. T. de Souza V. C. Mariani and L. dos Santos Coelho. 2023. Web pages from mockup design based on convolutional neural network and class activation mapping. Multimedia Tools and Applications (2023) 1\u201327.","DOI":"10.1007\/s11042-023-15108-3"},{"key":"e_1_3_1_23_2","unstructured":"Wenliang Dai Junnan Li Dongxu Li Anthony Meng Huat Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung and Steven C. H. Hoi. 2023. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. ArXiv abs\/2305.06500 (2023). https:\/\/api.semanticscholar.org\/CorpusID:258615266"},{"key":"e_1_3_1_24_2","unstructured":"Victor C. Dibia Adam Fourney Gagan Bansal Forough Poursabzi-Sangdeh Han Liu and Saleema Amershi. 2022. Aligning Offline Metrics and Human Judgments of Value of AI-Pair Programmers. ArXiv abs\/2210.16494 (2022). https:\/\/api.semanticscholar.org\/CorpusID:253237523"},{"key":"e_1_3_1_25_2","unstructured":"Hantian Ding Varun Kumar Yuchen Tian Zijian Wang Robert Kwiatkowski Xiaopeng Li Murali Krishna Ramanathan Baishakhi Ray Parminder Bhatia Sudipta Sengupta Dan Roth and Bing Xiang. 2023. A Static Evaluation of Code Completion by Large Language Models. ArXiv abs\/2306.03203 (2023). https:\/\/api.semanticscholar.org\/CorpusID:259088657"},{"key":"e_1_3_1_26_2","unstructured":"Yihong Dong Xue Jiang Zhi Jin and Ge Li. 2023. Self-collaboration Code Generation via ChatGPT. ArXiv abs\/2304.07590 (2023). https:\/\/api.semanticscholar.org\/CorpusID:258179537"},{"key":"e_1_3_1_27_2","first-page":"8469","volume-title":"International Conference on Machine Learning.","author":"Driess Danny","year":"2023","unstructured":"Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. 2023. PaLM-E: An Embodied Multimodal Language Model. In International Conference on Machine Learning. PMLR, 8469\u20138488."},{"key":"e_1_3_1_28_2","unstructured":"Xueying Du Mingwei Liu Kaixin Wang Hanlin Wang Junwei Liu Yixuan Chen Jiayi Feng Chaofeng Sha Xin Peng and Yiling Lou. 2023. ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation. ArXiv abs\/2308.01861 (2023). https:\/\/api.semanticscholar.org\/CorpusID:260439062"},{"key":"e_1_3_1_29_2","unstructured":"Yann Dubois Xuechen Li Rohan Taori Tianyi Zhang Ishaan Gulrajani Jimmy Ba Carlos Guestrin Percy Liang and Tatsunori Hashimoto. 2023. AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback. ArXiv abs\/2305.14387 (2023). https:\/\/api.semanticscholar.org\/CorpusID:258865545"},{"key":"e_1_3_1_30_2","doi-asserted-by":"crossref","unstructured":"Sidong Feng and Chunyang Chen. 2024. Prompting Is All You Need: Automated Android Bug Replay with Large Language Models. In Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering. 1\u201313.","DOI":"10.1145\/3597503.3608137"},{"key":"e_1_3_1_31_2","unstructured":"Shuzheng Gao Xinjie Wen Cuiyun Gao Wenxuan Wang and Michael R. Lyu. 2023. Constructing Effective In-Context Demonstration for Code Intelligence Tasks: An Empirical Study. ArXiv abs\/2304.07575 (2023). https:\/\/api.semanticscholar.org\/CorpusID:263867793"},{"key":"e_1_3_1_32_2","doi-asserted-by":"crossref","unstructured":"Henry Gilbert Michael Sandborn Douglas C. Schmidt Jesse Spencer-Smith and Jules White. 2023. Semantic Compression with Large Language Models. 2023 Tenth International Conference on Social Networks Analysis Management and Security (SNAMS) (2023) 1\u20138. https:\/\/api.semanticscholar.org\/CorpusID:258309482","DOI":"10.1109\/SNAMS60348.2023.10375400"},{"key":"e_1_3_1_33_2","unstructured":"Google. 2024. Gemini API. https:\/\/ai.google.dev\/gemini-api Accessed: 2024-06-06."},{"key":"e_1_3_1_34_2","unstructured":"Google AI. 2024. API Overview. https:\/\/ai.google.dev\/gemini-api\/docs\/api-overview Accessed: 2024-06-06."},{"key":"e_1_3_1_35_2","doi-asserted-by":"crossref","unstructured":"Jian Gu Pasquale Salza and Harald C. Gall. 2022. Assemble Foundation Models for Automatic Code Summarization. 2022 IEEE International Conference on Software Analysis Evolution and Reengineering (SANER) (2022) 935\u2013946. https:\/\/api.semanticscholar.org\/CorpusID:245986582","DOI":"10.1109\/SANER53432.2022.00112"},{"key":"e_1_3_1_36_2","unstructured":"Rishin Haldar and Debajyoti Mukhopadhyay. 2011. Levenshtein Distance Technique in Dictionary Lookup Methods: An Improved Approach. ArXiv abs\/1101.1232 (2011). https:\/\/api.semanticscholar.org\/CorpusID:1049293"},{"key":"e_1_3_1_37_2","unstructured":"Xinying Hou Yanjie Zhao Yue Liu Zhou Yang Kailong Wang Li Li Xiapu Luo David Lo John C. Grundy and Haoyu Wang. 2023. Large Language Models for Software Engineering: A Systematic Literature Review. ArXiv abs\/2308.10620 (2023). https:\/\/api.semanticscholar.org\/CorpusID:261048648"},{"issue":"2018","key":"e_1_3_1_38_2","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1016\/j.jss.2018.08.028","article-title":"Software engineering process models for mobile app development: A systematic literature review","volume":"145","author":"Jabangwe Ronald","year":"2018","unstructured":"Ronald Jabangwe, Henry Edison, and Anh Nguyen-Duc. 2018. Software engineering process models for mobile app development: A systematic literature review. J. Syst. Softw. 145 (2018), 98\u2013111. https:\/\/api.semanticscholar.org\/CorpusID:52961621","journal-title":"J. Syst. Softw."},{"key":"e_1_3_1_39_2","unstructured":"Shuyang Jiang Yuhao Wang and Yu Wang. 2023. SelfEvolve: A Code Evolution Framework via Large Language Models. ArXiv abs\/2306.02907 (2023). https:\/\/api.semanticscholar.org\/CorpusID:259076266"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.9734\/BJAST\/2015\/14975"},{"issue":"2013","key":"e_1_3_1_41_2","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1109\/APSEC.2013.77","article-title":"Designing User Experience for Mobile Apps: Long-Term Product Owner Perspective","volume":"1","author":"Kuusinen Kati","year":"2013","unstructured":"Kati Kuusinen and Tommi Mikkonen. 2013. Designing User Experience for Mobile Apps: Long-Term Product Owner Perspective. 2013 20th Asia-Pacific Software Engineering Conference (APSEC) 1 (2013), 535\u2013540. https:\/\/api.semanticscholar.org\/CorpusID:18632493","journal-title":"2013 20th Asia-Pacific Software Engineering Conference (APSEC)"},{"key":"e_1_3_1_42_2","doi-asserted-by":"crossref","unstructured":"Val\u00e9ria Lelli Arnaud Blouin and Beno\u00eet Baudry. 2015. Classifying and Qualifying GUI Defects. 2015 IEEE 8th International Conference on Software Testing Verification and Validation (ICST) (2015) 1\u201310. https:\/\/api.semanticscholar.org\/CorpusID:2288032","DOI":"10.1109\/ICST.2015.7102582"},{"key":"e_1_3_1_43_2","first-page":"19730","volume-title":"International conference on machine learning","author":"Li Junnan","year":"2023","unstructured":"Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning. PMLR, 19730\u201319742."},{"key":"e_1_3_1_44_2","unstructured":"Jia Li Ge Li Yongming Li and Zhi Jin. 2023. Enabling Programming Thinking in Large Language Models Toward Code Generation. ArXiv abs\/2305.06599 (2023). https:\/\/api.semanticscholar.org\/CorpusID:263896057"},{"key":"e_1_3_1_45_2","doi-asserted-by":"crossref","unstructured":"Tsz On Li Wen yi Zong Yibo Wang Haoye Tian Y. Wang and S. C. Cheung. 2023. Nuances are the Key: Unlocking ChatGPT to Find Failure-Inducing Tests with Differential Prompting. 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE) (2023) 14\u201326. https:\/\/api.semanticscholar.org\/CorpusID:258298446","DOI":"10.1109\/ASE56229.2023.00089"},{"key":"e_1_3_1_46_2","first-page":"1238","article-title":"CCTEST: Testing and Repairing Code Completion Systems","author":"Li Zongjie","year":"2022","unstructured":"Zongjie Li, Chaozheng Wang, Zhibo Liu, Hao Wang, Shuai Wang, and Cuiyun Gao. 2022. CCTEST: Testing and Repairing Code Completion Systems. 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE) (2022), 1238\u20131250. https:\/\/api.semanticscholar.org\/CorpusID:251623193","journal-title":"2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE)"},{"key":"e_1_3_1_47_2","doi-asserted-by":"crossref","unstructured":"Antonio Mastropaolo Luca Pascarella and Gabriele Bavota. 2022. Using Deep Learning to Generate Complete Log Statements. 2022 IEEE\/ACM 44th International Conference on Software Engineering (ICSE) (2022) 2279\u20132290. https:\/\/api.semanticscholar.org\/CorpusID:245906103","DOI":"10.1145\/3510003.3511561"},{"key":"e_1_3_1_48_2","doi-asserted-by":"crossref","unstructured":"Antonio Mastropaolo Simone Scalabrino Nathan Cooper David Nader-Palacio Denys Poshyvanyk Rocco Oliveto and Gabriele Bavota. 2021. Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks. 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE) (2021) 336\u2013347. https:\/\/api.semanticscholar.org\/CorpusID:231786586","DOI":"10.1109\/ICSE43902.2021.00041"},{"issue":"7","key":"e_1_3_1_49_2","first-page":"3523","article-title":"Image segmentation using deep learning: A survey","volume":"44","author":"Minaee Shervin","year":"2021","unstructured":"Shervin Minaee, Yuri Boykov, Fatih Porikli, Antonio Plaza, Nasser Kehtarnavaz, and Demetri Terzopoulos. 2021. Image segmentation using deep learning: A survey. IEEE transactions on pattern analysis and machine intelligence 44, 7 (2021), 3523\u20133542.","journal-title":"IEEE transactions on pattern analysis and machine intelligence"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2844788"},{"key":"e_1_3_1_51_2","doi-asserted-by":"crossref","unstructured":"Kevin Moran Boyang Li Carlos Bernal-C\u00e1rdenas Dan Jelf and Denys Poshyvanyk. 2018. Automated Reporting of GUI Design Violations for Mobile Apps. 2018 IEEE\/ACM 40th International Conference on Software Engineering (ICSE) (2018) 165\u2013175. https:\/\/api.semanticscholar.org\/CorpusID:3634687","DOI":"10.1145\/3180155.3180246"},{"key":"e_1_3_1_52_2","doi-asserted-by":"crossref","unstructured":"S. Natarajan and C. Csallner. 2018. P2A: A tool for converting pixels to animated mobile application user interfaces. In Proceedings of the 5th International Conference on Mobile Software Engineering and Systems. 224\u2013235.","DOI":"10.1145\/3197231.3197249"},{"key":"e_1_3_1_53_2","doi-asserted-by":"crossref","unstructured":"T. A. Nguyen and C. Csallner. 2015. Reverse engineering mobile application user interfaces with remaui (t). In 2015 30th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 248\u2013259.","DOI":"10.1109\/ASE.2015.32"},{"key":"e_1_3_1_54_2","unstructured":"Erik Nijkamp Bo Pang Hiroaki Hayashi Lifu Tu Haiquan Wang Yingbo Zhou Silvio Savarese and Caiming Xiong. 2022. CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. In International Conference on Learning Representations. https:\/\/api.semanticscholar.org\/CorpusID:252668917"},{"key":"e_1_3_1_55_2","unstructured":"OpenAI. [n.d.]. GPT-4 System Card. https:\/\/openai.com\/index\/gpt-4v-system-card\/. Accessed: 2024-05-30."},{"key":"e_1_3_1_56_2","unstructured":"OpenAI. 2024. Hello GPT-4o. https:\/\/openai.com\/index\/hello-gpt-4o\/ Accessed: 2024-06-06."},{"key":"e_1_3_1_57_2","unstructured":"OpenAI. 2024. Vision Guide. https:\/\/platform.openai.com\/docs\/guides\/vision Accessed: 2024-06-06."},{"key":"e_1_3_1_58_2","doi-asserted-by":"crossref","unstructured":"Victor Le Pochat Tom van Goethem Samaneh Tajalizadehkhoob Maciej Korczy\u0144ski and Wouter Joosen. 2018. Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation. Proceedings 2019 Network and Distributed System Security Symposium. https:\/\/api.semanticscholar.org\/CorpusID:56367624","DOI":"10.14722\/ndss.2019.23386"},{"key":"e_1_3_1_59_2","unstructured":"Alec Radford Jong Wook Kim Chris Hallacy Aditya Ramesh Gabriel Goh Sandhini Agarwal Girish Sastry Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In International Conference on Machine Learning. https:\/\/api.semanticscholar.org\/CorpusID:231591445"},{"key":"e_1_3_1_60_2","unstructured":"RapidFuzz. 2024. Levenshtein Distance \u2014 RapidFuzz. https:\/\/rapidfuzz.github.io\/Levenshtein\/levenshtein.html Accessed: 2024-02-26."},{"key":"e_1_3_1_61_2","doi-asserted-by":"crossref","unstructured":"Joseph Redmon Santosh Kumar Divvala Ross B. Girshick and Ali Farhadi. 2015. You Only Look Once: Unified Real-Time Object Detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015) 779\u2013788. https:\/\/api.semanticscholar.org\/CorpusID:206594738","DOI":"10.1109\/CVPR.2016.91"},{"key":"e_1_3_1_62_2","unstructured":"Raj Sanghvi. 2024. Differences Between Web and Mobile App Development. BitCot (2024). https:\/\/www.bitcot.com\/blog\/differences-between-web-and-mobile-app-development\/"},{"key":"e_1_3_1_63_2","unstructured":"Chenglei Si Yanzhe Zhang Zhengyuan Yang Ruibo Liu and Diyi Yang. 2024. Design2Code: How Far Are We From Automating Front-End Engineering? ArXiv abs\/2403.03163 (2024). https:\/\/api.semanticscholar.org\/CorpusID:268248801"},{"key":"e_1_3_1_64_2","unstructured":"Shengbang Tong Zhuang Liu Yuexiang Zhai Yi Ma Yann LeCun and Saining Xie. 2024. Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs. ArXiv abs\/2401.06209 (2024). https:\/\/api.semanticscholar.org\/CorpusID:266976992"},{"issue":"2021","key":"e_1_3_1_65_2","first-page":"200","article-title":"Multimodal few-shot learning with frozen language models","volume":"34","author":"Tsimpoukelli Maria","year":"2021","unstructured":"Maria Tsimpoukelli, Jacob L Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. 2021. Multimodal few-shot learning with frozen language models. Advances in Neural Information Processing Systems 34 (2021), 200\u2013212.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_66_2","unstructured":"Yuxuan Wan Yi Dong Jingyu Xiao Yintong Huo Wenxuan Wang and Michael R. Lyu. 2024. MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs. ArXiv abs\/2412.15310 (2024). https:\/\/api.semanticscholar.org\/CorpusID:274965541"},{"key":"e_1_3_1_67_2","unstructured":"Jason Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Ed Huai hsin Chi F. Xia Quoc Le and Denny Zhou. 2022. Chain of Thought Prompting Elicits Reasoning in Large Language Models. ArXiv abs\/2201.11903 (2022). https:\/\/api.semanticscholar.org\/CorpusID:246411621"},{"issue":"2022","key":"e_1_3_1_68_2","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824\u201324837.","journal-title":"Advances in neural information processing systems"},{"key":"e_1_3_1_69_2","doi-asserted-by":"crossref","unstructured":"J. Wu X. Zhang J. Nichols and J. P. Bigham. 2021. Screen parsing: Towards reverse engineering of UI models from screenshots. In The 34th Annual ACM Symposium on User Interface Software and Technology. 470\u2013483.","DOI":"10.1145\/3472749.3474763"},{"key":"e_1_3_1_70_2","unstructured":"Jingyu Xiao Yuxuan Wan Yintong Huo Zhiyao Xu and Michael R. Lyu. 2024. Interaction2Code: How Far Are We From Automatic Interactive Webpage Generation? ArXiv abs\/2411.03292 (2024). https:\/\/api.semanticscholar.org\/CorpusID:273821629"},{"issue":"8","key":"e_1_3_1_71_2","first-page":"e2369","article-title":"image2emmet: Automatic code generation from web user interface image","volume":"33","author":"Xu Y.","year":"2021","unstructured":"Y. Xu, L. Bo, X. Sun, B. Li, J. Jiang, and W. Zhou. 2021. image2emmet: Automatic code generation from web user interface image. Journal of Software: Evolution and Process 33, 8 (2021), e2369.","journal-title":"Journal of Software: Evolution and Process"},{"key":"e_1_3_1_72_2","unstructured":"Zhengyuan Yang Linjie Li Kevin Lin Jianfeng Wang Chung-Ching Lin Zicheng Liu and Lijuan Wang. 2023. The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision). ArXiv abs\/2309.17421 (2023). https:\/\/api.semanticscholar.org\/CorpusID:263310951"},{"key":"e_1_3_1_73_2","unstructured":"Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Xu and Enhong Chen. 2023. A Survey on Multimodal Large Language Models. ArXiv abs\/2306.13549 (2023). https:\/\/api.semanticscholar.org\/CorpusID:259243718"},{"key":"e_1_3_1_74_2","doi-asserted-by":"crossref","unstructured":"Hao Yu Bo Shen Dezhi Ran Jiaxin Zhang Qi Zhang Yu Ma Guangtai Liang Ying Li Tao Xie and Qianxiang Wang. 2023. CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models. In International Conference on Software Engineering. https:\/\/api.semanticscholar.org\/CorpusID:256459413","DOI":"10.1145\/3597503.3623316"},{"key":"e_1_3_1_75_2","doi-asserted-by":"crossref","unstructured":"Clemens Zeidler Christof Lutteroth Wolfgang Stuerzlinger and Gerald Weber. 2013. Evaluating Direct Manipulation Operations for Constraint-Based Layout. In IFIP TC13 International Conference on Human-Computer Interaction. https:\/\/api.semanticscholar.org\/CorpusID:8243987","DOI":"10.1007\/978-3-642-40480-1_35"},{"key":"e_1_3_1_76_2","unstructured":"Chunting Zhou Pengfei Liu Puxin Xu Srini Iyer Jiao Sun Yuning Mao Xuezhe Ma Avia Efrat Ping Yu L. Yu Susan Zhang Gargi Ghosh Mike Lewis Luke Zettlemoyer and Omer Levy. 2023. LIMA: Less Is More for Alignment. ArXiv abs\/2305.11206 (2023). https:\/\/api.semanticscholar.org\/CorpusID:258822910"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3729364","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T10:58:32Z","timestamp":1787482712000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3729364"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,19]]},"references-count":75,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2025,6,19]]}},"alternative-id":["10.1145\/3729364"],"URL":"https:\/\/doi.org\/10.1145\/3729364","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,19]]}}}