{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,27]],"date-time":"2026-06-27T20:22:28Z","timestamp":1782591748877,"version":"3.54.5"},"reference-count":40,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T00:00:00Z","timestamp":1646352000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Department of Communications and Networking"},{"name":"Finnish Center for Artificial Intelligence (FCAI), Academy of Finland"},{"name":"HumaneAI Net"},{"DOI":"10.13039\/501100003816","name":"Huawei Technologies","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003816","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2022,3,31]]},"abstract":"<jats:p>\n            A key objective in the design of\n            <jats:bold>graphical user interfaces (GUIs)<\/jats:bold>\n            is to ensure consistency across screens of the same product. However, designing a compliant layout is time-consuming and can distract designers from creative thinking. This paper studies layout recommendation methods that fulfill such consistency requirements using machine learning. Given a desired element type and size, the methods suggest element placements following real-world GUI design processes. Consistency requirements are given implicitly through previous layouts from which patterns are to be learned, comparable to existing screens of a software product. We adopt two recently proposed methods for this task, a\n            <jats:bold>Graph Neural Network (GNN)<\/jats:bold>\n            and a Transformer model, and compare them with a custom approach based on sequence alignment and\n            <jats:bold>nearest neighbor search (kNN)<\/jats:bold>\n            . The methods were tested on handcrafted datasets with explicit layout patterns, as well as large-scale public datasets of diverse mobile design layouts. Our results show that our instance-based learning algorithm outperforms both neural network approaches. Ultimately, this work contributes to establishing smarter design tools for professional designers with explainable algorithms that increase their efficacy.\n          <\/jats:p>","DOI":"10.1145\/3490034","type":"journal-article","created":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T10:14:01Z","timestamp":1646388841000},"page":"1-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Learning GUI Completions with User-defined Constraints"],"prefix":"10.1145","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6949-5820","authenticated-orcid":false,"given":"Lukas","family":"Br\u00fcckner","sequence":"first","affiliation":[{"name":"Aalto University, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luis A.","family":"Leiva","sequence":"additional","affiliation":[{"name":"University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antti","family":"Oulasvirta","sequence":"additional","affiliation":[{"name":"Aalto University, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,3,4]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-36901-5_42"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/2034691.2034695"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397481.3450652"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376553"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3126594.3126651"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/964442.964461"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2010.05.005"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/1294211.1294253"},{"key":"e_1_3_3_10_2","volume-title":"Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)","author":"Grave Edouard","year":"2018","unstructured":"Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. Learning word vectors for 157 languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Japan. https:\/\/www.aclweb.org\/anthology\/L18-1550."},{"key":"e_1_3_3_11_2","article-title":"Layout generation and completion with self-attention","volume":"2006","author":"Gupta Kamal","year":"2020","unstructured":"Kamal Gupta, Alessandro Achille, Justin Lazarow, Larry Davis, Vijay Mahadevan, and Abhinav Shrivastava. 2020. Layout generation and completion with self-attention. CoRR abs\/2006.14615 (2020). arxiv:2006.14615https:\/\/arxiv.org\/abs\/2006.14615.","journal-title":"CoRR"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/0307-904X(94)00033-3"},{"key":"e_1_3_3_13_2","volume-title":"Proceedings of the 27th Australasian Conference on Information Systems, ACIS 2016","author":"Hilliard Bruce","year":"2016","unstructured":"Bruce Hilliard, Jocelyn Armarego, and Tanya McGill. 2016. Optimising visual layout for training and learning technologies. In Proceedings of the 27th Australasian Conference on Information Systems, ACIS 2016. University of Wollongong, Faculty of Business."},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/360825.360861"},{"key":"e_1_3_3_15_2","article-title":"Image generation from scene graphs","volume":"1804","author":"Johnson Justin","year":"2018","unstructured":"Justin Johnson, Agrim Gupta, and Li Fei-Fei. 2018. Image generation from scene graphs. CoRR abs\/1804.01622 (2018). arxiv:1804.01622http:\/\/arxiv.org\/abs\/1804.01622.","journal-title":"CoRR"},{"key":"e_1_3_3_16_2","article-title":"LayoutVAE: Stochastic scene layout generation from a label set","volume":"1907","author":"Jyothi Akash Abdu","year":"2019","unstructured":"Akash Abdu Jyothi, Thibaut Durand, Jiawei He, Leonid Sigal, and Greg Mori. 2019. LayoutVAE: Stochastic scene layout generation from a label set. CoRR abs\/1907.10719 (2019).","journal-title":"CoRR"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979262"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-50578-3_2"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376327"},{"key":"e_1_3_3_20_2","article-title":"Neural design network: Graphic layout generation with constraints","volume":"1912","author":"Lee Hsin-Ying","year":"2019","unstructured":"Hsin-Ying Lee, Weilong Yang, Lu Jiang, Madison Le, Irfan Essa, Haifeng Gong, and Ming-Hsuan Yang. 2019. Neural design network: Graphic layout generation with constraints. CoRR abs\/1912.09421 (2019). arxiv:1912.09421http:\/\/arxiv.org\/abs\/1912.09421.","journal-title":"CoRR"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2166966.2167028"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3406324.3410710"},{"key":"e_1_3_3_23_2","volume-title":"7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\u20139, 2019","author":"Li Jianan","year":"2019","unstructured":"Jianan Li, Jimei Yang, Aaron Hertzmann, Jianming Zhang, and Tingfa Xu. 2019. LayoutGAN: Generating graphic layouts with wireframe discriminators. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\u20139, 2019. OpenReview.net. https:\/\/openreview.net\/forum?id=HJxB5sRcFQ."},{"key":"e_1_3_3_24_2","first-page":"1","article-title":"Auto completion of user interface layout design using transformer-based tree decoders","author":"Li Yang","year":"2020","unstructured":"Yang Li, Julien Amelot, Xin Zhou, Samy Bengio, and Si Si. 2020. Auto completion of user interface layout design using transformer-based tree decoders. ArXiv (2020), 1\u201311.","journal-title":"ArXiv"},{"key":"e_1_3_3_25_2","volume-title":"7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\u20139, 2019","author":"Loshchilov Ilya","year":"2019","unstructured":"Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\u20139, 2019. OpenReview.net. https:\/\/openreview.net\/forum?id=Bkg6RiCqY7."},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/0953-5438(93)90003-C"},{"key":"e_1_3_3_27_2","volume-title":"Advances in Neural Information Processing Systems","author":"M\u00fcller Rafael","year":"2019","unstructured":"Rafael M\u00fcller, Simon Kornblith, and Geoffrey E. Hinton. 2019. When does label smoothing help?. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch\u00e9-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32. Curran Associates, Inc.https:\/\/proceedings.neurips.cc\/paper\/2019\/file\/f1748d6b0fd9d439f71450117eba2725-Paper.pdf."},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/316542.316550"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.48"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2020.2969687"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1080\/001401300184332"},{"key":"e_1_3_3_32_2","volume-title":"Creativity and Technology in the Age of AI","author":"Pfeiffer Andreas","year":"2018","unstructured":"Andreas Pfeiffer. 2018. Creativity and Technology in the Age of AI. Technical Report. Pfeiffer Consulting. Retrieved 18.09.2020 from http:\/\/www.pfeifferreport.com\/essays\/creativity-and-technology-in-the-age-of-ai\/."},{"key":"e_1_3_3_33_2","volume-title":"Optimising User Interface Layouts for Design System Compliance","author":"Santala Simo","year":"2020","unstructured":"Simo Santala. 2020. Optimising User Interface Layouts for Design System Compliance. Master\u2019s thesis. Aalto University."},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376593"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_3_36_2","series-title":"SIGGRAPH\u201919","volume-title":"ACM SIGGRAPH 2019 Posters","author":"Tabata Sou","year":"2019","unstructured":"Sou Tabata, Hiroki Yoshihara, Haruka Maeda, and Kei Yokoyama. 2019. Automatic layout generation for graphical design magazines. In ACM SIGGRAPH 2019 Posters (Los Angeles, California) (SIGGRAPH\u201919). Association for Computing Machinery, New York, NY, USA, Article 9, 2 pages."},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380127"},{"key":"e_1_3_3_38_2","first-page":"3780","article-title":"Sketchplore: Sketch and explore layout designs with an optimiser","volume":"07","author":"Todi Kashyap","year":"2016","unstructured":"Kashyap Todi, Daryl Weir, and Antti Oulasvirta. 2016. Sketchplore: Sketch and explore layout designs with an optimiser. Conference on Human Factors in Computing Systems - Proceedings 07\u201312-May- (2016), 3780\u20133783.","journal-title":"Conference on Human Factors in Computing Systems - Proceedings"},{"key":"e_1_3_3_39_2","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc.https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf."},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/2642918.2647398"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322971"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3490034","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3490034","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:22Z","timestamp":1750188682000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3490034"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,4]]},"references-count":40,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,3,31]]}},"alternative-id":["10.1145\/3490034"],"URL":"https:\/\/doi.org\/10.1145\/3490034","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,4]]},"assertion":[{"value":"2021-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-03-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}