{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T23:22:29Z","timestamp":1776122549461,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":57,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,5,11]],"date-time":"2024-05-11T00:00:00Z","timestamp":1715385600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,5,11]]},"DOI":"10.1145\/3613904.3641920","type":"proceedings-article","created":{"date-parts":[[2024,5,11]],"date-time":"2024-05-11T08:39:12Z","timestamp":1715416752000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["Jigsaw: Supporting Designers to Prototype Multimodal Applications by Chaining AI Foundation Models"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0116-0463","authenticated-orcid":false,"given":"David Chuan-En","family":"Lin","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1824-0243","authenticated-orcid":false,"given":"Nikolas","family":"Martelaro","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, United States"}]}],"member":"320","published-online":{"date-parts":[[2024,5,11]]},"reference":[{"key":"e_1_3_3_2_1_1","unstructured":"2013. ComfyUI. https:\/\/github.com\/comfyanonymous\/ComfyUI."},{"key":"e_1_3_3_2_2_1","unstructured":"2013. FlowiseAI. https:\/\/github.com\/FlowiseAI\/Flowise."},{"key":"e_1_3_3_2_3_1","unstructured":"2013. Langflow. https:\/\/github.com\/logspace-ai\/langflow."},{"key":"e_1_3_3_2_4_1","volume-title":"Retrieved","year":"2023","unstructured":"2023. AI Magic Tools - Runway. Retrieved December 10, 2023 from https:\/\/runwayml.com\/ai-magic-tools\/"},{"key":"e_1_3_3_2_5_1","unstructured":"2023. ChatGPT. Retrieved August 15 2023 from https:\/\/chat.openai.com\/"},{"key":"e_1_3_3_2_6_1","volume-title":"Retrieved","author":"Card System","year":"2023","unstructured":"2023. GPT-4V(ision) System Card. Retrieved December 10, 2023 from https:\/\/cdn.openai.com\/papers\/GPTV_System_Card.pdf"},{"key":"e_1_3_3_2_7_1","unstructured":"2023. Midjourney. Retrieved August 15 2023 from https:\/\/www.midjourney.com\/"},{"key":"e_1_3_3_2_8_1","volume-title":"Retrieved","year":"2023","unstructured":"2023. Reflections on Foundation Models. Retrieved August 15, 2023 from https:\/\/hai.stanford.edu\/news\/reflections-foundation-models"},{"key":"e_1_3_3_2_9_1","volume-title":"Rhino - Grasshopper - New in Rhino 6. Retrieved","year":"2023","unstructured":"2023. Rhino - Grasshopper - New in Rhino 6. Retrieved August 15, 2023 from https:\/\/www.rhino3d.com\/6\/new\/grasshopper\/"},{"key":"e_1_3_3_2_10_1","volume-title":"Retrieved","year":"2023","unstructured":"2023. Transfer learning and fine-tuning. Retrieved August 15, 2023 from https:\/\/www.tensorflow.org\/tutorials\/images\/transfer_learning"},{"key":"e_1_3_3_2_11_1","volume-title":"Retrieved","year":"2023","unstructured":"2023. Unity Visual Scripting. Retrieved August 15, 2023 from https:\/\/unity.com\/features\/unity-visual-scripting"},{"key":"e_1_3_3_2_12_1","volume-title":"Retrieved","year":"2023","unstructured":"2023. What is Max? | Cycling \u201974. Retrieved August 15, 2023 from https:\/\/cycling74.com\/products\/max"},{"key":"e_1_3_3_2_13_1","volume-title":"On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258","author":"Bommasani Rishi","year":"2021","unstructured":"Rishi Bommasani, Drew\u00a0A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael\u00a0S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)."},{"key":"e_1_3_3_2_14_1","volume-title":"Language models are few-shot learners. Advances in neural information processing systems 33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared\u00a0D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877\u20131901."},{"key":"e_1_3_3_2_15_1","doi-asserted-by":"crossref","unstructured":"Michelle Carney Barron Webster Irene Alvarado Kyle Phillips Noura Howell Jordan Griffith Jonas Jongejan Amit Pitaru and Alexander Chen. 2020. Teachable machine: Approachable Web-based tool for exploring machine learning classification. In Extended abstracts of the 2020 CHI conference on human factors in computing systems. 1\u20138.","DOI":"10.1145\/3334480.3382839"},{"key":"e_1_3_3_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3563657.3596001"},{"key":"e_1_3_3_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/WVL.1989.77057"},{"key":"e_1_3_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581338"},{"key":"e_1_3_3_2_19_1","volume-title":"Blockly: A visual programming editor","author":"Neil Fraser","year":"2013","unstructured":"Neil Fraser 2013. Blockly: A visual programming editor. URL: https:\/\/code. google. com\/p\/blockly 42 (2013)."},{"key":"e_1_3_3_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580999"},{"key":"e_1_3_3_2_21_1","volume-title":"Audiogpt: Understanding and generating speech, music, sound, and talking head. arXiv preprint arXiv:2304.12995","author":"Huang Rongjie","year":"2023","unstructured":"Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jiawei Huang, Jinglin Liu, 2023. Audiogpt: Understanding and generating speech, music, sound, and talking head. arXiv preprint arXiv:2304.12995 (2023)."},{"key":"e_1_3_3_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581403"},{"key":"e_1_3_3_2_23_1","volume-title":"The blank page: Effects of constraint on creativity","author":"Joyce K","unstructured":"Caneel\u00a0K Joyce. 2009. The blank page: Effects of constraint on creativity. University of California, Berkeley."},{"key":"e_1_3_3_2_24_1","volume-title":"Shap-e: Generating conditional 3d implicit functions. arXiv preprint arXiv:2305.02463","author":"Jun Heewoo","year":"2023","unstructured":"Heewoo Jun and Alex Nichol. 2023. Shap-e: Generating conditional 3d implicit functions. arXiv preprint arXiv:2305.02463 (2023)."},{"key":"e_1_3_3_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.366147"},{"key":"e_1_3_3_2_26_1","volume-title":"Segment anything. arXiv preprint arXiv:2304.02643","author":"Kirillov Alexander","year":"2023","unstructured":"Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander\u00a0C Berg, Wan-Yen Lo, 2023. Segment anything. arXiv preprint arXiv:2304.02643 (2023)."},{"key":"e_1_3_3_2_27_1","volume-title":"LabVIEW. Proceedings of the ACM on Programming Languages 4, HOPL","author":"Kodosky Jeffrey","year":"2020","unstructured":"Jeffrey Kodosky. 2020. LabVIEW. Proceedings of the ACM on Programming Languages 4, HOPL (2020), 1\u201354."},{"key":"e_1_3_3_2_28_1","volume-title":"Audiogen: Textually guided audio generation. arXiv preprint arXiv:2209.15352","author":"Kreuk Felix","year":"2022","unstructured":"Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre D\u00e9fossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi. 2022. Audiogen: Textually guided audio generation. arXiv preprint arXiv:2209.15352 (2022)."},{"key":"e_1_3_3_2_29_1","volume-title":"Visual instruction tuning. arXiv preprint arXiv:2304.08485","author":"Liu Haotian","year":"2023","unstructured":"Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong\u00a0Jae Lee. 2023. Visual instruction tuning. arXiv preprint arXiv:2304.08485 (2023)."},{"key":"e_1_3_3_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501825"},{"key":"e_1_3_3_2_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1045-926X(05)80036-9"},{"key":"e_1_3_3_2_32_1","volume-title":"Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350","author":"Press Ofir","year":"2022","unstructured":"Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah\u00a0A Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350 (2022)."},{"key":"e_1_3_3_2_33_1","volume-title":"International conference on machine learning. PMLR, 8748\u20138763","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong\u00a0Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748\u20138763."},{"key":"e_1_3_3_2_34_1","volume-title":"International Conference on Machine Learning. PMLR, 8821\u20138831","author":"Ramesh Aditya","year":"2021","unstructured":"Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image generation. In International Conference on Machine Learning. PMLR, 8821\u20138831."},{"key":"e_1_3_3_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1592761.1592779"},{"key":"e_1_3_3_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"e_1_3_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445538"},{"key":"e_1_3_3_2_38_1","volume-title":"Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint arXiv:2303.17580","author":"Shen Yongliang","year":"2023","unstructured":"Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint arXiv:2303.17580 (2023)."},{"key":"e_1_3_3_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610217"},{"key":"e_1_3_3_2_40_1","volume-title":"Designing Interactive Systems Conference","author":"Subramonyam Hariharan","year":"2021","unstructured":"Hariharan Subramonyam, Colleen Seifert, and Eytan Adar. 2021. Towards a process model for co-creating AI experiences. In Designing Interactive Systems Conference 2021. 1529\u20131543."},{"key":"e_1_3_3_2_41_1","volume-title":"Attention is all you need. Advances in neural information processing systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan\u00a0N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_3_3_2_43_1","volume-title":"Emergent abilities of large language models. arXiv preprint arXiv:2206.07682","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, 2022. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 (2022)."},{"key":"e_1_3_3_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2771839.2771860"},{"key":"e_1_3_3_2_45_1","doi-asserted-by":"crossref","unstructured":"Kirsten\u00a0N Whitley and Alan\u00a0F Blackwell. 1997. Visual programming: the outlook from academia and industry. In Papers presented at the seventh workshop on Empirical studies of programmers. 180\u2013208.","DOI":"10.1145\/266399.266415"},{"key":"e_1_3_3_2_46_1","volume-title":"Graphical User Interface for Creating TinyML Models. In 2022 IEEE Symposium on Visual Languages and Human-Centric Computing (VL\/HCC). IEEE, 1\u20135.","author":"Williams Randi","year":"2022","unstructured":"Randi Williams, Micha\u0142 Moskal, and Peli De\u00a0Halleux. 2022. ML Blocks: A Block-Based, Graphical User Interface for Creating TinyML Models. In 2022 IEEE Symposium on Visual Languages and Human-Centric Computing (VL\/HCC). IEEE, 1\u20135."},{"key":"e_1_3_3_2_47_1","volume-title":"Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671","author":"Wu Chenfei","year":"2023","unstructured":"Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. 2023. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671 (2023)."},{"key":"e_1_3_3_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3519729"},{"key":"e_1_3_3_2_49_1","unstructured":"Qian Yang. 2018. Machine learning as a UX design material: how can we imagine beyond automation recommenders and reminders?. In AAAI Spring Symposia Vol.\u00a01. 2\u20136."},{"key":"e_1_3_3_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196709.3196730"},{"key":"e_1_3_3_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376301"},{"key":"e_1_3_3_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517491"},{"key":"e_1_3_3_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3563657.3596058"},{"key":"e_1_3_3_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581388"},{"key":"e_1_3_3_2_55_1","volume-title":"Socratic models: Composing zero-shot multimodal reasoning with language. arXiv preprint arXiv:2204.00598","author":"Zeng Andy","year":"2022","unstructured":"Andy Zeng, Maria Attarian, Brian Ichter, Krzysztof Choromanski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani, 2022. Socratic models: Composing zero-shot multimodal reasoning with language. arXiv preprint arXiv:2204.00598 (2022)."},{"key":"e_1_3_3_2_56_1","doi-asserted-by":"crossref","unstructured":"Lvmin Zhang Anyi Rao and Maneesh Agrawala. 2023. Adding Conditional Control to Text-to-Image Diffusion Models.","DOI":"10.1109\/ICCV51070.2023.00355"},{"key":"e_1_3_3_2_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-20077-9_21"},{"key":"e_1_3_3_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240704"}],"event":{"name":"CHI '24: CHI Conference on Human Factors in Computing Systems","location":"Honolulu HI USA","acronym":"CHI '24","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction","SIGACCESS ACM Special Interest Group on Accessible Computing"]},"container-title":["Proceedings of the CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3613904.3641920","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3613904.3641920","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:57:28Z","timestamp":1750291048000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3613904.3641920"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,11]]},"references-count":57,"alternative-id":["10.1145\/3613904.3641920","10.1145\/3613904"],"URL":"https:\/\/doi.org\/10.1145\/3613904.3641920","relation":{},"subject":[],"published":{"date-parts":[[2024,5,11]]},"assertion":[{"value":"2024-05-11","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}