{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T14:44:52Z","timestamp":1787237092893,"version":"build-2736575974"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,5,29]],"date-time":"2024-05-29T00:00:00Z","timestamp":1716940800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2024,5,29]]},"abstract":"<jats:p>Spreadsheets are widely recognized as the most popular end-user programming tools, which blend the power of formula-based computation, with an intuitive table-based interface. Today, spreadsheets are used by billions of users to manipulate tables, most of whom are neither database experts nor professional programmers.<\/jats:p>\n                  <jats:p>Despite the success of spreadsheets, authoring complex formulas remains challenging, as non-technical users need to look up and understand non-trivial formula syntax. To address this pain point, we leverage the observation that there is often an abundance of similar-looking spreadsheets in the same organization, which not only have similar data, but also share similar computation logic encoded as formulas. We develop an Auto-Formula system that can accurately predict formulas that users want to author in a target spreadsheet cell, by learning and adapting formulas that already exist in similar spreadsheets, using contrastive-learning techniques inspired by \"similar-face recognition\" from compute vision. Extensive evaluations on over 2K test formulas extracted from real enterprise spreadsheets show the effectiveness of Auto-Formula over alternatives. Our benchmark data is available at https:\/\/github.com\/microsoft\/Auto-Formula to facilitate future research.<\/jats:p>","DOI":"10.1145\/3654925","type":"journal-article","created":{"date-parts":[[2024,5,30]],"date-time":"2024-05-30T09:44:53Z","timestamp":1717062293000},"page":"1-27","source":"Crossref","is-referenced-by-count":5,"title":["Auto-Formula: Recommend Formulas in Spreadsheets using Contrastive Learning for Table Representations"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-5331-5829","authenticated-orcid":false,"given":"Sibei","family":"Chen","sequence":"first","affiliation":[{"name":"Renmin University of China, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2824-5299","authenticated-orcid":false,"given":"Yeye","family":"He","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0870-7628","authenticated-orcid":false,"given":"Weiwei","family":"Cui","sequence":"additional","affiliation":[{"name":"Microsoft Research, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4729-9903","authenticated-orcid":false,"given":"Ju","family":"Fan","sequence":"additional","affiliation":[{"name":"Renmin University of China, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2178-3086","authenticated-orcid":false,"given":"Song","family":"Ge","sequence":"additional","affiliation":[{"name":"Microsoft Research, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7530-9553","authenticated-orcid":false,"given":"Haidong","family":"Zhang","sequence":"additional","affiliation":[{"name":"Microsoft Research, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9230-2799","authenticated-orcid":false,"given":"Dongmei","family":"Zhang","sequence":"additional","affiliation":[{"name":"Microsoft Research, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8252-5270","authenticated-orcid":false,"given":"Surajit","family":"Chaudhuri","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,5,30]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"[n. d.]. Auto-Formula: benchmark spreadsheet data. https:\/\/github.com\/microsoft\/Auto-Formula https:\/\/1drv.ms\/f\/s! AkvY8ho1gepOiptfygjBTFLp_V3rtg?e=Ls1ses."},{"key":"e_1_2_1_2_1","unstructured":"[n. d.]. Auto-Formula: Recommend Formulas in Spreadsheets using Contrastive Learning for Table Representations (extended version). https:\/\/arxiv.org\/abs\/2404.12608."},{"key":"e_1_2_1_3_1","unstructured":"[n. d.]. Excel formula. https:\/\/support.microsoft.com\/en-au\/office\/overview-of-formulas-in-excel-ecfdc708--9162--49e8-b993-c311f47ca173."},{"key":"e_1_2_1_4_1","unstructured":"[n. d.]. Excel Forum: 20K questions tagged as \"formulas and functions\" (Retrieved 2023-09). https:\/\/techcommunity. microsoft.com\/t5\/forums\/filteredbylabelpage\/board-id\/ExcelGeneral\/label-name\/formulas%20and%20functions\/."},{"key":"e_1_2_1_5_1","unstructured":"[n. d.]. Formula suggestion experience:. https:\/\/1drv.ms\/i\/s!AkvY8ho1gepOipteE2g_8Mjj5TFQlg?e=f6C2x9."},{"key":"e_1_2_1_6_1","unstructured":"[n. d.]. Github Copilot. https:\/\/github.com\/features\/copilot."},{"key":"e_1_2_1_7_1","unstructured":"[n. d.]. Google blog: New intelligent suggestions for formulas and functions in Google Sheets (Retrieved 2023-09). https: \/\/workspaceupdates.googleblog.com\/2021\/08\/intelligent-formula-and-function-suggestions-in-google-sheets.html."},{"key":"e_1_2_1_8_1","unstructured":"[n. d.]. Google Sheets formula. https:\/\/support.google.com\/docs\/table\/25273."},{"key":"e_1_2_1_9_1","unstructured":"[n. d.]. IntelliSense: Auto-Complete in Visual Studio. https:\/\/code.visualstudio.com\/docs\/editor\/intellisense."},{"key":"e_1_2_1_10_1","unstructured":"[n. d.]. List of Excel functions. https:\/\/support.microsoft.com\/en-us\/office\/excel-functions-alphabetical-b3944572-255d-4efb-bb96-c6d90033e188."},{"key":"e_1_2_1_11_1","unstructured":"[n. d.]. List of Google Sheets functions. https:\/\/support.google.com\/docs\/table\/25273?hl=en."},{"key":"e_1_2_1_12_1","unstructured":"[n. d.]. Mondrian on GitHub (Retrieved 2023-09). https:\/\/github.com\/HPI-Information-Systems\/Mondrian."},{"key":"e_1_2_1_13_1","unstructured":"[n. d.]. Spreadsheet workbook. https:\/\/support.microsoft.com\/en-us\/office\/insert-or-delete-a-worksheet-19d3d21ea3b3-4e13-a422-d1f43f1faaf2."},{"key":"e_1_2_1_14_1","doi-asserted-by":"crossref","unstructured":"Robin Abraham Margaret M Burnett and Martin Erwig. 2008. Spreadsheet Programming.","DOI":"10.1002\/9780470050118.ecse415"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1921048"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3360594"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-1133"},{"key":"e_1_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Laure Berti-Equille. 2019. Learn2clean: Optimizing the sequence of tasks for web data preparation. In The world wide web conference. 2580--2586.","DOI":"10.1145\/3308558.3313602"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/27641.28058"},{"key":"e_1_2_1_20_1","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020","author":"Brown Tom B.","year":"2020","unstructured":"Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6--12, 2020, virtual, Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2010.33"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588945"},{"key":"e_1_2_1_23_1","volume-title":"International conference on machine learning. PMLR, 1597--1607","author":"Chen Ting","year":"2020","unstructured":"Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597--1607."},{"key":"e_1_2_1_24_1","volume-title":"International Conference on Machine Learning. PMLR, 1661--1672","author":"Chen Xinyun","year":"2021","unstructured":"Xinyun Chen, Petros Maniatis, Rishabh Singh, Charles Sutton, Hanjun Dai, Max Lin, and Denny Zhou. 2021. Spreadsheetcoder: Formula prediction from semi-structured context. In International Conference on Machine Learning. PMLR, 1661--1672."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW50498.2020.00359"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2009.140"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2011.5767843"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1925844.1926423"},{"key":"e_1_2_1_29_1","first-page":"3","article-title":"Programming by examples","volume":"45","author":"Gulwani Sumit","year":"2016","unstructured":"Sumit Gulwani. 2016. Programming by examples. Dependable Software Systems Engineering 45, 137 (2016), 3--15.","journal-title":"Dependable Software Systems Engineering"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.14778\/3231751.3231766"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3193539"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2016.86"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.129"},{"key":"e_1_2_1_34_1","volume-title":"Measuring spreadsheet formula understandability. arXiv preprint arXiv:1209.3517","author":"Hermans Felienne","year":"2012","unstructured":"Felienne Hermans, Martin Pinzger, and Arie van Deursen. 2012. Measuring spreadsheet formula understandability. arXiv preprint arXiv:1209.3517 (2012)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3064034"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.14778\/3407790.3407831"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2019.2921572"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14778\/1880172.1880175"},{"key":"e_1_2_1_39_1","first-page":"92","article-title":"Introducing the Enron corpus","volume":"45","author":"Klimt Bryan","year":"2004","unstructured":"Bryan Klimt and Yiming Yang. 2004. Introducing the Enron corpus.. In CEAS, Vol. 45. 92--96.","journal-title":"CEAS"},{"key":"e_1_2_1_40_1","unstructured":"Eugenie Yujing Lai Zainab Zolaktaf Mostafa Milani Omar AlOmeir Jianhao Cao and Rachel Pottinger. 2023. Workload-Aware Query Recommendation Using Deep Learning.. In EDBT. 53--65."},{"key":"e_1_2_1_41_1","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive nlp tasks","volume":"33","author":"Lewis Patrick","year":"2020","unstructured":"Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\u00fcttler, Mike Lewis, Wen-tau Yih, Tim Rockt\u00e4schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33 (2020), 9459--9474.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_42_1","volume-title":"Auto-Tables: Synthesizing multi-step transformations to relationalize tables without using examples. arXiv preprint arXiv:2307.14565","author":"Li Peng","year":"2023","unstructured":"Peng Li, Yeye He, Cong Yan, Yue Wang, and Surajit Chauduri. 2023. Auto-Tables: Synthesizing multi-step transformations to relationalize tables without using examples. arXiv preprint arXiv:2307.14565 (2023)."},{"key":"e_1_2_1_43_1","volume-title":"Dongmei Zhang, and Surajit Chaudhuri.","author":"Li Peng","year":"2023","unstructured":"Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, and Surajit Chaudhuri. 2023. Table-GPT: Table-tuned gpt for diverse table tasks. arXiv preprint arXiv:2310.09263 (2023)."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2909204"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3545995"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2021.3084827"},{"key":"e_1_2_1_47_1","volume-title":"An investigation of practical approximate nearest neighbor algorithms. Advances in neural information processing systems 17","author":"Liu Ting","year":"2004","unstructured":"Ting Liu, Andrew Moore, Ke Yang, and Alexander Gray. 2004. An investigation of practical approximate nearest neighbor algorithms. Advances in neural information processing systems 17 (2004)."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.713"},{"key":"e_1_2_1_49_1","volume-title":"Large Language Model for Table Processing: A Survey. arXiv preprint arXiv:2402.05121","author":"Lu Weizheng","year":"2024","unstructured":"Weizheng Lu, Jiaming Zhang, Jing Zhang, and Yueguo Chen. 2024. Large Language Model for Table Processing: A Survey. arXiv preprint arXiv:2402.05121 (2024)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.14778\/3192965.3192973"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5555\/287893.287899"},{"key":"e_1_2_1_52_1","volume-title":"Compositional semantic parsing on semi-structured tables. arXiv preprint arXiv:1508.00305","author":"Pasupat Panupong","year":"2015","unstructured":"Panupong Pasupat and Percy Liang. 2015. Compositional semantic parsing on semi-structured tables. arXiv preprint arXiv:1508.00305 (2015)."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.4018\/joeuc.2009070102"},{"key":"e_1_2_1_55_1","volume-title":"International conference on machine learning. PMLR, 8748--8763","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748--8763."},{"key":"e_1_2_1_56_1","volume-title":"Classification of spreadsheet errors. arXiv preprint arXiv:0805.4224","author":"Rajalingham Kamalasen","year":"2008","unstructured":"Kamalasen Rajalingham, David R Chadwick, and Brian Knight. 2008. Classification of spreadsheet errors. arXiv preprint arXiv:0805.4224 (2008)."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-019-0197-0"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3520152"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00552"},{"key":"e_1_2_1_62_1","volume-title":"Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533","author":"Wang Liang","year":"2022","unstructured":"Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533 (2022)."},{"key":"e_1_2_1_63_1","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824--24837.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389738"},{"key":"e_1_2_1_65_1","volume-title":"Auto-pipeline: synthesizing complex data pipelines by-target using reinforcement learning and search. arXiv preprint arXiv:2106.13861","author":"Yang Junwen","year":"2021","unstructured":"Junwen Yang, Yeye He, and Surajit Chaudhuri. 2021. Auto-pipeline: synthesizing complex data pipelines by-target using reinforcement learning and search. arXiv preprint arXiv:2106.13861 (2021)."},{"key":"e_1_2_1_66_1","doi-asserted-by":"crossref","unstructured":"Xiaokang Zhang Jing Zhang Zeyao Ma Yang Li Bohan Zhang Guanlin Li Zijun Yao Kangli Xu Jinchang Zhou Daniel Zhang-Li et al. 2024. TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios. arXiv preprint arXiv:2403.19318 (2024).","DOI":"10.18653\/v1\/2025.findings-acl.538"},{"key":"e_1_2_1_67_1","volume-title":"NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries. arXiv preprint arXiv:2402.14853","author":"Zhao Wei","year":"2024","unstructured":"Wei Zhao, Zhitao Hou, Siyuan Wu, Yan Gao, Haoyu Dong, Yao Wan, Hongyu Zhang, Yulei Sui, and Haidong Zhang. 2024. NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries. arXiv preprint arXiv:2402.14853 (2024)."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.14778\/3115404.3115409"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654925","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3654925","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T14:41:32Z","timestamp":1755787292000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654925"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,29]]},"references-count":68,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,5,29]]}},"alternative-id":["10.1145\/3654925"],"URL":"https:\/\/doi.org\/10.1145\/3654925","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,29]]}}}