{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T08:03:12Z","timestamp":1772524992527,"version":"3.50.1"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2024,7,12]]},"abstract":"<jats:p>Symbolic execution is an automated test input generation technique that models individual program paths as logical constraints. However, the realism of concrete test inputs generated by SMT solvers often comes into question. Existing symbolic execution tools only seek arbitrary solutions for given path constraints. These constraints do not incorporate the naturalness of inputs that observe statistical distributions, range constraints, or preferred string constants. This results in unnatural-looking inputs that fail to emulate real-world data.<\/jats:p>\n                  <jats:p>\n                    In this paper, we extend symbolic execution with consideration for incorporating naturalness. Our key insight is that users typically understand the semantics of program inputs, such as the distribution of height or possible values of\n                    <jats:monospace>zipcode<\/jats:monospace>\n                    , which can be leveraged to advance the ability of symbolic execution to produce natural test inputs. We instantiate this idea in N\n                    <jats:sc>atural<\/jats:sc>\n                    S\n                    <jats:sc>ym<\/jats:sc>\n                    , a symbolic execution-based test generation tool for data-intensive scalable computing (DISC) applications. NaturalSym generates natural-looking data that mimics real-world distributions by utilizing user-provided input semantics to drastically enhance the naturalness of inputs, while preserving strong bug-finding potential.\n                  <\/jats:p>\n                  <jats:p>\n                    On DISC applications and commercial big data test benchmarks, N\n                    <jats:sc>atural<\/jats:sc>\n                    S\n                    <jats:sc>ym<\/jats:sc>\n                    achieves a higher degree of realism \u2014as evidenced by a perplexity score 35.1 points lower on median, and detects 1.29\u00d7 injected faults compared to the state-of-the-art symbolic executor for DISC, B\n                    <jats:sc>ig<\/jats:sc>\n                    T\n                    <jats:sc>est<\/jats:sc>\n                    . This is because B\n                    <jats:sc>ig<\/jats:sc>\n                    T\n                    <jats:sc>est<\/jats:sc>\n                    draws inputs purely based on the satisfiability of path constraints constructed from branch predicates, while N\n                    <jats:sc>atural<\/jats:sc>\n                    S\n                    <jats:sc>ym<\/jats:sc>\n                    is able to draw natural concrete values based on user-specified semantics and prioritize using these values in input generation. Our empirical results demonstrate that NaturalSym finds injected faults 47.8\u00d7 more than N\n                    <jats:sc>atural<\/jats:sc>\n                    F\n                    <jats:sc>uzz<\/jats:sc>\n                    (a coverage-guided fuzzer) and 19.1\u00d7 more than ChatGPT. Meanwhile, TestMiner (a mining-based approach) fails to detect any injected faults. N\n                    <jats:sc>atural<\/jats:sc>\n                    S\n                    <jats:sc>ym<\/jats:sc>\n                    is the first symbolic executor that combines the notion of input naturalness in symbolic path constraints during SMT-based input generation. We make our code available at\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/UCLA-SEAL\/NaturalSym\">https:\/\/github.com\/UCLA-SEAL\/NaturalSym<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1145\/3660825","type":"journal-article","created":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T10:22:09Z","timestamp":1720779729000},"page":"2677-2700","source":"Crossref","is-referenced-by-count":6,"title":["Natural Symbolic Execution-Based Testing for Big Data Analytics"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-0623-4110","authenticated-orcid":false,"given":"Yaoxuan","family":"Wu","sequence":"first","affiliation":[{"name":"University of California at Los Angeles, Los Angeles, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5707-4487","authenticated-orcid":false,"given":"Ahmad","family":"Humayun","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8007-8662","authenticated-orcid":false,"given":"Muhammad Ali","family":"Gulzar","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3802-1512","authenticated-orcid":false,"given":"Miryung","family":"Kim","sequence":"additional","affiliation":[{"name":"University of California at Los Angeles, Los Angeles, USA"}]}],"member":"320","published-online":{"date-parts":[[2024,7,12]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"Accessed: 2023. Airport Codes Dataset. https:\/\/datahub.io\/core\/airport-codes#resource-airport-codes"},{"key":"e_1_3_1_3_2","unstructured":"Accessed: 2023. Carvana Car Sales Dataset. https:\/\/aws.amazon.com\/marketplace\/pp\/prodview-y77x3t6zisn4w"},{"key":"e_1_3_1_4_2","unstructured":"Accessed: 2023. ChatGPT. https:\/\/openai.com\/blog\/chatgpt"},{"key":"e_1_3_1_5_2","unstructured":"Accessed: 2023. DistilGPT2. https:\/\/huggingface.co\/distilgpt2"},{"key":"e_1_3_1_6_2","unstructured":"Accessed: 2023. Faker. https:\/\/faker.readthedocs.io\/en\/master\/"},{"key":"e_1_3_1_7_2","unstructured":"Accessed: 2023. IMDB Dataset. https:\/\/www.imdb.com\/search\/title\/?title_type=feature tv_movie&ref_=adv_prv"},{"key":"e_1_3_1_8_2","unstructured":"Accessed: 2024. Maven Central. https:\/\/central.sonatype.com"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.08774"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2302.10352"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2018.00012"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/1159789.1159792"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2015.7107466"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/2560217.2560219"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-99524-9_24"},{"issue":"14","key":"e_1_3_1_16_2","article-title":"The smt-lib standard: Version 2.0","volume":"13","author":"Barrett Clark","year":"2010","unstructured":"Clark Barrett, Aaron Stump, Cesare Tinelli, et al. 2010. The smt-lib standard: Version 2.0. In Proceedings of the 8th international workshop on satisfiability modulo theories (Edinburgh, UK), Vol. 13. 14.","journal-title":"Proceedings of the 8th international workshop on satisfiability modulo theories (Edinburgh, UK)"},{"key":"e_1_3_1_17_2","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877-1901. https:\/\/dl.acm.org\/doi\/abs\/10.5555\/3495724.3495883","journal-title":"Advances in neural information processing systems"},{"key":"e_1_3_1_18_2","first-page":"1097","article-title":"Flexible Database Generators (VLDB \u201905)","author":"Bruno Nicolas","year":"2005","unstructured":"Nicolas Bruno and Surajit Chaudhuri. 2005. Flexible Database Generators (VLDB \u201905). VLDB Endowment, 1097-1107. https:\/\/dl.acm.org\/doi\/10.5555\/1083592.1083719","journal-title":"VLDB Endowment"},{"key":"e_1_3_1_19_2","first-page":"209","article-title":"Klee: unassisted and automatic generation of high-coverage tests for complex systems programs","volume":"8","author":"Cadar Cristian","year":"2008","unstructured":"Cristian Cadar, Daniel Dunbar, Dawson R Engler, et al. 2008. Klee: unassisted and automatic generation of high-coverage tests for complex systems programs. In OSDI, Vol. 8. 209-224. https:\/\/dl.acm.org\/doi\/10.5555\/1855741.1855756","journal-title":"In OSDI"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/1455518.1455522"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2012.31"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2110356.2110358"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/351240.351266"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025204"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2024.107468"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-99524-9_24"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2017.8115617"},{"key":"e_1_3_1_29_2","first-page":"151","article-title":"Automated whitebox fuzz testing","volume":"8","author":"Godefroid Patrice","year":"2008","unstructured":"Patrice Godefroid, Michael Y Levin, David A Molnar, et al. 2008. Automated whitebox fuzz testing. In NDSS, Vol. 8. 151-166.","journal-title":"In NDSS"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3624032.3624035"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338953"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3264586"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.24432\/C5NC77"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.5555\/1182635.1164254"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-30936-1_9"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","unstructured":"John Hughes Benjamin C Pierce Thomas Arts and Ulf Norell. 2016. Mysteries of dropbox: property-based testing of a distributed synchronization service. In 2016 IEEE International Conference on Software Testing Verification and Validation (ICST). IEEE 135-145. https:\/\/doi.org\/10.1109\/ICST.2016.37 10.1109\/ICST.2016.37","DOI":"10.1109\/ICST.2016.37"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00034"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.62"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-36577-X_40"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3360607"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1061\/(ASCE)SU.1943-5428.0000112"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00085"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2310.00483"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.21105\/joss.01891"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3183297"},{"key":"e_1_3_1_46_2","first-page":"1049","article-title":"The Making of TPC-DS","volume":"6","author":"Nambiar Raghunath Othayoth","year":"2006","unstructured":"Raghunath Othayoth Nambiar and Meikel Poess. 2006. The Making of TPC-DS. In VLDB, Vol. 6. 1049-1058.","journal-title":"In VLDB"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00205"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","unstructured":"Pengyu Nie Rahul Banerjee Junyi Jessy Li Raymond J Mooney and Milos Gligoric. 2023. Learning deep semantics for test completion. In 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE). IEEE 2111-2123. https:\/\/doi.org\/10.1109\/ICSE48619.2023.00178 10.1109\/ICSE48619.2023.00178","DOI":"10.1109\/ICSE48619.2023.00178"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/11814948_18"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2007.37"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3339002"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/1858996.1859035"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-013-0122-2"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSAA.2016.49"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2310.06320"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.5120\/ijca2016907355"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00193"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2308.08033"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2310.02368"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2302.13971"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2009.05617"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524481.3527220"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.04346"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/1007512.1007526"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3368208"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","unstructured":"Yaoxuan Wu Ahmad Humayun Muhammad Ali Gulzar and Miryung Kim. 2024. Reproduction Package of \u2018Natural Symbolic Execution-based Testing for Big Data Analytics\u2019. https:\/\/doi.org\/10.5281\/zenodo.11090237 10.5281\/zenodo.11090237","DOI":"10.5281\/zenodo.11090237"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.04764"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2013.6693071"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.04207"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.5555\/1863103.1863113"},{"key":"e_1_3_1_71_2","unstructured":"Michal Zalewski. 2015. afl-fuzz: making up grammar with a dictionary in hand. (2015). https:\/\/lcamtuf.blogspot.com\/2015\/01\/afl-fuzz-making-up-grammar-with.html"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416641"},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2310.00710"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3660825","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3660825","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3660825","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T07:57:04Z","timestamp":1770191824000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3660825"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,12]]},"references-count":72,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2024,7,12]]}},"alternative-id":["10.1145\/3660825"],"URL":"https:\/\/doi.org\/10.1145\/3660825","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,12]]}}}