{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T22:12:05Z","timestamp":1782771125893,"version":"3.54.5"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,10,31]],"date-time":"2020-10-31T00:00:00Z","timestamp":1604102400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Model. Comput. Simul."],"published-print":{"date-parts":[[2020,10,31]]},"abstract":"<jats:p>Data farming is a descriptive metaphor that captures the notion of generating data purposefully to maximize the information \u201cyield\u201d from simulation models. Large-scale designed experiments let us grow the simulation output efficiently and effectively. We can explore massive input spaces, uncover interesting features of complex simulation response surfaces, and explicitly identify cause-and-effect relationships. Data farming has been used in the defense community over the past two decades, and has resulted in quantum leaps in the breadth, depth, and timeliness of the insights yielded by simulation models. In this article, we provide an overview of current data farming capabilities and their relationship to emerging techniques in data science and analytics. We use graphics to motivate insight into some of the benefits of a data farming approach. Finally, we share some thoughts about opportunities and challenges for further improving the state of the art, and transforming the state of the practice, in the data farming domain.<\/jats:p>","DOI":"10.1145\/3425398","type":"journal-article","created":{"date-parts":[[2020,11,26]],"date-time":"2020-11-26T19:45:23Z","timestamp":1606419923000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":23,"title":["Data Farming"],"prefix":"10.1145","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1976-3409","authenticated-orcid":false,"given":"Susan M.","family":"Sanchez","sequence":"first","affiliation":[{"name":"Naval Postgraduate School, Monterey, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,11,25]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/WSC.2015.7408294"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/72.298224"},{"key":"e_1_2_1_3_1","article-title":"Random search for hyper-parameter optimization","author":"Bergstra James","year":"2012","journal-title":"Journal of Machine Learning Research 13"},{"key":"e_1_2_1_4_1","first-page":"3","article-title":"Data science: A comprehensive overview","volume":"50","author":"Cao Longbing","year":"2017","journal-title":"ACM Computing Surveys"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1287\/ijoc.1120.0498"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2567895"},{"key":"e_1_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Niclas Feldkamp Soeren Bergmann and Steffen Strassburger. 2020. Knowledge discovery in simulation data. ACM Transactions on Modeling and Computer Simulation. Forthcoming. Niclas Feldkamp Soeren Bergmann and Steffen Strassburger. 2020. Knowledge discovery in simulation data. ACM Transactions on Modeling and Computer Simulation. Forthcoming.","DOI":"10.1145\/3391299"},{"key":"e_1_2_1_8_1","volume-title":"Automated Machine Learning: Methods, Systems, Challenges","author":"Feurer Matthias"},{"key":"e_1_2_1_9_1","first-page":"44","article-title":"R Package \u201cNeuralnet","volume":"1","author":"Fritsch Stefan","year":"2019","journal-title":"Training of Neural Networks. Version"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1287\/ijoc.14.3.192.113"},{"key":"e_1_2_1_11_1","volume-title":"Investigating the Naval Logistics Role in Humanitarian Assistance Activities. Master\u2019s Thesis. Naval Postgraduate School","author":"Gardner Maxine J."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1080\/01605682.2019.1678406"},{"key":"e_1_2_1_13_1","article-title":"Explained: Neural networks","author":"Hardesty Larry","year":"2017","journal-title":"MIT News"},{"key":"e_1_2_1_14_1","volume-title":"Teng Suyan, and Li Juxin.","author":"Hay Lee Loo","year":"2009"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2379810.2379813"},{"key":"e_1_2_1_16_1","volume-title":"Retrieved","year":"2020"},{"key":"e_1_2_1_17_1","volume-title":"Nelson","author":"Hunter Susan R.","year":"2017"},{"key":"e_1_2_1_18_1","volume-title":"Design and Analysis of Simulation Experiments","author":"Kleijnen Jack P. C.","edition":"2"},{"key":"e_1_2_1_19_1","volume-title":"Advances in Modeling and Simulation","author":"Kleijnen Jack P. C."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2016.06.041"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1287\/ijoc.1050.0136"},{"key":"e_1_2_1_22_1","volume-title":"Data Mining and Knowledge Discovery Approaches Based on Rule Induction Techniques","author":"Kusiak Andrew"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/72.977291"},{"key":"e_1_2_1_24_1","volume-title":"Simulation Modeling and Analysis","author":"Law Averill M.","edition":"5"},{"key":"e_1_2_1_25_1","volume-title":"Improving the Taiwan Military\u2019s Disaster Relief Response to Typhoons. Master\u2019s Thesis. Naval Postgraduate School","author":"Hung-Xin Li."},{"key":"e_1_2_1_26_1","volume-title":"Proceedings of the 2016 Winter Simulation Conference. IEEE","author":"Lin Yujing"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1287\/ijoc.2018.0839"},{"key":"e_1_2_1_28_1","first-page":"5","article-title":"Testing policies and key influences on long-term aircraft fleet management using designed simulation experiments","volume":"24","author":"Marlow David O.","year":"2019","journal-title":"Military Operations Research"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/3320516.3320563"},{"key":"e_1_2_1_30_1","unstructured":"Yukihiro Matsumoto. n.d. Ruby Programming Language. Available at https:\/\/www.ruby-lang.org\/. Yukihiro Matsumoto. n.d. Ruby Programming Language. Available at https:\/\/www.ruby-lang.org\/."},{"key":"e_1_2_1_31_1","volume-title":"Review of input variable delection methods for artificial neural networks","author":"May Robert"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.adro.2016.10.001"},{"key":"e_1_2_1_33_1","volume-title":"Design and Analysis of Experiments","author":"Montgomery D. C.","edition":"9"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1287\/inte.2017.0900"},{"key":"e_1_2_1_35_1","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1080\/00031305.1992.10475869","article-title":"Response surface alternatives to the Taguchi robust parameter design approach","volume":"46","author":"Myers Raymond H.","year":"1992","journal-title":"American Statistician"},{"key":"e_1_2_1_37_1","volume-title":"Foundations and Methods of Stochastic Simulation: A First Course","author":"Nelson Barry L."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.5555\/3320516.3320569"},{"key":"e_1_2_1_39_1","volume-title":"e-Handbook of Statistical Methods. Accessed","year":"2010"},{"key":"e_1_2_1_40_1","volume-title":"Analysis of the Performance of an Optimization Model for Time-Shiftable Electrical Load Scheduling Under Uncertainty. Master\u2019s Thesis. Naval Postgraduate School","author":"Olabode John A."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1214\/09-SS057"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511803161"},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the 2018 Winter Simulation Conference. IEEE","author":"Peng Yijie","year":"1909"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1937-5956.2009.01119.x"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/WSC.2007.4419588"},{"key":"e_1_2_1_46_1","volume-title":"Retrieved","author":"Sanchez Paul J.","year":"2018"},{"key":"e_1_2_1_47_1","volume-title":"Retrieved","author":"Sanchez Paul J.","year":"2018"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1002\/qre.2424"},{"key":"e_1_2_1_49_1","volume-title":"Sanchez","author":"Sanchez Paul J.","year":"2020"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/WSC.2018.8632383"},{"key":"e_1_2_1_51_1","volume-title":"Final Report: Testing-Based Interventions for COVID Pandemic Policies. Technical Report. Naval Postgraduate School","author":"Sanchez Susan M.","year":"2020"},{"key":"e_1_2_1_52_1","volume-title":"Proceedings of the 2015 Winter Simulation Conference. IEEE","author":"Susan"},{"key":"e_1_2_1_53_1","volume-title":"Sanchez","author":"Sanchez Susan M.","year":"2017"},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the 2020 Winter Simulation Conference. IEEE","author":"Susan"},{"key":"e_1_2_1_55_1","volume-title":"Proceedings of the 2018 Winter Simulation Conference. IEEE","author":"Sanchez Susan M."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/1502787.1502790"},{"key":"e_1_2_1_57_1","volume-title":"Notz","author":"Santner Thomas J.","year":"2003"},{"key":"e_1_2_1_58_1","volume-title":"Retrieved","author":"SAS Institute Inc.","year":"2020"},{"key":"e_1_2_1_59_1","volume-title":"Retrieved","author":"School Naval Postgraduate","year":"2020"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.2307\/1269414"},{"key":"e_1_2_1_61_1","volume-title":"WIRED. Retrieved","author":"Simonite Tom","year":"2018"},{"key":"e_1_2_1_62_1","volume-title":"Retrieved","author":"OptTek Systems Inc.","year":"2020"},{"key":"e_1_2_1_63_1","volume-title":"System of Experimental Design. Vols. 1 and 2. UNIPUB\/Krauss International","author":"Taguchi Genechi"},{"key":"e_1_2_1_64_1","volume-title":"Retrieved","author":"van Rossum Guido","year":"2020"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/WSC40007.2019.9004904"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1057\/jos.2013.14"},{"key":"e_1_2_1_67_1","volume-title":"Retrieved","author":"Vigen Tyler","year":"2014"},{"key":"e_1_2_1_68_1","volume-title":"Hachette Book Group","author":"Vigen Tyler"},{"key":"e_1_2_1_69_1","volume-title":"Proceedings of the 2018 Winter Simulation Conference. IEEE","author":"Warnke Tom"},{"key":"e_1_2_1_70_1","first-page":"84","article-title":"R Package \u201cCorrplot","volume":"0","author":"Wei Taiyun","year":"2017","journal-title":"Visualization of a Correlation Matrix. Version"},{"key":"e_1_2_1_71_1","volume-title":"Pal","author":"Witten Ian H.","year":"2017"},{"key":"e_1_2_1_72_1","volume-title":"Proceedings of the 37th International Conference on Machine Learning. 14","author":"Witty Sam","year":"2020"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174053"}],"container-title":["ACM Transactions on Modeling and Computer Simulation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3425398","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3425398","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:55Z","timestamp":1750195915000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3425398"}},"subtitle":["Methods for the Present, Opportunities for the Future"],"short-title":[],"issued":{"date-parts":[[2020,10,31]]},"references-count":72,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,10,31]]}},"alternative-id":["10.1145\/3425398"],"URL":"https:\/\/doi.org\/10.1145\/3425398","relation":{},"ISSN":["1049-3301","1558-1195"],"issn-type":[{"value":"1049-3301","type":"print"},{"value":"1558-1195","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,31]]},"assertion":[{"value":"2020-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-11-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}