{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:21:59Z","timestamp":1750220519091,"version":"3.41.0"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2021,10,22]],"date-time":"2021-10-22T00:00:00Z","timestamp":1634860800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100011039","name":"Intelligence Advanced Research Projects Activity","doi-asserted-by":"crossref","award":["2016-16031400006"],"award-info":[{"award-number":["2016-16031400006"]}],"id":[{"id":"10.13039\/100011039","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Digital Threats"],"published-print":{"date-parts":[[2022,3,31]]},"abstract":"<jats:p>Organizations that use data to assess insider threats, or other workforce outcomes, need to evaluate the quality of their assessment methods. This evaluation relies on an accurate representation of the predictors and criterion variables within the organization's workforce. However, privacy concerns often limit the information that is available for evaluation. For example, the organization might anonymize identifying information of its workforce, or the evaluation might be restricted to use group statistics, such as marginal distributions of predictors and criteria, along with their intercorrelations. In this paper we demonstrate a hybrid approach for simulating workforce data to support inference-enterprise evaluation, including the crowdsourced elicitation of marginal distributions and correlations of predictors and the simulation of a workforce population from the statistical properties of a redacted set of predictor distributions. The methods provide a way to simulate a population that has statistical characteristics of the workforce, in order to assess the performance of the assessment methods. The statistical methods are supplemented by expert judgments for situations where required information is not available. We evaluate these methods using anonymized data from an actual organization.<\/jats:p>","DOI":"10.1145\/3457910","type":"journal-article","created":{"date-parts":[[2021,4,16]],"date-time":"2021-04-16T00:47:40Z","timestamp":1618534060000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Simulating Organizational Data from Redacted Input for Inference Enterprise Modeling"],"prefix":"10.1145","volume":"3","author":[{"given":"Paul J.","family":"Sticha","sequence":"first","affiliation":[{"name":"Human Resources Research Organization (HumRRO), Alexandria, VA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tirso E.","family":"Diaz","sequence":"additional","affiliation":[{"name":"Human Resources Research Organization (HumRRO), Alexandria, VA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elise T.","family":"Axelrad","sequence":"additional","affiliation":[{"name":"Innovative Decisions International, an ITA International Company, Falls Church VA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sean D.","family":"Vermillion","sequence":"additional","affiliation":[{"name":"Innovative Decisions International, an ITA International Company, Austin TX"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dennis M.","family":"Buede","sequence":"additional","affiliation":[{"name":"Innovative Decisions International, an ITA International Company, Vienna VA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,10,22]]},"reference":[{"key":"e_1_3_1_2_1","volume-title":"An Introduction to Multivariate Statistical Analysis","author":"Anderson Theodore W.","year":"1984","unstructured":"Theodore W. Anderson. 1984. An Introduction to Multivariate Statistical Analysis (2nd ed.). Wiley, New York, NY.","edition":"2"},{"key":"e_1_3_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/SPW.2013.35"},{"key":"e_1_3_1_4_1","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.procs.2019.05.060","article-title":"Improving insider threat detection through multi-modelling\/data fusion","volume":"153","author":"Brown David P.","year":"2019","unstructured":"David P. Brown, Denis M. Buede, and Sean D. Vermillion. 2019. Improving insider threat detection through multi-modelling\/data fusion. Procedia Computer Science 153 (2019), 100\u2013107. https:\/\/doi.org\/10.1016\/j.procs.2019.05.060","journal-title":"Procedia Computer Science"},{"key":"e_1_3_1_5_1","doi-asserted-by":"crossref","first-page":"e1277","DOI":"10.1002\/widm.1277","article-title":"Inference enterprise models: An approach to organizational performance improvement","author":"Buede Dennis M.","year":"2018","unstructured":"Dennis M. Buede, Elise T. Axelrad, David P. Brown, Daniel W. Hudson, Kathryn B. Laskey, Paul J. Sticha, and Jordan L. Thomas. 2018. Inference enterprise models: An approach to organizational performance improvement. WIREs Data Mining Knowl Discov. 2018, e1277. https:\/\/doi.org\/10.1002\/widm.1277","journal-title":"WIREs Data Mining Knowl Discov"},{"key":"e_1_3_1_6_1","volume-title":"Proceedings of the 24th International System Dynamics Conference","author":"Cappelli Dawn M.","year":"2006","unstructured":"Dawn M. Cappelli, Akash G. Desai, Andrew P. Moore, Timothy J. Shimeall, Elise A. Weaver, and Bradford J. Willke. 2006. Management and education of the risk of insider threat (MERIT): Mitigating the risk of sabotage to employers\u2019 information, systems, or networks. In Proceedings of the 24th International System Dynamics Conference. Nijmegen, Netherlands. https:\/\/proceedings.systemdynamics.org\/2006\/proceed\/papers\/MOORE333.pdf."},{"key":"e_1_3_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1966913.1966959"},{"issue":"5","key":"e_1_3_1_8_1","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1017\/S1930297500004599","article-title":"Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments","volume":"11","author":"Chang Welton","year":"2016","unstructured":"Welton Chang, Eva Chen, Barbara Mellers, and Philip Tetlock. 2016. Developing expert political judgment: The impact of training and practice on judgmental accuracy in geopolitical forecasting tournaments. Judgment and Decision Making 11, 5 (September 2016) 509\u2013526.","journal-title":"Judgment and Decision Making"},{"issue":"1","key":"e_1_3_1_9_1","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/S0022-5371(67)80066-5","article-title":"Illusory correlation in observational report","volume":"6","author":"Chapman Loren J.","year":"1967","unstructured":"Loren J. Chapman. 1967. Illusory correlation in observational report. Journal of Verbal Learning and Verbal Behavior 6, 1 (February 1967), 151\u2013155. https:\/\/doi.org\/10.1016\/S0022-5371(67)80066-5","journal-title":"Journal of Verbal Learning and Verbal Behavior"},{"key":"e_1_3_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02293811"},{"key":"e_1_3_1_11_1","volume-title":"Calculated Risks: How to Know when Numbers Deceive You","author":"Gigerenzer Gerd","year":"2002","unstructured":"Gerd Gigerenzer. 2002. Calculated Risks: How to Know when Numbers Deceive You. Simon & Schuster, New York, NY."},{"key":"e_1_3_1_12_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.327.7417.741"},{"key":"e_1_3_1_13_1","doi-asserted-by":"crossref","first-page":"132","DOI":"10.4018\/978-1-61692-245-0.ch007","volume-title":"Information Assurance and Security Ethics in Complex Systems: Interdisciplinary Perspectives. Chapter 7","author":"Greitzer Frank L.","year":"2011","unstructured":"Frank L. Greitzer, Deborah A. Frincke, and Mariah. Zabriskie. 2011. Social\/ethical issues in predictive insider threat monitoring. In: MJ Dark (Ed.), Information Assurance and Security Ethics in Complex Systems: Interdisciplinary Perspectives. Chapter 7, 132\u2013161. IGI Global, Hershey, Pennsylvania. https:\/\/doi.org\/10.4018\/978-1-61692-245-0.ch007"},{"key":"e_1_3_1_14_1","doi-asserted-by":"publisher","DOI":"10.1177\/014920630002600305"},{"issue":"2","key":"e_1_3_1_15_1","doi-asserted-by":"crossref","first-page":"757","DOI":"10.2307\/2532163","article-title":"Two guidelines for bootstrap hypothesis testing","volume":"47","author":"Hall Peter","year":"1991","unstructured":"Peter Hall and Susan R. Wilson. 1991. Two guidelines for bootstrap hypothesis testing. Biometrics 47, 2 (June 1991), 757\u2013762. https:\/\/doi.org\/10.2307\/2532163","journal-title":"Biometrics"},{"key":"e_1_3_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-9473(02)00072-5"},{"key":"e_1_3_1_17_1","doi-asserted-by":"publisher","DOI":"10.1093\/imanum\/22.3.329"},{"issue":"2","key":"e_1_3_1_18_1","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1037\/0033-295X.106.2.417","article-title":"How to improve Bayesian reasoning: Comment on Gigerenzer and Hoffrage (1995)","volume":"106","author":"Mellers Barbara A.","year":"1999","unstructured":"Barbara A. Mellers and A. Peter McGraw. 1999. How to improve Bayesian reasoning: Comment on Gigerenzer and Hoffrage (1995). Psychological Review, 106, 2 (April 1999), 417\u2013424. https:\/\/doi.org\/10.1037\/0033-295X.106.2.417","journal-title":"Psychological Review"},{"issue":"1","key":"e_1_3_1_19_1","first-page":"28","article-title":"A preliminary model of insider theft of intellectual property","volume":"2","author":"Moore Andrew P.","year":"2011","unstructured":"Andrew P. Moore, Dawn M. Cappelli, Thomas C. Caron, Eric Shaw, Derrick Spooner, and Randall F. Trzeciak. 2011. A preliminary model of insider theft of intellectual property. Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications 2, 1 (March 2011), 28\u201349. https:\/\/doi.org\/10.22667\/JOWUA.2011.03.31.028","journal-title":"Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications"},{"issue":"3","key":"e_1_3_1_20_1","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1111\/j.1744-6570.2006.00048.x","article-title":"Relationship of personality traits and counterproductive work behaviors: The mediating effects of job satisfaction","volume":"59","author":"Mount Michael","year":"2006","unstructured":"Michael Mount, Remus Ilies, and Erin Johnson. 2006. Relationship of personality traits and counterproductive work behaviors: The mediating effects of job satisfaction. Personnel Psychology 59, 3 (Autumn 2006), 591\u2013622. https:\/\/doi.org\/10.1111\/j.1744-6570.2006.00048.x","journal-title":"Personnel Psychology"},{"key":"e_1_3_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/1952073"},{"issue":"4","key":"e_1_3_1_22_1","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1177\/0013164414548894","article-title":"A cautionary note on the use of the Vale and Maurelli method to generate multivariate, nonnormal data for simulation purposes","volume":"75","author":"Olvera Astivia Oscar L.","year":"2015","unstructured":"Oscar L. Olvera Astivia and Bruno D. Zumbo. 2015. A cautionary note on the use of the Vale and Maurelli method to generate multivariate, nonnormal data for simulation purposes. Educ Psychol Meas. 75, 4 (August 2015), 541\u2013567. https:\/\/doi.org\/10.1177\/0013164414548894","journal-title":"Educ Psychol Meas"},{"key":"e_1_3_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_3_1_24_1","doi-asserted-by":"publisher","DOI":"10.2307\/256693"},{"issue":"3","key":"e_1_3_1_25_1","first-page":"335","article-title":"Simulating multivariate nonnormal data using an iterative algorithm","volume":"43","author":"Ruscio John","year":"2008","unstructured":"John Ruscio and Walter Kaczetow. 2008. Simulating multivariate nonnormal data using an iterative algorithm. Multivariate Behavioral Research 43, 3 (July-Sep 2008), 335\u2013381. https:\/\/doi.org\/10.1080\/00273170802285693","journal-title":"Multivariate Behavioral Research"},{"key":"e_1_3_1_26_1","first-page":"229","article-title":"Fonctions de repartition an dimensions et leurs marges","volume":"8","author":"Sklar Abe","year":"1959","unstructured":"Abe Sklar. 1959. Fonctions de repartition an dimensions et leurs marges. Publ. inst. statist. univ. Paris 8, 229\u2013231.","journal-title":"Publ. inst. statist. univ. Paris"},{"key":"e_1_3_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/2896336"},{"key":"e_1_3_1_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0285(73)90033-9"},{"issue":"3","key":"e_1_3_1_29_1","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1007\/BF02293687","article-title":"Simulating multivariate nonnormal distributions","volume":"48","author":"David Vale C.","year":"1983","unstructured":"C. David Vale and Vincent A. Maurelli. 1983. Simulating multivariate nonnormal distributions. Psychometrika 48, 3 (September 1983), 465\u2013471. https:\/\/doi.org\/10.1007\/BF02293687","journal-title":"Psychometrika"},{"key":"e_1_3_1_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/3320516.3320587"}],"container-title":["Digital Threats: Research and Practice"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3457910","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3457910","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:24:38Z","timestamp":1750195478000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3457910"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,22]]},"references-count":29,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,3,31]]}},"alternative-id":["10.1145\/3457910"],"URL":"https:\/\/doi.org\/10.1145\/3457910","relation":{},"ISSN":["2692-1626","2576-5337"],"issn-type":[{"type":"print","value":"2692-1626"},{"type":"electronic","value":"2576-5337"}],"subject":[],"published":{"date-parts":[[2021,10,22]]},"assertion":[{"value":"2020-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-10-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}