{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T20:22:09Z","timestamp":1791231729619,"version":"4.1.0"},"reference-count":46,"publisher":"Wiley","issue":"10","license":[{"start":{"date-parts":[[2014,6,2]],"date-time":"2014-06-02T00:00:00Z","timestamp":1401667200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/100000199","name":"U.S. Department of Agriculture","doi-asserted-by":"publisher","award":["2010 51110 21076"],"award-info":[{"award-number":["2010 51110 21076"]}],"id":[{"id":"10.13039\/100000199","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000199","name":"U.S. Department of Agriculture","doi-asserted-by":"publisher","award":["2013-67011-21117"],"award-info":[{"award-number":["2013-67011-21117"]}],"id":[{"id":"10.13039\/100000199","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Risk Analysis"],"published-print":{"date-parts":[[2014,10]]},"abstract":"<jats:p>\n                    Increasing evidence suggests that persistence of\n                    <jats:italic>Listeria monocytogenes<\/jats:italic>\n                    in food processing plants has been the underlying cause of a number of human listeriosis outbreaks. This study extracts criteria used by food safety experts in determining bacterial persistence in the environment, using retail delicatessen operations as a model. Using the Delphi method, we conducted an expert elicitation with 10 food safety experts from academia, industry, and government to classify\n                    <jats:italic>L. monocytogenes<\/jats:italic>\n                    persistence based on environmental sampling results collected over six months for 30 retail delicatessen stores. The results were modeled using variations of random forest, support vector machine, logistic regression, and linear regression; variable importance values of random forest and support vector machine models were consolidated to rank important variables in the experts\u2019 classifications. The duration of subtype isolation ranked most important across all expert categories. Sampling site category also ranked high in importance and validation errors doubled when this covariate was removed. Support vector machine and random forest models successfully classified the data with average validation errors of 3.1% and 2.2% (\n                    <jats:italic>n<\/jats:italic>\n                    = 144), respectively. Our findings indicate that (i) the frequency of isolations over time and sampling site information are critical factors for experts determining subtype persistence, (ii) food safety experts from different sectors may not use the same criteria in determining persistence, and (iii) machine learning models have potential for future use in environmental surveillance and risk management programs. Future work is necessary to validate the accuracy of expert and machine classification against biological measurement of\n                    <jats:italic>L. monocytogenes<\/jats:italic>\n                    persistence.\n                  <\/jats:p>","DOI":"10.1111\/risa.12218","type":"journal-article","created":{"date-parts":[[2014,6,2]],"date-time":"2014-06-02T19:20:56Z","timestamp":1401736856000},"page":"1830-1845","source":"Crossref","is-referenced-by-count":17,"title":["Classification of\n                    <i>Listeria monocytogenes<\/i>\n                    Persistence in Retail Delicatessen Environments Using Expert Elicitation and Machine Learning"],"prefix":"10.1111","volume":"34","author":[{"given":"P.","family":"Vangay","sequence":"first","affiliation":[{"name":"Department of Food Science Cornell University  Ithaca NY USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"J.","family":"Steingrimsson","sequence":"additional","affiliation":[{"name":"Department of Statistical Science Cornell University  Ithaca NY USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M.","family":"Wiedmann","sequence":"additional","affiliation":[{"name":"Department of Food Science Cornell University  Ithaca NY USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M. J.","family":"Stasiewicz","sequence":"additional","affiliation":[{"name":"Department of Food Science Cornell University  Ithaca NY USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2014,6,2]]},"reference":[{"key":"e_1_2_7_2_1","doi-asserted-by":"publisher","DOI":"10.3201\/eid1701.P11101"},{"key":"e_1_2_7_3_1","doi-asserted-by":"publisher","DOI":"10.4315\/0362-028X-73.4.620"},{"key":"e_1_2_7_4_1","doi-asserted-by":"publisher","DOI":"10.4315\/0362-028X-73.4.612"},{"key":"e_1_2_7_5_1","unstructured":"USDA FSIS Recall Summaries2012. Available at:http:\/\/www.fsis.usda.gov\/wps\/portal\/fsis\/topics\/recalls\u2013and\u2013public\u2013health\u2013alerts\/recall\u2013summaries\/recall\u2013summaries\u20132012 2013 Accessed June 1 2013."},{"key":"e_1_2_7_6_1","doi-asserted-by":"publisher","DOI":"10.4315\/0362-028X-67.7.1417"},{"key":"e_1_2_7_7_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0950268805005376"},{"key":"e_1_2_7_8_1","doi-asserted-by":"publisher","DOI":"10.1086\/428575"},{"issue":"10","key":"e_1_2_7_9_1","doi-asserted-by":"crossref","first-page":"2259","DOI":"10.1128\/jcm.28.10.2259-2263.1990","article-title":"Characterization by DNA restriction endonuclease analysis of Listeria monocytogenes strains related to the Swiss epidemic of listeriosis","volume":"28","author":"Nocera D","year":"1990","journal-title":"Journal of Clinical Microbiology"},{"key":"e_1_2_7_10_1","unstructured":"Lessons Learned: Public Health Agency of Canada's Response to the 2008 Listeriosis Outbreak 2013. Available at:http:\/\/www.phac\u2013aspc.gc.ca\/fs\u2013sa\/listeria\/2008\u2013lessons\u2013lecons\u2013eng.php Accessed June 1 2013."},{"key":"e_1_2_7_11_1","doi-asserted-by":"publisher","DOI":"10.4315\/0362-028X.JFP-11-417"},{"key":"e_1_2_7_12_1","unstructured":"SteenhuysenJ DorfmanB.Peanut Costs Adding Up for Food Companies. Available at:http:\/\/www.reuters.com\/article\/2009\/02\/05\/us\u2013usa\u2013salmonella\u2013idUSTRE5146KL20090205 Accessed October 1 2013."},{"key":"e_1_2_7_13_1","unstructured":"Grocery Manufacturers Association: Capturing Recall Losts. Available at:http:\/\/www.gmaonline.org\/file\u2010manager\/images\/gmapublications\/Capturing_Recall_Costs_GMA_Whitepaper_FINAL.pdf Accessed October 1 2013."},{"key":"e_1_2_7_14_1","doi-asserted-by":"publisher","DOI":"10.1078\/072320203770865846"},{"key":"e_1_2_7_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijfoodmicro.2011.01.005"},{"key":"e_1_2_7_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0168-1605(00)00442-6"},{"key":"e_1_2_7_17_1","doi-asserted-by":"publisher","DOI":"10.4315\/0362-028X-66.11.2062"},{"key":"e_1_2_7_18_1","doi-asserted-by":"publisher","DOI":"10.4315\/0362-028X.JFP-12-236"},{"key":"e_1_2_7_19_1","unstructured":"Expert Elicitation Task Force White Paper.U.S. Environmental Protection Agency 2011. Available at:http:\/\/www.epa.gov\/stpc\/pdfs\/ee\u2010white\u2010paper\u2010final.pdf Accessed June 1 2013."},{"key":"e_1_2_7_20_1","unstructured":"KarnsSA Muth MK Coglaiti MC. Results of an Additional Expert Elicitation on the Relative Risks of Meat and Poultry Products: Final Report. RTI Project No. 0208893.027. Prepared for U.S. Department of Agriculture Food Safety and Inspection Service Washington DC by RTI International Research Triangle Park NC July2007. Available at:http:\/\/www.rti.org\/pubs\/08893_027_final.pdf Accessed June 13 2013."},{"key":"e_1_2_7_21_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0950268811001257"},{"key":"e_1_2_7_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.foodcont.2011.12.025"},{"key":"e_1_2_7_23_1","doi-asserted-by":"publisher","DOI":"10.4315\/0362-028X-70.5.1220"},{"key":"e_1_2_7_24_1","doi-asserted-by":"publisher","DOI":"10.1186\/1476-069X-9-19"},{"key":"e_1_2_7_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1539-6924.2011.01729.x"},{"key":"e_1_2_7_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0040-1625(71)80019-7"},{"key":"e_1_2_7_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-2070(99)00018-7"},{"key":"e_1_2_7_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.techfore.2008.10.006"},{"issue":"5","key":"e_1_2_7_29_1","first-page":"408","article-title":"An experimental study of group opinion: The Delphi method","volume":"1","author":"Dalkey N","year":"1969","journal-title":"Journal of Family Theory and Review"},{"key":"e_1_2_7_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ress.2005.11.054"},{"key":"e_1_2_7_31_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0013965"},{"key":"e_1_2_7_32_1","unstructured":"SimmonsC WrightE MalleyT StasiewiczM WarchockiS KauseJ AkingbadeD WiedmannM OliverHF.Ecology ofListeria monocytogenesin the retail deli environment. Poster presentation 2011."},{"key":"e_1_2_7_33_1","volume-title":"A Longitudinal Study of L. monocytogenes Control in Retail Delis Evaluating ATP Fluorescence as a Monitoring Tool","author":"Stasiewicz MJ","year":"2011"},{"key":"e_1_2_7_34_1","doi-asserted-by":"publisher","DOI":"10.4315\/0362-028X.JFP-10-567"},{"key":"e_1_2_7_35_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1992.4.2.249"},{"key":"e_1_2_7_36_1","volume-title":"Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change","author":"Solomon S","year":"2007"},{"key":"e_1_2_7_37_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v011.i09"},{"key":"e_1_2_7_38_1","article-title":"Classification and regression by randomForest","author":"Liaw A","year":"2012","journal-title":"R News"},{"key":"e_1_2_7_39_1","unstructured":"BatesD MaechlerD BolkerB WalkerS.lme4: Linear mixed\u2010effects models using Eigen and S4.2014. R package version 1.0\u20136. Available at:http:\/\/CRAN.R\u2010project.org\/package=lme4 Accessed March 15 2013."},{"key":"e_1_2_7_40_1","article-title":"neuralnet: Training of neural networks","author":"G\u00fcnther F","year":"2012","journal-title":"R Journal"},{"key":"e_1_2_7_41_1","unstructured":"KuhnM. Wing J. Weston S. Williams A. Keefer C. Engelhardt A. Cooper T. Mayer Z. the R Core Team. caret: Classification and Regression Training.2014. R package version 6.0\u201324. Available at:http:\/\/CRAN.R\u2010project.org\/package=caret Accessed March 13 2013."},{"key":"e_1_2_7_42_1","doi-asserted-by":"publisher","DOI":"10.4315\/0362-028X.JFP-13-150"},{"key":"e_1_2_7_43_1","doi-asserted-by":"publisher","DOI":"10.4315\/0362-028X-67.7.1417"},{"key":"e_1_2_7_44_1","unstructured":"Full Text of the Food Safety Modernization Act (FSMA).U.\u00a0S. Food and Drug Administration 2011;111\u2013353."},{"key":"e_1_2_7_45_1","unstructured":"Butts J.2003. Seek & destroy: Identifying and controlling Listeria monocytogenes growth niches. Available at:http:\/\/www.foodsafetymagazine.com\/magazine\u2010archive1\/aprilmay\u20102003\/seek\u2010destroy\u2010identifying\u2010and\u2010controlling\u2010listeria\u2010monocytogenes\u2010growth\u2010niches Accessed March 1 2013."},{"issue":"7","key":"e_1_2_7_46_1","first-page":"7","article-title":"The cost and benefit of Listeria monocytogenes food safety measures","volume":"44","author":"Ivanek R","year":"2004","journal-title":"Critical Reviews in Food Science and Nutrition"},{"key":"e_1_2_7_47_1","article-title":"Caract\u00e9risation des populations de Listeria monocytogenes dans la fili\u00e8re poisson fum\u00e9","author":"Ragimbeau C","year":"2002","journal-title":"Universit\u00e9 des sciences et technologies de Lille"}],"container-title":["Risk Analysis"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1111%2Frisa.12218","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/risa.12218","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,1]],"date-time":"2023-10-01T07:02:21Z","timestamp":1696143741000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/risa.12218"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,6,2]]},"references-count":46,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2014,10]]}},"alternative-id":["10.1111\/risa.12218"],"URL":"https:\/\/doi.org\/10.1111\/risa.12218","archive":["Portico"],"relation":{},"ISSN":["0272-4332","1539-6924"],"issn-type":[{"value":"0272-4332","type":"print"},{"value":"1539-6924","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,6,2]]}}}