{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T23:23:58Z","timestamp":1788996238368,"version":"build-2803163510"},"reference-count":122,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2022,12,27]],"date-time":"2022-12-27T00:00:00Z","timestamp":1672099200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/100001341","name":"Richard and Susan Smith Family Foundation","doi-asserted-by":"publisher","award":["Odyssey award"],"award-info":[{"award-number":["Odyssey award"]}],"id":[{"id":"10.13039\/100001341","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000060","name":"National Institute of Allergy and Infectious Diseases","doi-asserted-by":"publisher","award":["R01 AI132606"],"award-info":[{"award-number":["R01 AI132606"]}],"id":[{"id":"10.13039\/100000060","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000060","name":"National Institute of Allergy and Infectious Diseases","doi-asserted-by":"publisher","award":["R01 AI153521"],"award-info":[{"award-number":["R01 AI153521"]}],"id":[{"id":"10.13039\/100000060","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["nyaspubs.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Annals of the New York Academy of Sciences"],"published-print":{"date-parts":[[2023,2]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p><jats:italic>Neisseria gonorrhoeae<\/jats:italic> is an urgent public health threat due to the emergence of antibiotic resistance. As most isolates in the United States are susceptible to at least one antibiotic, rapid molecular antimicrobial susceptibility tests (ASTs) would offer the opportunity to tailor antibiotic therapy, thereby expanding treatment options. With genome sequence and antibiotic resistance phenotype data for nearly 20,000 clinical <jats:italic>N. gonorrhoeae<\/jats:italic> isolates now available, there is an opportunity to use statistical methods to develop sequence\u2010based diagnostics that predict antibiotic susceptibility from genotype. <jats:italic>N. gonorrhoeae<\/jats:italic>, therefore, provides a useful example illustrating how to apply machine learning models to aid in the design of sequence\u2010based ASTs. We present an overview of this framework, which begins with establishing the assay technology, the performance criteria, the population in which the diagnostic will be used, and the clinical goals, and extends to the choices that must be made to arrive at a set of features with the desired properties for predicting susceptibility phenotype from genotype. While we focus on the example of <jats:italic>N. gonorrhoeae<\/jats:italic>, the framework generalizes to other organisms for which large\u2010scale genotype and antibiotic resistance data can be combined to aid in diagnostics development.<\/jats:p>","DOI":"10.1111\/nyas.14549","type":"journal-article","created":{"date-parts":[[2022,12,27]],"date-time":"2022-12-27T13:31:11Z","timestamp":1672147871000},"page":"74-88","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Machine learning models for <i>Neisseria gonorrhoeae<\/i> antimicrobial susceptibility tests"],"prefix":"10.1111","volume":"1520","author":[{"given":"Skylar L.","family":"Martin","sequence":"first","affiliation":[{"name":"Department of Immunology and Infectious Diseases Harvard T.H. Chan School of Public Health  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6255-690X","authenticated-orcid":false,"given":"Tatum D.","family":"Mortimer","sequence":"additional","affiliation":[{"name":"Department of Immunology and Infectious Diseases Harvard T.H. Chan School of Public Health  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5646-1314","authenticated-orcid":false,"given":"Yonatan H.","family":"Grad","sequence":"additional","affiliation":[{"name":"Department of Immunology and Infectious Diseases Harvard T.H. Chan School of Public Health  Boston Massachusetts USA"},{"name":"Division of Infectious Diseases, Brigham and Women's Hospital Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2022,12,27]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(21)02724-0"},{"key":"e_1_2_8_3_1","unstructured":"Centers for Disease Control and Prevention (U.S.). (2019).Antibiotic resistance threats in the United States 2019.https:\/\/stacks.cdc.gov\/view\/cdc\/82532"},{"key":"e_1_2_8_4_1","volume-title":"Mandell, Douglas, and Bennett's principles and practice of infectious diseases","author":"Bennett J. E.","year":"2020"},{"key":"e_1_2_8_5_1","volume-title":"Diagnoses of gonorrhoea in Australia","author":"Australian Government Department of Health","year":"2018"},{"key":"e_1_2_8_6_1","volume-title":"National overview\u2013\u2013Sexually transmitted disease surveillance, 2019","author":"Centers for Disease Control and Prevention","year":"2021"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m3425"},{"key":"e_1_2_8_8_1","unstructured":"Centers for Disease Control and Prevention. (2022).Gonococcal infections among adolescents and adults \u2010 STI treatment guidelines. Sexually Transmitted Infections Treatment Guidelines 2021."},{"key":"e_1_2_8_9_1","doi-asserted-by":"publisher","DOI":"10.1080\/14737159.2020.1778467"},{"key":"e_1_2_8_10_1","doi-asserted-by":"publisher","DOI":"10.1128\/CMR.00010-14"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1038\/nrmicro3217"},{"key":"e_1_2_8_12_1","first-page":"1900147","article-title":"Detection in the United Kingdom of the Neisseria gonorrhoeae FC428 clone, with ceftriaxone resistance and intermediate resistance to azithromycin, October to December 2018","volume":"24","author":"Eyre D. W.","year":"2019","journal-title":"Euro Surveillance"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1810840115"},{"key":"e_1_2_8_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1473-3099(18)30274-3"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa1706988"},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1093\/cid\/ciy145"},{"key":"e_1_2_8_17_1","doi-asserted-by":"publisher","DOI":"10.1093\/cid\/ciaa1229"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1473-3099(13)70693-5"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-17980-1"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.1093\/jac\/dkx520"},{"key":"e_1_2_8_21_1","first-page":"2100197","article-title":"Surveillance and epidemiology of syphilis, gonorrhoea and chlamydia in the non\u2010European union countries of the World Health Organization European region, 2015 to 2020","volume":"27","author":"Barbaric J.","year":"2022","journal-title":"Euro Surveillance"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2666-5247(22)00044-1"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1093\/jacamr\/dlac076"},{"key":"e_1_2_8_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1473-3099(18)30122-1"},{"key":"e_1_2_8_25_1","doi-asserted-by":"publisher","DOI":"10.3201\/eid2404.171873"},{"key":"e_1_2_8_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1473-3099(16)30157-8"},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.ppat.1005611"},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.3001579"},{"key":"e_1_2_8_29_1","doi-asserted-by":"publisher","DOI":"10.1093\/infdis\/jiy551"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1473-3099(15)00002-X"},{"key":"e_1_2_8_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2666-5247(22)00093-3"},{"key":"e_1_2_8_32_1","doi-asserted-by":"publisher","DOI":"10.1128\/mBio.02962-20"},{"key":"e_1_2_8_33_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13073-021-00868-0"},{"key":"e_1_2_8_34_1","doi-asserted-by":"publisher","DOI":"10.1038\/ng.3767"},{"key":"e_1_2_8_35_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2958.1994.tb01297.x"},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.01506-09"},{"key":"e_1_2_8_37_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.42.11.2799"},{"key":"e_1_2_8_38_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.49.10.4327-4334.2005"},{"key":"e_1_2_8_39_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.49.1.137-143.2005"},{"key":"e_1_2_8_40_1","doi-asserted-by":"publisher","DOI":"10.1034\/j.1600-0463.2002.1100909.x"},{"key":"e_1_2_8_41_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.30.5.664"},{"key":"e_1_2_8_42_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.46.9.3020-3025.2002"},{"key":"e_1_2_8_43_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.43.6.1367"},{"key":"e_1_2_8_44_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.46.3.769-777.2002"},{"key":"e_1_2_8_45_1","doi-asserted-by":"publisher","DOI":"10.1128\/jb.124.2.740-749.1975"},{"key":"e_1_2_8_46_1","doi-asserted-by":"publisher","DOI":"10.1038\/332173a0"},{"key":"e_1_2_8_47_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.00304-09"},{"key":"e_1_2_8_48_1","doi-asserted-by":"publisher","DOI":"10.1099\/13500872-141-3-611"},{"key":"e_1_2_8_49_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.11.3.528"},{"key":"e_1_2_8_50_1","doi-asserted-by":"publisher","DOI":"10.1099\/mgen.0.000359"},{"key":"e_1_2_8_51_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-20015-x"},{"key":"e_1_2_8_52_1","doi-asserted-by":"publisher","DOI":"10.1093\/infdis\/jiy240"},{"key":"e_1_2_8_53_1","doi-asserted-by":"publisher","DOI":"10.1093\/jac\/dkx067"},{"key":"e_1_2_8_54_1","doi-asserted-by":"publisher","DOI":"10.1093\/jac\/dkab082"},{"key":"e_1_2_8_55_1","doi-asserted-by":"publisher","DOI":"10.1128\/JCM.03195-15"},{"key":"e_1_2_8_56_1","doi-asserted-by":"publisher","DOI":"10.1128\/JCM.02589-14"},{"key":"e_1_2_8_57_1","doi-asserted-by":"publisher","DOI":"10.1128\/mBio.00525-16"},{"key":"e_1_2_8_58_1","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.806"},{"key":"e_1_2_8_59_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2666-5247(20)30059-8"},{"key":"e_1_2_8_60_1","doi-asserted-by":"publisher","DOI":"10.1093\/jac\/dkaa318"},{"key":"e_1_2_8_61_1","doi-asserted-by":"publisher","DOI":"10.1136\/sextrans-2017-053287"},{"key":"e_1_2_8_62_1","doi-asserted-by":"publisher","DOI":"10.1093\/jac\/dkx405"},{"key":"e_1_2_8_63_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.02514-19"},{"key":"e_1_2_8_64_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eclinm.2019.01.010"},{"key":"e_1_2_8_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10096-020-03907-7"},{"key":"e_1_2_8_66_1","doi-asserted-by":"publisher","DOI":"10.1097\/OLQ.0000000000001471"},{"key":"e_1_2_8_67_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13073-021-00858-2"},{"key":"e_1_2_8_68_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-24072-1"},{"key":"e_1_2_8_69_1","doi-asserted-by":"publisher","DOI":"10.3201\/eid2603.190732"},{"key":"e_1_2_8_70_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-019-12053-4"},{"key":"e_1_2_8_71_1","first-page":"e000205","article-title":"Genomic surveillance of Neisseria gonorrhoeae to investigate the distribution and evolution of antimicrobial\u2010resistance determinants and lineages","volume":"4","author":"Yahara K.","year":"2018","journal-title":"Microbial Genomics"},{"key":"e_1_2_8_72_1","doi-asserted-by":"publisher","DOI":"10.5365\/wpsar.2020.11.1.005"},{"key":"e_1_2_8_73_1","doi-asserted-by":"publisher","DOI":"10.1093\/infdis\/jiw420"},{"key":"e_1_2_8_74_1","doi-asserted-by":"publisher","DOI":"10.1111\/nyas.13871"},{"key":"e_1_2_8_75_1","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.51407"},{"key":"e_1_2_8_76_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-19250-6"},{"key":"e_1_2_8_77_1","doi-asserted-by":"publisher","DOI":"10.1128\/mBio.01509-20"},{"key":"e_1_2_8_78_1","doi-asserted-by":"publisher","DOI":"10.1128\/AAC.00885-20"},{"key":"e_1_2_8_79_1","doi-asserted-by":"publisher","DOI":"10.1128\/mBio.02281-18"},{"key":"e_1_2_8_80_1","doi-asserted-by":"publisher","DOI":"10.1128\/mBio.01419-18"},{"key":"e_1_2_8_81_1","first-page":"e02005","article-title":"Equations to predict antimicrobial MICs in Neisseria gonorrhoeae using molecular antimicrobial resistance determinants","volume":"64","author":"Demczuk W.","year":"2019","journal-title":"Antimicrobial Agents and Chemotherapy"},{"key":"e_1_2_8_82_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(76)92467-3"},{"key":"e_1_2_8_83_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-017-0881-x"},{"key":"e_1_2_8_84_1","doi-asserted-by":"publisher","DOI":"10.1093\/infdis\/jix450"},{"key":"e_1_2_8_85_1","doi-asserted-by":"publisher","DOI":"10.2807\/1560-7917.ES.2020.25.43.1900210"},{"key":"e_1_2_8_86_1","doi-asserted-by":"publisher","DOI":"10.1080\/14737159.2017.1360137"},{"key":"e_1_2_8_87_1","doi-asserted-by":"publisher","DOI":"10.1128\/JCM.02214-10"},{"key":"e_1_2_8_88_1","doi-asserted-by":"publisher","DOI":"10.1128\/JCM.01195-19"},{"key":"e_1_2_8_89_1","doi-asserted-by":"publisher","DOI":"10.1093\/jac\/dkaa381"},{"key":"e_1_2_8_90_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1007349"},{"key":"e_1_2_8_91_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2666-5247(22)00034-9"},{"key":"e_1_2_8_92_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41564-019-0656-6"},{"key":"e_1_2_8_93_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-40561-2"},{"key":"e_1_2_8_94_1","doi-asserted-by":"publisher","DOI":"10.1071\/SH18227"},{"key":"e_1_2_8_95_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.sjbs.2022.02.047"},{"key":"e_1_2_8_96_1","doi-asserted-by":"publisher","DOI":"10.1128\/JCM.01260-20"},{"key":"e_1_2_8_97_1","doi-asserted-by":"publisher","DOI":"10.3389\/fcimb.2021.610348"},{"key":"e_1_2_8_98_1","doi-asserted-by":"publisher","DOI":"10.1128\/cmr.00179-21"},{"key":"e_1_2_8_99_1","doi-asserted-by":"publisher","DOI":"10.1093\/jac\/dkab447"},{"key":"e_1_2_8_100_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-020-2279-8"},{"key":"e_1_2_8_101_1","volume-title":"Antimicrobial susceptibility test (AST) systems \u2010 Class II special controls guidance for industry and FDA","author":"U.S. Food and Drug Administration","year":"2018"},{"key":"e_1_2_8_102_1","doi-asserted-by":"publisher","DOI":"10.1093\/jac\/dkaa040"},{"key":"e_1_2_8_103_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1473-3099(18)30225-1"},{"key":"e_1_2_8_104_1","volume-title":"CDC pilots drug\u2010resistant gonorrhea testing","author":"Centers for Disease Control and Prevention","year":"2022"},{"key":"e_1_2_8_105_1","volume-title":"Performance standards for antimicrobial susceptibility testing","author":"CLSI","year":"2018"},{"key":"e_1_2_8_106_1","doi-asserted-by":"publisher","DOI":"10.1093\/jac\/dkab308"},{"key":"e_1_2_8_107_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41580-021-00407-0"},{"key":"e_1_2_8_108_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/bty539"},{"key":"e_1_2_8_109_1","doi-asserted-by":"publisher","DOI":"10.1038\/nmicrobiol.2016.41"},{"key":"e_1_2_8_110_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.98.1.182"},{"key":"e_1_2_8_111_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.micro.55.1.561"},{"key":"e_1_2_8_112_1","unstructured":"EUCAST. (2022).EUCAST: Clinical breakpoints and dosing of antibiotics.https:\/\/www.eucast.org\/clinical_breakpoints\/"},{"key":"e_1_2_8_113_1","unstructured":"British Association for Sexual Health and HIV. (2018).British Association for Sexual Health and HIV 2018 Accessed July 21 2022.https:\/\/www.bashh.org\/guidelines\/"},{"key":"e_1_2_8_114_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.332.7549.1080"},{"key":"e_1_2_8_115_1","doi-asserted-by":"crossref","unstructured":"Reshetnikov K. O. Bykova D. I. Kuleshov K. V. Chukreev K. Guguchkin E. P. Akimkin V. G. Neverov A. D. &Fedonin G. G.(2022). Feature selection and aggregation for antibiotic resistance GWAS inMycobacterium tuberculosis: A comparative study. bioRxiv 2022.03.16.484601.","DOI":"10.1101\/2022.03.16.484601"},{"key":"e_1_2_8_116_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13015-021-00198-1"},{"key":"e_1_2_8_117_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41564-021-00962-4"},{"key":"e_1_2_8_118_1","doi-asserted-by":"publisher","DOI":"10.1099\/mic.0.036830-0"},{"key":"e_1_2_8_119_1","first-page":"19874","article-title":"False\u2010negative results using Neisseria gonorrhoeae porA pseudogene PCR \u2010 A clinical gonococcal isolate with an N. meningitidis porA sequence, Australia, March 2011","volume":"16","author":"Whiley D. M.","year":"2011","journal-title":"Euro Surveillance"},{"key":"e_1_2_8_120_1","article-title":"A variant of Chlamydia trachomatis with deletion in cryptic plasmid: Implications for use of PCR diagnostic tests","volume":"11","author":"Ripa T.","year":"2006","journal-title":"Euro Surveillance"},{"key":"e_1_2_8_121_1","doi-asserted-by":"publisher","DOI":"10.1128\/JCM.01337-19"},{"key":"e_1_2_8_122_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.3000547"},{"key":"e_1_2_8_123_1","doi-asserted-by":"publisher","DOI":"10.1128\/spectrum.02065-21"}],"container-title":["Annals of the New York Academy of Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/nyas.14549","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/nyas.14549","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/nyaspubs.onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/nyas.14549","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,18]],"date-time":"2023-08-18T07:37:12Z","timestamp":1692344232000},"score":1,"resource":{"primary":{"URL":"https:\/\/nyaspubs.onlinelibrary.wiley.com\/doi\/10.1111\/nyas.14549"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,27]]},"references-count":122,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,2]]}},"alternative-id":["10.1111\/nyas.14549"],"URL":"https:\/\/doi.org\/10.1111\/nyas.14549","archive":["Portico"],"relation":{},"ISSN":["0077-8923","1749-6632"],"issn-type":[{"value":"0077-8923","type":"print"},{"value":"1749-6632","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,27]]},"assertion":[{"value":"2022-12-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}