{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,28]],"date-time":"2026-10-28T03:44:45Z","timestamp":1793159085535,"version":"4.5.0"},"reference-count":60,"publisher":"Public Library of Science (PLoS)","issue":"12","license":[{"start":{"date-parts":[[2021,12,2]],"date-time":"2021-12-02T00:00:00Z","timestamp":1638403200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000030","name":"Centers for Disease Control and Prevention","doi-asserted-by":"publisher","award":["U01CK000505"],"award-info":[{"award-number":["U01CK000505"]}],"id":[{"id":"10.13039\/100000030","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000030","name":"Centers for Disease Control and Prevention","doi-asserted-by":"publisher","award":["U01CK000505"],"award-info":[{"award-number":["U01CK000505"]}],"id":[{"id":"10.13039\/100000030","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>\n                    Ticks and tick-borne diseases represent a growing public health threat in North America and Europe. The number of ticks, their geographical distribution, and the incidence of tick-borne diseases, like Lyme disease, are all on the rise. Accurate, real-time tick-image identification through a smartphone app or similar platform could help mitigate this threat by informing users of the risks associated with encountered ticks and by providing researchers and public health agencies with additional data on tick activity and geographic range. Here we outline the requirements for such a system, present a model that meets those requirements, and discuss remaining challenges and frontiers in automated tick identification. We compiled a user-generated dataset of more than 12,000 images of the three most common tick species found on humans in the U.S.:\n                    <jats:italic>Amblyomma americanum<\/jats:italic>\n                    ,\n                    <jats:italic>Dermacentor variabilis<\/jats:italic>\n                    , and\n                    <jats:italic>Ixodes scapularis<\/jats:italic>\n                    . We used image augmentation to further increase the size of our dataset to more than 90,000 images. Here we report the development and validation of a convolutional neural network which we call \u201cTickIDNet,\u201d that scores an 87.8% identification accuracy across all three species, outperforming the accuracy of identifications done by a member of the general public or healthcare professionals. However, the model fails to match the performance of experts with formal entomological training. We find that image quality, particularly the size of the tick in the image (measured in pixels), plays a significant role in the network\u2019s ability to correctly identify an image: images where the tick is small are less likely to be correctly identified because of the small object detection problem in deep learning. TickIDNet\u2019s performance can be increased by using confidence thresholds to introduce an \u201cunsure\u201d class and building image submission pipelines that encourage better quality photos. Our findings suggest that deep learning represents a promising frontier for tick identification that should be further explored and deployed as part of the toolkit for addressing the public health consequences of tick-borne diseases.\n                  <\/jats:p>","DOI":"10.1371\/journal.pone.0260622","type":"journal-article","created":{"date-parts":[[2021,12,2]],"date-time":"2021-12-02T14:18:03Z","timestamp":1638454683000},"page":"e0260622","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":14,"title":["Identification of public submitted tick images: A neural network approach"],"prefix":"10.1371","volume":"16","author":[{"given":"Lennart","family":"Justen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0614-2934","authenticated-orcid":true,"given":"Duncan","family":"Carlsmith","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Susan M.","family":"Paskewitz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lyric C.","family":"Bartholomay","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4431-2482","authenticated-orcid":true,"given":"Gebbiena M.","family":"Bron","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2021,12,2]]},"reference":[{"issue":"3","key":"pone.0260622.ref001","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1093\/ilar\/ilx005","article-title":"Tick-Borne Zoonoses in the United States: Persistent and Emerging Threats to Human Health","volume":"58","author":"RJ Eisen","year":"2017","journal-title":"ILAR Journal"},{"issue":"17","key":"pone.0260622.ref002","doi-asserted-by":"crossref","first-page":"496","DOI":"10.15585\/mmwr.mm6717e1","article-title":"Vital Signs: Trends in Reported Vectorborne Disease Cases\u2014United States and Territories, 2004\u20132016","volume":"67","author":"R Rosenberg","year":"2018","journal-title":"MMWR Morb Mortal Wkly Rep"},{"issue":"2","key":"pone.0260622.ref003","doi-asserted-by":"crossref","first-page":"616","DOI":"10.3201\/eid2702.202731","article-title":"Estimating the Frequency of Lyme Disease Diagnoses, United States, 2010\u20132018","volume":"27","author":"K Kugeler","year":"2021","journal-title":"Emerging Infectious Disease journal"},{"issue":"2","key":"pone.0260622.ref004","doi-asserted-by":"crossref","first-page":"499","DOI":"10.3201\/eid2702.202728","article-title":"Use of Commercial Claims Data for Evaluating Trends in Lyme Disease Diagnoses, United States, 2010\u20132018","volume":"27","author":"A Schwartz","year":"2021","journal-title":"Emerging Infectious Disease journal"},{"issue":"5","key":"pone.0260622.ref005","doi-asserted-by":"crossref","first-page":"676","DOI":"10.1093\/cid\/ciu397","article-title":"Lyme Disease Testing by Large Commercial Laboratories in the United States","volume":"59","author":"AF Hinckley","year":"2014","journal-title":"Clinical Infectious Diseases"},{"issue":"9","key":"pone.0260622.ref006","doi-asserted-by":"crossref","first-page":"1625","DOI":"10.3201\/eid2109.150417","article-title":"Incidence of Clinician-Diagnosed Lyme Disease, United States, 2005\u20132010","volume":"21","author":"C Nelson","year":"2015","journal-title":"Emerging Infectious Disease journal"},{"issue":"4","key":"pone.0260622.ref007","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1016\/j.pt.2017.12.006","article-title":"The Blacklegged Tick, Ixodes scapularis: An Increasing Public Health Concern","volume":"34","author":"RJ Eisen","year":"2018","journal-title":"Trends in Parasitology"},{"issue":"5","key":"pone.0260622.ref008","doi-asserted-by":"crossref","first-page":"1176","DOI":"10.1093\/jme\/tjw076","article-title":"Modeling the Geographic Distribution of Ixodes scapularis and Ixodes pacificus (Acari: Ixodidae) in the Contiguous United States","volume":"53","author":"MB Hahn","year":"2016","journal-title":"Journal of Medical Entomology"},{"issue":"8","key":"pone.0260622.ref009","doi-asserted-by":"crossref","first-page":"1455","DOI":"10.3201\/eid2108.141878","article-title":"Geographic Distribution and Expansion of Human Lyme Disease, United States","volume":"21","author":"K Kugeler","year":"2015","journal-title":"Emerging Infectious Disease journal"},{"issue":"2","key":"pone.0260622.ref010","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1093\/jme\/tjv237","article-title":"County-Scale Distribution of Ixodes scapularis and Ixodes pacificus (Acari: Ixodidae) in the Continental United States","volume":"53","author":"RJ Eisen","year":"2016","journal-title":"Journal of Medical Entomology"},{"issue":"9","key":"pone.0260622.ref011","doi-asserted-by":"crossref","first-page":"1089","DOI":"10.1086\/508667","article-title":"The Clinical Assessment, Treatment, and Prevention of Lyme Disease, Human Granulocytic Anaplasmosis, and Babesiosis: Clinical Practice Guidelines by the Infectious Diseases Society of America","volume":"43","author":"GP Wormser","year":"2006","journal-title":"Clinical Infectious Diseases"},{"issue":"2","key":"pone.0260622.ref012","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1016\/j.ttbdis.2016.11.010","article-title":"Use of a tick-borne disease manual increases accuracy of tick identification among primary care providers in Lyme disease endemic areas","volume":"8","author":"AD Butler","year":"2017","journal-title":"Ticks and Tick-borne Diseases"},{"key":"pone.0260622.ref013","article-title":"A Survey of Tick Surveillance and Control Practices in the United States","author":"EM Mader","year":"2020","journal-title":"Journal of Medical Entomology"},{"issue":"2","key":"pone.0260622.ref014","doi-asserted-by":"crossref","first-page":"e0211778","DOI":"10.1371\/journal.pone.0211778","article-title":"The growing importance of lone star ticks in a Lyme disease endemic county: Passive tick surveillance in Monmouth County, NJ, 2006\u20132016","volume":"14","author":"RA Jordan","year":"2019","journal-title":"PLOS ONE"},{"issue":"10","key":"pone.0260622.ref015","doi-asserted-by":"crossref","first-page":"e14769","DOI":"10.2196\/14769","article-title":"Usability and Feasibility of a Smartphone App to Assess Human Behavioral Factors Associated with Tick Exposure (The Tick App): Quantitative and Qualitative Study","volume":"7","author":"MP Fernandez","year":"2019","journal-title":"JMIR Mhealth Uhealth"},{"issue":"1","key":"pone.0260622.ref016","doi-asserted-by":"crossref","DOI":"10.3390\/healthcare6010022","article-title":"Citizen Science and Community Engagement in Tick Surveillance\u2014A Canadian Case Study","volume":"6","author":"J Lewis","year":"2018","journal-title":"Healthcare"},{"issue":"1","key":"pone.0260622.ref017","doi-asserted-by":"crossref","first-page":"744","DOI":"10.1186\/s12879-017-2836-4","article-title":"Prevention of tick bites: an evaluation of a smartphone app","volume":"17","author":"L Antonise-Kamp","year":"2017","journal-title":"BMC Infectious Diseases"},{"issue":"6","key":"pone.0260622.ref018","doi-asserted-by":"crossref","first-page":"1955","DOI":"10.1093\/jme\/tjaa140","article-title":"Crowdsourced Photographs as an Effective Method for Large-Scale Passive Tick Surveillance","volume":"57","author":"HL Kopsco","year":"2020","journal-title":"Journal of Medical Entomology"},{"issue":"1","key":"pone.0260622.ref019","first-page":"1","article-title":"Benefits and Drawbacks of Citizen Science to Complement Traditional Data Gathering Approaches for Medically Important Hard Ticks (Acari: Ixodidae) in the United States","volume":"58","author":"L Eisen","year":"2021","journal-title":"Journal of Medical Entomology"},{"issue":"4","key":"pone.0260622.ref020","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1016\/j.ttbdis.2017.03.001","article-title":"A comparative test of ixodid tick identification by a network of European researchers","volume":"8","author":"A Estrada-Pe\u00f1a","year":"2017","journal-title":"Ticks and Tick-borne Diseases"},{"issue":"7312","key":"pone.0260622.ref021","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1038\/467154a","article-title":"Time to automate identification","volume":"467","author":"N MacLeod","year":"2010","journal-title":"Nature"},{"issue":"2","key":"pone.0260622.ref022","doi-asserted-by":"crossref","first-page":"866","DOI":"10.1093\/jme\/tjaa234","article-title":"Assessing Recognition of the Vector of Lyme Disease Using Resin-Embedded Specimens in a Lyme Endemic Area","volume":"58","author":"GM Bron","year":"2021","journal-title":"Journal of Medical Entomology"},{"issue":"2","key":"pone.0260622.ref023","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1093\/jme\/tjaa196","article-title":"Assessing Public Tick Identification Ability and Tick Bite Riskiness Using Passive Photograph-Based Crowdsourced Tick Surveillance","volume":"58","author":"HL Kopsco","year":"2021","journal-title":"Journal of Medical Entomology"},{"issue":"7639","key":"pone.0260622.ref024","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"A Esteva","year":"2017","journal-title":"Nature"},{"key":"pone.0260622.ref025","doi-asserted-by":"crossref","unstructured":"S. Liu, S. Liu, W. Cai, S. Pujol, R. Kikinis, D. Feng. Early diagnosis of Alzheimer\u2019s disease with deep learning. In: 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI). 2014. p. 1015\u20138.","DOI":"10.1109\/ISBI.2014.6868045"},{"issue":"9","key":"pone.0260622.ref026","doi-asserted-by":"crossref","first-page":"1342","DOI":"10.1038\/s41591-018-0107-6","article-title":"Clinically applicable deep learning for diagnosis and referral in retinal disease","volume":"24","author":"J De Fauw","year":"2018","journal-title":"Nat Med"},{"issue":"3","key":"pone.0260622.ref027","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1038\/s41551-018-0195-0","article-title":"Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning","volume":"2","author":"R Poplin","year":"2018","journal-title":"Nature Biomedical Engineering"},{"key":"pone.0260622.ref028","article-title":"Automatic Detection and Classification of Tick-borne Skin Lesions using Deep Learning","author":"LM Pfeifer","year":"2020","journal-title":"CoRR"},{"key":"pone.0260622.ref029","doi-asserted-by":"crossref","first-page":"103977","DOI":"10.1016\/j.compbiomed.2020.103977","article-title":"AI-based detection of erythema migrans and disambiguation against other skin lesions","volume":"125","author":"PM Burlina","year":"2020","journal-title":"Computers in Biology and Medicine"},{"key":"pone.0260622.ref030","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.compbiomed.2018.12.007","article-title":"Automated detection of erythema migrans and other confounding skin lesions via deep learning","volume":"105","author":"PM Burlina","year":"2019","journal-title":"Computers in Biology and Medicine"},{"key":"pone.0260622.ref031","doi-asserted-by":"crossref","unstructured":"S. H. Lee, C. S. Chan, P. Wilkin, P. Remagnino. Deep-plant: Plant identification with convolutional neural networks. In: 2015 IEEE International Conference on Image Processing (ICIP). 2015. p. 452\u20136.","DOI":"10.1109\/ICIP.2015.7350839"},{"issue":"1","key":"pone.0260622.ref032","doi-asserted-by":"crossref","first-page":"150026","DOI":"10.1038\/sdata.2015.26","article-title":"Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna","volume":"2","author":"A Swanson","year":"2015","journal-title":"Scientific Data"},{"issue":"9","key":"pone.0260622.ref033","first-page":"570","article-title":"Fish species classification in unconstrained underwater environments based on deep learning","volume":"14","author":"A Salman","year":"2016","journal-title":"Limnology and Oceanography: Methods"},{"issue":"12","key":"pone.0260622.ref034","doi-asserted-by":"crossref","first-page":"e0008904","DOI":"10.1371\/journal.pntd.0008904","article-title":"Delimiting cryptic morphological variation among human malaria vector species using convolutional neural networks","volume":"14","author":"J Couret","year":"2020","journal-title":"PLOS Neglected Tropical Diseases"},{"key":"pone.0260622.ref035","article-title":"A Computer Vision Approach to Combat Lyme Disease","author":"S Akbarian","year":"2020","journal-title":"CoRR"},{"issue":"3","key":"pone.0260622.ref036","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1007\/s10493-021-00639-x","article-title":"Using convolutional neural networks for tick image recognition\u2013a preliminary exploration","volume":"84","author":"O Omodior","year":"2021","journal-title":"Experimental and Applied Acarology"},{"key":"pone.0260622.ref037","article-title":"Understanding How Image Quality Affects Deep Neural Networks","author":"SF Dodge","year":"2016","journal-title":"CoRR"},{"key":"pone.0260622.ref038","article-title":"Google\u2019s medical AI was super accurate in a lab. Real life was a different story","author":"WD Heaven","year":"2020","journal-title":"MIT Technology Review"},{"issue":"1","key":"pone.0260622.ref039","doi-asserted-by":"crossref","first-page":"10972","DOI":"10.1038\/s41598-020-67573-7","article-title":"A new method to control error rates in automated species identification with deep learning algorithms","volume":"10","author":"S Villon","year":"2020","journal-title":"Scientific Reports"},{"key":"pone.0260622.ref040","unstructured":"GBIF.org. Occurrence Download. The Global Biodiversity Information Facility; 2020. Available from: https:\/\/doi.org\/10.15468\/dl.4gbcs6"},{"key":"pone.0260622.ref041","unstructured":"GBIF.org. Occurrence Download. The Global Biodiversity Information Facility; 2020. Available from: https:\/\/doi.org\/10.15468\/dl.sq29u5"},{"key":"pone.0260622.ref042","unstructured":"GBIF.org. Occurrence Download. The Global Biodiversity Information Facility; 2020. Available from: https:\/\/doi.org\/10.15468\/dl.tyybke"},{"key":"pone.0260622.ref043","article-title":"Small Object Detection using Context and Attention","author":"J-S Lim","year":"2019","journal-title":"CoRR"},{"key":"pone.0260622.ref044","doi-asserted-by":"crossref","first-page":"103910","DOI":"10.1016\/j.imavis.2020.103910","article-title":"Recent advances in small object detection based on deep learning: A review","volume":"97","author":"K Tong","year":"2020","journal-title":"Image and Vision Computing"},{"key":"pone.0260622.ref045","article-title":"The Effectiveness of Data Augmentation in Image Classification using Deep Learning","author":"L Perez","year":"2017","journal-title":"CoRR"},{"issue":"3","key":"pone.0260622.ref046","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1007\/s41664-018-0068-2","article-title":"On Splitting Training and Validation Set: A Comparative Study of Cross-Validation, Bootstrap and Systematic Sampling for Estimating the Generalization Performance of Supervised Learning","volume":"2","author":"Y Xu","year":"2018","journal-title":"Journal of Analysis and Testing"},{"key":"pone.0260622.ref047","article-title":"Adam: A Method for Stochastic Optimization","author":"DP Kingma","year":"2015","journal-title":"CoRR"},{"issue":"2","key":"pone.0260622.ref048","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1007\/s13748-019-00203-0","article-title":"Convolutional neural network: a review of models, methodologies and applications to object detection","volume":"9","author":"A Dhillon","year":"2020","journal-title":"Progress in Artificial Intelligence"},{"key":"pone.0260622.ref049","article-title":"Rethinking the Inception Architecture for Computer Vision","author":"C Szegedy","year":"2015","journal-title":"CoRR"},{"key":"pone.0260622.ref050","article-title":"TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems","author":"M Abadi","year":"2016","journal-title":"CoRR"},{"key":"pone.0260622.ref051","unstructured":"Chollet F, others. Keras. 2015. Available from: https:\/\/keras.io"},{"issue":"5","key":"pone.0260622.ref052","first-page":"360","article-title":"Understanding interobserver agreement: the kappa statistic","volume":"37","author":"AJ Viera","year":"2005","journal-title":"Fam Med"},{"key":"pone.0260622.ref053","first-page":"105","volume-title":"Frontiers in Econometrics","author":"D. McFadden","year":"1973"},{"issue":"1","key":"pone.0260622.ref054","first-page":"1","article-title":"Fitting Linear Mixed-Effects Models Using lme4","volume":"67","author":"D Bates","year":"2015","journal-title":"Journal of Statistical Software, Articles"},{"issue":"1","key":"pone.0260622.ref055","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1186\/1471-2105-12-77","article-title":"pROC: an open-source package for R and S+ to analyze and compare ROC curves","volume":"12","author":"X Robin","year":"2011","journal-title":"BMC Bioinformatics"},{"key":"pone.0260622.ref056","author":"SR Lele","year":"2019","journal-title":"ResourceSelection: Resource Selection (Probability) Functions for Use-Availability Data"},{"issue":"2","key":"pone.0260622.ref057","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","article-title":"Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization","volume":"128","author":"RR Selvaraju","year":"2019","journal-title":"International Journal of Computer Vision"},{"issue":"3","key":"pone.0260622.ref058","first-page":"1","article-title":"Multi-Label Classification: An Overview","volume":"3","author":"G Tsoumakas","year":"2007","journal-title":"IJDWM"},{"key":"pone.0260622.ref059","doi-asserted-by":"crossref","unstructured":"Huang R, Pedoeem J, Chen C. YOLO-LITE: A Real-Time Object Detection Algorithm Optimized for Non-GPU Computers. 2018 IEEE International Conference on Big Data (Big Data). 2018 Nov;2503\u201310.","DOI":"10.1109\/BigData.2018.8621865"},{"key":"pone.0260622.ref060","article-title":"Rich feature hierarchies for accurate object detection and semantic segmentation","author":"RB Girshick","year":"2013","journal-title":"CoRR"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0260622","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,2]],"date-time":"2021-12-02T14:19:07Z","timestamp":1638454747000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0260622"}},"subtitle":[],"editor":[{"given":"Yan Chai","family":"Hum","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,12,2]]},"references-count":60,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,12,2]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0260622","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.06.24.449695","asserted-by":"object"}]},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,2]]}}}