{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T14:26:47Z","timestamp":1776868007985,"version":"3.51.2"},"reference-count":10,"publisher":"Georg Thieme Verlag KG","issue":"03\/04","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Methods Inf Med"],"published-print":{"date-parts":[[2022,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>\n          Background\u2003Clinical procedures are often performed in outpatient clinics without prior scheduling at the administrative level, and documentation of the procedure often occurs solely in free-text clinical electronic notes. Natural language processing (NLP), particularly named entity recognition (NER), may provide a solution to extracting procedure data from free-text electronic notes.<\/jats:p><jats:p>\n          Methods\u2003Free-text notes from outpatient ophthalmology visits were collected from the electronic clinical records at a single institution over 3 months. The Prodigy low-code annotation tool was used to create an annotation dataset and train a custom NER model for clinical procedures. Clinical procedures were extracted from the entire set of clinical notes.<\/jats:p><jats:p>\n          Results\u2003There were a total of 5,098 clinic notes extracted for the study period; 1,923 clinic notes were used to build the NER model, which included a total of 231 manual annotations. The NER model achieved an F-score of 0.767, a precision of 0.810, and a recall of 0.729. The most common procedures performed included intravitreal injections of therapeutic substances, removal of corneal foreign bodies, and epithelial debridement of corneal ulcers.<\/jats:p><jats:p>\n          Conclusion\u2003The use of a low-code annotation software tool allows the rapid creation of a custom annotation dataset to train a NER model to identify clinical procedures stored in free-text electronic clinical notes. This enables clinicians to rapidly gather previously unidentified procedural data for quality improvement and auditing purposes. Low-code annotation tools may reduce time and coding barriers to clinician participation in NLP research.<\/jats:p>","DOI":"10.1055\/s-0042-1749358","type":"journal-article","created":{"date-parts":[[2022,9,12]],"date-time":"2022-09-12T22:48:05Z","timestamp":1663022885000},"page":"084-089","source":"Crossref","is-referenced-by-count":2,"title":["Automated Identification of Clinical Procedures in Free-Text Electronic Clinical Records with a Low-Code Named Entity Recognition Workflow"],"prefix":"10.1055","volume":"61","author":[{"given":"Carmelo","family":"Macri","sequence":"additional","affiliation":[{"name":"Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia"},{"name":"Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia"},{"name":"Discipline of Ophthalmology and Visual Sciences, The University of Adelaide, Adelaide, South Australia, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ian","family":"Teoh","sequence":"additional","affiliation":[{"name":"Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stephen","family":"Bacchi","sequence":"additional","affiliation":[{"name":"Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia"},{"name":"Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michelle","family":"Sun","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dinesh","family":"Selva","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert","family":"Casson","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"WengOnn","family":"Chan","sequence":"additional","affiliation":[{"name":"Machine Learning Division, Ophthalmic Research Laboratory, University of Adelaide, Adelaide, South Australia, Australia"},{"name":"Department of Ophthalmology, Royal Adelaide Hospital, Adelaide, South Australia, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"194","published-online":{"date-parts":[[2022,9,12]]},"reference":[{"issue":"03","key":"ref1","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1136\/jamia.2009.002733","article-title":"An overview of MetaMap: historical perspective and recent advances","volume":"17","author":"A R Aronson","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"01","key":"ref4","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1093\/bib\/bbz130","article-title":"An extensive review of tools for manual annotation of documents","volume":"22","author":"M Neves","year":"2021","journal-title":"Brief Bioinform"},{"issue":"02","key":"ref5","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1136\/bjophthalmol-2018-313173","article-title":"Artificial intelligence and deep learning in ophthalmology","volume":"103","author":"D SW Ting","year":"2019","journal-title":"Br J Ophthalmol"},{"issue":"02","key":"ref6","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1167\/tvst.9.2.13","article-title":"Applications of artificial intelligence to electronic health record data in ophthalmology","volume":"9","author":"W C Lin","year":"2020","journal-title":"Transl Vis Sci Technol"},{"issue":"04","key":"ref7","doi-asserted-by":"crossref","first-page":"328","DOI":"10.5301\/ejo.5000706","article-title":"Predictive modeling of risk factors and complications of cataract surgery","volume":"26","author":"G L Gaskin","year":"2016","journal-title":"Eur J Ophthalmol"},{"key":"ref8","doi-asserted-by":"crossref","first-page":"104007","DOI":"10.1016\/j.ijmedinf.2019.104007","article-title":"Automated extraction of ophthalmic surgery outcomes from the electronic health record","volume":"133","author":"S Y Wang","year":"2020","journal-title":"Int J Med Inform"},{"issue":"02","key":"ref9","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1167\/tvst.6.2.2","article-title":"Validation of the Total Visual Acuity Extraction Algorithm (TOVA) for automated extraction of visual acuity data from free text, unstructured clinical records","volume":"6","author":"D M Baughman","year":"2017","journal-title":"Transl Vis Sci Technol"},{"issue":"04","key":"ref10","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1002\/pds.4149","article-title":"Natural language processing to ascertain two key variables from operative reports in ophthalmology","volume":"26","author":"L Liu","year":"2017","journal-title":"Pharmacoepidemiol Drug Saf"},{"issue":"01","key":"ref11","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1136\/ebmental-2019-300134","article-title":"Natural language processing for structuring clinical text data on depression using UK-CRIS","volume":"23","author":"N Vaci","year":"2020","journal-title":"Evid Based Ment Health"},{"key":"ref12","doi-asserted-by":"crossref","first-page":"268","DOI":"10.3389\/fpsyt.2020.00268","article-title":"Identifying predictors of suicide in severe mental illness: a feasibility study of a clinical prediction rule (Oxford Mental Illness and Suicide Tool or OxMIS)","volume":"11","author":"M Senior","year":"2020","journal-title":"Front Psychiatry"}],"container-title":["Methods of Information in Medicine"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/s-0042-1749358.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,2]],"date-time":"2022-11-02T00:09:50Z","timestamp":1667347790000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/s-0042-1749358"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9]]},"references-count":10,"journal-issue":{"issue":"03\/04","published-online":{"date-parts":[[2022,11,1]]},"published-print":{"date-parts":[[2022,9]]}},"URL":"https:\/\/doi.org\/10.1055\/s-0042-1749358","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["0026-1270","2511-705X"],"issn-type":[{"value":"0026-1270","type":"print"},{"value":"2511-705X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9]]}}}