{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T06:15:40Z","timestamp":1788934540079,"version":"build-2803163510"},"reference-count":22,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2023,12,26]],"date-time":"2023-12-26T00:00:00Z","timestamp":1703548800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"},{"start":{"date-parts":[[2023,12,26]],"date-time":"2023-12-26T00:00:00Z","timestamp":1703548800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Global Spine Journal"],"published-print":{"date-parts":[[2025,3]]},"abstract":"<jats:sec>\n                    <jats:title>Study Design<\/jats:title>\n                    <jats:p>Retrospective cohort study.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>This study assessed the effectiveness of a popular large language model, ChatGPT-4, in predicting Current Procedural Terminology (CPT) codes from surgical operative notes. By employing a combination of prompt engineering, natural language processing (NLP), and machine learning techniques on standard operative notes, the study sought to enhance billing efficiency, optimize revenue collection, and reduce coding errors.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The model was given 3 different types of prompts for 50 surgical operative notes from 2 spine surgeons. The first trial was simply asking the model to generate CPT codes for a given OP note. The second trial included 3 OP notes and associated CPT codes to, and the third trial included a list of every possible CPT code in the dataset to prime the model. CPT codes generated by the model were compared to those generated by the billing department. Model evaluation was performed in the form of calculating the area under the ROC (AUROC), and area under precision-recall curves (AUPRC).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The trial that involved priming ChatGPT with a list of every possible CPT code performed the best, with an AUROC of .87 and an AUPRC of .67, and an AUROC of .81 and AUPRC of .76 when examining only the most common CPT codes.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>ChatGPT-4 can aid in automating CPT billing from orthopedic surgery operative notes, driving down healthcare expenditures and enhancing billing code precision as the model evolves and fine-tuning becomes available.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/21925682231224753","type":"journal-article","created":{"date-parts":[[2023,12,26]],"date-time":"2023-12-26T10:39:36Z","timestamp":1703587176000},"page":"1113-1120","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":34,"title":["Artificially Intelligent Billing in Spine Surgery: An Analysis of a Large Language Model"],"prefix":"10.1177","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8823-720X","authenticated-orcid":false,"given":"Bashar","family":"Zaidat","sequence":"first","affiliation":[{"name":"Icahn School of Medicine at Mount Sinai"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yash S.","family":"Lahoti","sequence":"additional","affiliation":[{"name":"Icahn School of Medicine at Mount Sinai"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7246-2269","authenticated-orcid":false,"given":"Alexander","family":"Yu","sequence":"additional","affiliation":[{"name":"Icahn School of Medicine at Mount Sinai"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kareem S.","family":"Mohamed","sequence":"additional","affiliation":[{"name":"Icahn School of Medicine at Mount Sinai"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samuel K.","family":"Cho","sequence":"additional","affiliation":[{"name":"Icahn School of Medicine at Mount Sinai"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun S.","family":"Kim","sequence":"additional","affiliation":[{"name":"Icahn School of Medicine at Mount Sinai"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2023,12,26]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jacr.2004.05.003"},{"key":"e_1_3_3_3_2","unstructured":"Tollen L Keating E Weil A. How Administrative Spending Contributes to Excess US Health Spending. Health Affairs; 2020. Published online February 20 2020. doi:10.1377\/forefront.20200218.375060. https:\/\/www.healthaffairs.org\/content\/forefront\/administrative-spending-contributes-excess-us-health-spending"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2019.13978"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.15265\/iy-2014-0020"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1177\/21925682211062831"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1177\/21925682231164935"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2017.19148"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1377\/hlthaff.24.6.1629"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1377\/hlthaff.28.4.w533"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1001\/amajethics.2020.221"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.4103\/jpi.jpi_52_21"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1377\/hlthaff.2010.0893"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.2478\/pralin-2018-0002"},{"key":"e_1_3_3_15_2","first-page":"3579","article-title":"N-gram counts and language models from the common crawl","volume":"4","author":"Buck C","year":"2014","unstructured":"Buck C, Heafield K, Van Ooyen B. N-gram counts and language models from the common crawl. Proc. Int. Conf. Lang. Resources Eval. 2014;4:3579-3584.","journal-title":"Proc. Int. Conf. Lang. Resources Eval"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2019.05.024"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvs.2009.07.065"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jamcollsurg.2007.02.074"},{"issue":"8","key":"e_1_3_3_19_2","first-page":"401","article-title":"Benefiting from the government CERT audits","volume":"98","author":"Hammon M","year":"2005","unstructured":"Hammon M, Hammon WE. Benefiting from the government CERT audits. J Oklahoma State Med Assoc. 2005;98(8):401-402. PMID: 16206870.","journal-title":"J Oklahoma State Med Assoc"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40199-018-0227-z"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.4258\/hir.2022.28.1.16"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1097\/aln.0000000000003150"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.121186"}],"container-title":["Global Spine Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/21925682231224753","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/21925682231224753","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/21925682231224753","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T07:20:24Z","timestamp":1777447224000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/21925682231224753"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,26]]},"references-count":22,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,3]]}},"alternative-id":["10.1177\/21925682231224753"],"URL":"https:\/\/doi.org\/10.1177\/21925682231224753","relation":{},"ISSN":["2192-5682","2192-5690"],"issn-type":[{"value":"2192-5682","type":"print"},{"value":"2192-5690","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,26]]}}}