{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T00:58:14Z","timestamp":1791593894478,"version":"4.3.4"},"reference-count":30,"publisher":"SAGE Publications","issue":"5","license":[{"start":{"date-parts":[[2023,6,9]],"date-time":"2023-06-09T00:00:00Z","timestamp":1686268800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Parkinson\u2019s Disease"],"published-print":{"date-parts":[[2023,6,16]]},"abstract":"<jats:sec>\n                    <jats:title>Background:<\/jats:title>\n                    <jats:p>Free-text, verbatim replies in the words of people with Parkinson\u2019s disease (PD) have the potential to provide unvarnished information about their feelings and experiences. Challenges of processing such data on a large scale are a barrier to analyzing verbatim data collection in large cohorts.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective:<\/jats:title>\n                    <jats:p>To develop a method for curating responses from the Parkinson\u2019s Disease Patient Report of Problems (PD-PROP), open-ended questions that asks people with PD to report their most bothersome problems and associated functional consequences.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods:<\/jats:title>\n                    <jats:p>Human curation, natural language processing, and machine learning were used to develop an algorithm to convert verbatim responses to classified symptoms. Nine curators including clinicians, people with PD, and a non-clinician PD expert classified a sample of responses as reporting each symptom or not. Responses to the PD-PROP were collected within the Fox Insight cohort study.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>Approximately 3,500 PD-PROP responses were curated by a human team. Subsequently, approximately 1,500 responses were used in the validation phase; median age of respondents was 67 years, 55% were men and median years since PD diagnosis was 3 years. 168,260 verbatim responses were classified by machine. Accuracy of machine classification was 95% on a held-out test set. 65 symptoms were grouped into 14 domains. The most frequently reported symptoms at first report were tremor (by 46% of respondents), gait and balance problems (&gt;39%), and pain\/discomfort (33%).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion:<\/jats:title>\n                    <jats:p>A human-in-the-loop method of curation provides both accuracy and efficiency, permitting a clinically useful analysis of large datasets of verbatim reports about the problems that bother PD patients.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3233\/jpd-225083","type":"journal-article","created":{"date-parts":[[2023,6,13]],"date-time":"2023-06-13T10:30:54Z","timestamp":1686652254000},"page":"757-767","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":22,"title":["What Patients Say: Large-Scale Analyses of Replies to the Parkinson\u2019s Disease Patient Report of Problems (PD-PROP)"],"prefix":"10.1177","volume":"13","author":[{"given":"Connie","family":"Marras","sequence":"first","affiliation":[{"name":"Edmond J Safra Program in Parkinson\u2019s Disease, University Health Network, University of Toronto, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lakshmi","family":"Arbatti","sequence":"additional","affiliation":[{"name":"Grey Matter Technologies, a Wholly Owned Subsidiary of Modality.ai, San Francisco, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abhishek","family":"Hosamath","sequence":"additional","affiliation":[{"name":"Grey Matter Technologies, a Wholly Owned Subsidiary of Modality.ai, San Francisco, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amy","family":"Amara","sequence":"additional","affiliation":[{"name":"Department of Neurology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karen E.","family":"Anderson","sequence":"additional","affiliation":[{"name":"Departments of Psychiatry and Neurology, Georgetown University, Washington DC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lana M.","family":"Chahine","sequence":"additional","affiliation":[{"name":"Department of Neurology, University of Pittsburgh, Pittsburgh, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shirley","family":"Eberly","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Computational Biology, University of Rochester, Rochester, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dan","family":"Kinel","sequence":"additional","affiliation":[{"name":"Department of Neurology, University of Rochester, Rochester NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sneha","family":"Mantri","sequence":"additional","affiliation":[{"name":"Department of Neurology, Duke University, Durham, NC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Soania","family":"Mathur","sequence":"additional","affiliation":[{"name":"PD Avengers, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Oakes","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Computational Biology, University of Rochester, Rochester, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jennifer L.","family":"Purks","sequence":"additional","affiliation":[{"name":"Department of Neurology, University of Rochester, Rochester NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David G.","family":"Standaert","sequence":"additional","affiliation":[{"name":"PD Avengers, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Caroline M.","family":"Tanner","sequence":"additional","affiliation":[{"name":"Department of Neurology, Weill Institute for Neurosciences, University of California \u2013 San Francisco, San Francisco, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Weintraub","sequence":"additional","affiliation":[{"name":"Departments of Psychiatry and Neurology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ira","family":"Shoulson","sequence":"additional","affiliation":[{"name":"Grey Matter Technologies, a Wholly Owned Subsidiary of Modality.ai, San Francisco, CA, USA"},{"name":"Department of Neurology, University of Rochester, Rochester NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2023,6,9]]},"reference":[{"key":"ref001","doi-asserted-by":"publisher","DOI":"10.1186\/s41687-022-00444-z"},{"key":"ref002","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-020-0401-2"},{"key":"ref003","doi-asserted-by":"publisher","DOI":"10.1186\/s13326-022-00272-6"},{"key":"ref004","doi-asserted-by":"publisher","DOI":"10.1055\/s-0040-1708049"},{"key":"ref005","doi-asserted-by":"publisher","DOI":"10.3233\/JPD-212636"},{"key":"ref006","first-page":"993","volume":"3","author":"Blei DM","year":"2003","journal-title":"J Mach Learn Res"},{"key":"ref007","doi-asserted-by":"crossref","unstructured":"XuJ, TaoY, LinH (2016) Semantic word cloud generation based on word embeddings. IEEE Pacific Visualization Symposium (Pacific Vis), 239-243.","DOI":"10.1109\/PACIFICVIS.2016.7465278"},{"key":"ref008","doi-asserted-by":"publisher","DOI":"10.1109\/PACIFICVIS.2010.5429600"},{"key":"ref009","doi-asserted-by":"publisher","DOI":"10.3233\/JPD-223274"},{"key":"ref010","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkh061"},{"key":"ref011","unstructured":"National Library of Medicine, Unified Medical Language System, https:\/\/www.nlm.nih.gov\/research\/umls\/index.html, Accessed January 11, 2023."},{"key":"ref012","unstructured":"MikolovT, ChenK, CorradoG, DeanJ (2013) Efficient estimation of word representations in vector space. arXiv, arXiv:1301.3781."},{"key":"ref013","doi-asserted-by":"crossref","unstructured":"ChiuB, CrichtonG, KorhonenA, PyysaloS (2016) How to Train good Word Embeddings for Biomedical NLP. Proceedings of the 15th Workshop on Biomedical Natural Language Processing, Berlin, Germany, pp. 166-174.","DOI":"10.18653\/v1\/W16-2922"},{"key":"ref014","doi-asserted-by":"crossref","unstructured":"ChenQ, PengY, LuZ (2019) BioSentVec: Creating sentence embeddings for biomedical texts. 2019 IEEE International Conference on Healthcare Informatics (ICHI), pp. 1-5.","DOI":"10.1109\/ICHI.2019.8904728"},{"key":"ref015","first-page":"163","volume":"2017","author":"Sabbir A","year":"2017","journal-title":"Proc IEEE Int Symp Bioinformatics Bioeng"},{"key":"ref016","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330773"},{"key":"ref017","unstructured":"WuX, XiaoL, SunY, ZhangJ, MaT, HeL (2021) A survey of human-in-the-loop for machine learning. ArXiv210800941 Cs."},{"key":"ref018","unstructured":"Java Platform, Package java.lang, https:\/\/docs.oracle.com\/javase\/7\/docs\/api\/java\/lang\/package-summary.html, Accessed February 24, 2023."},{"key":"ref019","unstructured":"NorvigP, How to Write a Spelling Corrector, https:\/\/norvig.com\/spell-correct.html, February 24, 2023."},{"key":"ref020","unstructured":"Neo4j, Neo4j Graph Database Platform. https:\/\/neo4j.com, Accessed May 6, 2020."},{"key":"ref021","unstructured":"Neo4j, Full-text search index - Cypher Manual, Neo4j Graph Data Platform. https:\/\/neo4j.com\/docs\/cypher-manual\/5\/indexes-for-full-text-search, February 28, 2023."},{"key":"ref022","unstructured":"Keras, Keras documentation: Large-scale multi-label text classification. https:\/\/keras.io\/examples\/nlp\/multi_label_classification, Accessed January 12, 2023."},{"key":"ref023","doi-asserted-by":"crossref","unstructured":"HasanMM, DipST, RahmanT, AkterMS, SalehinI (2021) Multilabel movie genre classification from movie subtitle: Parameter optimized hybrid classifier. 2021 4th International Symposium on Advanced Electrical and Communication Technologies (ISAECT), pp. 1-6.","DOI":"10.1109\/ISAECT53699.2021.9668427"},{"key":"ref024","unstructured":"Keras, Keras: The Python deep learning API, https:\/\/keras.io. February 28, 2023."},{"key":"ref025","unstructured":"scikit learn, sklearn.model selection.train test split, scikitlearn, https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.model_selection.train_test_split.html, Accessed February 28, 2023."},{"key":"ref026","doi-asserted-by":"publisher","DOI":"10.3233\/JPD-202346"},{"key":"ref027","doi-asserted-by":"publisher","DOI":"10.3390\/informatics8030056"},{"key":"ref028","doi-asserted-by":"publisher","DOI":"10.1177\/2374373519864011"},{"key":"ref029","doi-asserted-by":"publisher","DOI":"10.1186\/s12955-020-01438-5"},{"key":"ref030","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocy169"}],"container-title":["Journal of Parkinson\u2019s Disease"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JPD-225083","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.3233\/JPD-225083","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JPD-225083","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T04:52:31Z","timestamp":1777611151000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.3233\/JPD-225083"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,9]]},"references-count":30,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,6,16]]}},"alternative-id":["10.3233\/JPD-225083"],"URL":"https:\/\/doi.org\/10.3233\/jpd-225083","relation":{},"ISSN":["1877-7171","1877-718X"],"issn-type":[{"value":"1877-7171","type":"print"},{"value":"1877-718X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,9]]}}}