{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T16:40:21Z","timestamp":1791996021488,"version":"4.4.0"},"reference-count":54,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Cancer"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Artificial intelligence (AI) is increasingly used to generate medical content, yet its performance in delivering clinically relevant and reliable information remains underexplored, especially in complex areas such as breast cancer.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to compare ChatGPT-4.0 and DeepSeek-V3 in generating breast cancer information, focusing on readability, content quality, and citation reliability.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n On the basis of publicly available patient education materials, 10 frequently asked questions were selected. Each model generated 60 responses. Three expert reviewers rated each response using a 7-point Likert scale across 5 dimensions (ie, accuracy, completeness, clarity, depth and insight, and alignment with expert answers). Readability was assessed using Flesch-Kincaid Grade Level scores. Information reliability was evaluated through interrater agreement metrics, including Cohen \u03ba and Fleiss \u03ba. Paired\n t<\/jats:italic>\n tests were used for statistical comparisons.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n AI models produced significantly more readable content than expert references (mean Flesch-Kincaid Grade Level difference \u22122.60;\n P<\/jats:italic>\n <.001). ChatGPT-4.0 responses were more stylistically consistent with a median Flesch-Kincaid Grade Level score of 10.66 (IQR 0.98), whereas DeepSeek-V3 showed greater variability with a median Flesch-Kincaid Grade Level score of 10.17 (IQR 1.41). Content quality scores were DeepSeek-V3 achieving a higher mean score than ChatGPT-4.0 (6.22 [SD 0.43] vs 6.01 [SD 0.49]). In the multiresponse analysis, DeepSeek-V3 demonstrated a statistically significant advantage in accuracy (\n P<\/jats:italic>\n =.041), while differences across other criteria were not statistically significant (\n P<\/jats:italic>\n >.05). Human raters showed almost perfect agreement when judging source reliability (Fleiss \u03ba=0.842 for ChatGPT\u2019s citations and 0.935 for DeepSeek\u2019s citations). Agreement between each model\u2019s citation reliability scores and the expert majority was substantial for ChatGPT (Cohen \u03ba=0.665) and higher for DeepSeek (Cohen \u03ba=0.782).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Both models generated readable and clinically relevant content with comparable overall performance. ChatGPT provided more consistent readability, while DeepSeek offered more diverse references with stronger alignment to expert ratings. Continued evaluation and quality assurance are essential for the responsible clinical use of AI-generated content.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/72839","type":"journal-article","created":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T22:05:08Z","timestamp":1763589908000},"page":"e72839-e72839","source":"Crossref","is-referenced-by-count":4,"title":["ChatGPT Versus DeepSeek for Breast Cancer Information Retrieval: Quantitative Comparative Study"],"prefix":"10.2196","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7090-5425","authenticated-orcid":false,"given":"Rima","family":"Hajjo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1428-5097","authenticated-orcid":false,"given":"Dima A","family":"Sabbah","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4823-0708","authenticated-orcid":false,"given":"Sanaa K","family":"Bardaweel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,2,27]]},"reference":[{"issue":"3","key":"R1","doi-asserted-by":"publisher","first-page":"209","DOI":"10.3322\/caac.21660","article-title":"Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"71","author":"Sung","journal-title":"CA Cancer J Clin"},{"issue":"1","key":"R2","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1007\/s42979-023-02534-1","article-title":"Challenges to the early diagnosis of breast cancer: current scenario and the challenges ahead","volume":"5","author":"Sinha","journal-title":"SN COMPUT SCI"},{"key":"R3","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijpharm.2023.123392","article-title":"Identifying synergistic combinations of doxorubicin-loaded polyquercetin nanoparticles and natural products: implications for breast cancer therapy","volume":"645","author":"Sunoqrot","journal-title":"Int J Pharm"},{"issue":"32","key":"R4","doi-asserted-by":"publisher","DOI":"10.1002\/slct.202202263","article-title":"Novel derivatives of 4,6\u2010dihydroxy\u20102\u2010quinolone\u20103\u2010carboxamides as potential PI3K\u03b1 inhibitors","volume":"7","author":"Sweidan","journal-title":"ChemistrySelect"},{"key":"R5","unstructured":"Global Breast Cancer Initiative Implementation Framework: Assessing, Strengthening and Scaling-up of Services for the Early Detection and Management of Breast Cancer. World Health Organization; 2023. URL: https:\/\/www.who.int\/publications\/i\/item\/9789240067134 [Accessed 14-01-2026] ISBN: 978-92-4-006713-4"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","DOI":"10.1186\/s12889-023-17252-9","article-title":"Effect of neighborhood and individual-level socioeconomic factors on breast cancer screening adherence in a multiethnic study","volume":"24","author":"Kasper","journal-title":"BMC Public Health"},{"issue":"1","key":"R7","doi-asserted-by":"publisher","DOI":"10.1186\/s12905-024-03509-x","article-title":"Online health information seeking behavior among breast cancer patients and survivors: a scoping review","volume":"25","author":"Chen","journal-title":"BMC Womens Health"},{"issue":"5","key":"R8","doi-asserted-by":"publisher","first-page":"453","DOI":"10.3322\/caac.21857","article-title":"Cancer misinformation on social media","volume":"74","author":"Loeb","journal-title":"CA Cancer J Clin"},{"issue":"7","key":"R9","doi-asserted-by":"publisher","first-page":"1036","DOI":"10.1093\/jnci\/djab141","article-title":"Cancer misinformation and harmful information on Facebook and other social media: a brief report","volume":"114","author":"Johnson","journal-title":"J Natl Cancer Inst"},{"key":"R10","doi-asserted-by":"publisher","DOI":"10.2196\/47479","article-title":"Reliability of medical information provided by ChatGPT: assessment against clinical guidelines and patient information quality instrument","volume":"25","author":"Walker","journal-title":"J Med Internet Res"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2023.1169595","article-title":"ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations","volume":"6","author":"Dave","journal-title":"Front Artif Intell"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","first-page":"1542","DOI":"10.1038\/s41598-024-51531-8","article-title":"Availability of ChatGPT to provide medical information for patients with kidney cancer","volume":"14","author":"Choi","journal-title":"Sci Rep"},{"key":"R13","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2025.104791","article-title":"From GPT to DeepSeek: significant gaps remain in realizing AI in health care","volume":"163","author":"Peng","journal-title":"J Biomed Inform"},{"issue":"2","key":"R14","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1007\/BF02904998","article-title":"Electronic authoring and delivery of technical information","volume":"11","author":"Kincaid","journal-title":"J Instr Dev"},{"issue":"4","key":"R15","doi-asserted-by":"publisher","first-page":"541","DOI":"10.4300\/JGME-5-4-18","article-title":"Analyzing and interpreting data from likert-type scales","volume":"5","author":"Sullivan","journal-title":"J Grad Med Educ"},{"issue":"23","key":"R16","doi-asserted-by":"publisher","first-page":"63479","DOI":"10.1007\/s11042-023-18056-0","article-title":"An intelligent health care monitoring system-based novel deep learning approach for detecting covid-19 from x-rays images","volume":"83","author":"AlZu\u2019bi","journal-title":"Multimed Tools Appl"},{"issue":"11","key":"R17","doi-asserted-by":"publisher","first-page":"28","DOI":"10.14445\/23488549\/IJECE-V11I11P104","article-title":"Leveraging machine learning for predictive pathways in higher education: a case study at Al-Zaytoonah University of Jordan","volume":"11","author":"Muhairat","journal-title":"SSRG-IJECE"},{"issue":"3","key":"R18","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1080\/09603123.2024.2363472","article-title":"Artificial intelligence utilization in the health care setting: perceptions of the public in the UAE","volume":"35","author":"Jarab","journal-title":"Int J Environ Health Res"},{"issue":"668","key":"R19","doi-asserted-by":"publisher","first-page":"143","DOI":"10.3399\/bjgp18X695213","article-title":"Artificial intelligence in medicine: current trends and future possibilities","volume":"68","author":"Buch","journal-title":"Br J Gen Pract"},{"issue":"1","key":"R20","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1038\/s41591-021-01614-0","article-title":"AI in health and medicine","volume":"28","author":"Rajpurkar","journal-title":"Nat Med"},{"issue":"1","key":"R21","doi-asserted-by":"publisher","DOI":"10.1007\/s11886-013-0441-8","article-title":"Artificial intelligence in medicine and cardiac imaging: harnessing big data and advanced computing to provide personalized medical diagnosis and treatment","volume":"16","author":"Dilsizian","journal-title":"Curr Cardiol Rep"},{"issue":"7","key":"R22","doi-asserted-by":"publisher","first-page":"2328","DOI":"10.4103\/jfmpc.jfmpc_440_19","article-title":"Overview of artificial intelligence in medicine","volume":"8","author":"Malik","journal-title":"J Family Med Prim Care"},{"issue":"1","key":"R23","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/bs.pmbts.2022.03.002","article-title":"Artificial intelligence and machine learning in precision medicine: a paradigm shift in big data analysis","volume":"190","author":"Sahu","journal-title":"Prog Mol Biol Transl Sci"},{"issue":"7","key":"R24","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics12071742","article-title":"Unlocking the potential of the human microbiome for identifying disease diagnostic biomarkers","volume":"12","author":"Hajjo","journal-title":"Diagnostics (Basel)"},{"issue":"1","key":"R25","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1097\/ACM.0000000000005439","article-title":"ChatGPT and generative artificial intelligence for medical education: potential impact and opportunity","volume":"99","author":"Boscardin","journal-title":"Acad Med"},{"issue":"5","key":"R26","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics11050742","article-title":"Identification of tumor-specific MRI biomarkers using machine learning (ML)","volume":"11","author":"Hajjo","journal-title":"Diagnostics (Basel)"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.2196\/63649","article-title":"Facilitators and barriers to implementing AI in routine medical imaging: systematic review and qualitative analysis","volume":"27","author":"Wenderott","journal-title":"J Med Internet Res"},{"key":"R28","doi-asserted-by":"publisher","DOI":"10.2196\/71982","article-title":"Key elements and theoretical foundations for the design and delivery of text messages to boost medication adherence in patients with diabetes, hypertension, and hyperlipidemia: scoping review","volume":"27","author":"Yang","journal-title":"J Med Internet Res"},{"key":"R29","doi-asserted-by":"publisher","DOI":"10.2196\/64685","article-title":"Predicting pathological complete response following neoadjuvant therapy in patients with breast cancer: development of machine learning-based prediction models in a retrospective study","volume":"11","author":"Lai","journal-title":"JMIR Cancer"},{"key":"R30","doi-asserted-by":"publisher","DOI":"10.2196\/66342","article-title":"Efficacy of Fuzheng Quxie formula against postoperative metastasis of lung cancer in stage IIA-IIIA with negative driver genes: protocol for a multicenter, double-blind, randomized controlled trial","volume":"14","author":"Xu","journal-title":"JMIR Res Protoc"},{"issue":"1","key":"R31","doi-asserted-by":"publisher","DOI":"10.1186\/s12905-024-03320-8","article-title":"An assessment of ChatGPT\u2019s responses to frequently asked questions about cervical and breast cancer","volume":"24","author":"Ye","journal-title":"BMC Womens Health"},{"key":"R32","unstructured":"A shocking chinese AI advancement called deepseek is sending US stocks plunging. CNN Business. URL: https:\/\/edition.cnn.com\/2025\/01\/27\/tech\/deepseek-stocks-ai-china\/index.html [Accessed 13-12-2025]"},{"key":"R33","unstructured":"Korea biomedical review. The DeepSeek dilemma: navigating innovation and security concerns in Korea\u2019s health care industry. URL: https:\/\/www.koreabiomed.com\/news\/articleView.html?idxno=26581 [Accessed 13-02-2025]"},{"issue":"3","key":"R34","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1037\/h0057532","article-title":"A new readability yardstick","volume":"32","author":"FLESCH","journal-title":"J Appl Psychol"},{"issue":"1","key":"R35","doi-asserted-by":"publisher","first-page":"199","DOI":"10.2106\/JBJS.G.00347","article-title":"Readability of patient education materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America web sites","volume":"90","author":"Badarudeen","journal-title":"J Bone Joint Surg Am"},{"key":"R36","unstructured":"Textstat. Python Package Index. URL: https:\/\/pypi.org\/project\/textstat\/ [Accessed 13-12-2025]"},{"issue":"3","key":"R37","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: fundamental algorithms for scientific computing in Python","volume":"17","author":"Virtanen","journal-title":"Nat Methods"},{"issue":"2","key":"R38","doi-asserted-by":"publisher","DOI":"10.1093\/jncics\/pkad015","article-title":"Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information","volume":"7","author":"Johnson","journal-title":"JNCI Cancer Spectr"},{"issue":"4","key":"R39","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230424","article-title":"Appropriateness of breast cancer prevention and screening recommendations provided by ChatGPT","volume":"307","author":"Haver","journal-title":"Radiology"},{"issue":"3\/4","key":"R40","doi-asserted-by":"publisher","DOI":"10.2307\/2333709","article-title":"An analysis of variance test for normality (complete samples)","volume":"52","author":"Shapiro","journal-title":"Biometrika"},{"issue":"7","key":"R41","doi-asserted-by":"publisher","first-page":"788","DOI":"10.3390\/math9070788","article-title":"An exhaustive power comparison of normality tests","volume":"9","author":"Arnastauskait\u0117","journal-title":"Mathematics"},{"issue":"4","key":"R42","doi-asserted-by":"publisher","first-page":"688","DOI":"10.1093\/beheco\/ark016","article-title":"The unequal variance t-test is an underused alternative to Student\u2019s t-test and the Mann\u2013Whitney U test","volume":"17","author":"Ruxton","journal-title":"Behav Ecol"},{"issue":"5","key":"R43","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1037\/h0031619","article-title":"Measuring nominal scale agreement among many raters","volume":"76","author":"Fleiss","journal-title":"Psychol Bull"},{"issue":"1","key":"R44","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","author":"Cohen","journal-title":"Educ Psychol Meas"},{"issue":"7825","key":"R45","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1038\/s41586-020-2649-2","article-title":"Array programming with NumPy","volume":"585","author":"Harris","journal-title":"Nature New Biol"},{"key":"R46","unstructured":"Scikit-learn: machine learning in python \u2014 scikit-learn 170 documentation. Scikit-learn. URL: https:\/\/scikit-learn.org\/stable\/ [Accessed 13-12-2025]"},{"key":"R47","unstructured":"Python data analysis library. pandas. URL: https:\/\/pandas.pydata.org\/ [Accessed 13-12-2025]"},{"issue":"1","key":"R48","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1159\/000492428","article-title":"Societal issues concerning the application of artificial intelligence in medicine","volume":"5","author":"Vellido","journal-title":"Kidney Dis (Basel)"},{"issue":"13","key":"R49","doi-asserted-by":"publisher","first-page":"7479","DOI":"10.3390\/app13137479","article-title":"Recent advances of artificial intelligence in health care: a systematic literature review","volume":"13","author":"Kitsios","journal-title":"Appl Sci (Basel)"},{"issue":"13","key":"R50","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1056\/NEJMsr2214184","article-title":"Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine","volume":"388","author":"Lee","journal-title":"N Engl J Med"},{"issue":"1-2","key":"R51","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1080\/13543776.2024.2327307","article-title":"Targeting the EGFR\/RAS\/RAF signaling pathway in anticancer research: a recent update on inhibitor design and clinical trials (2020-2023)","volume":"34","author":"Hajjo","journal-title":"Expert Opin Ther Pat"},{"issue":"3","key":"R52","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1080\/13543776.2024.2338100","article-title":"Targeting the PI3K\/AKT signaling pathway in anticancer research: a recent update on inhibitor design and clinical trials (2020-2023)","volume":"34","author":"Sabbah","journal-title":"Expert Opin Ther Pat"},{"key":"R53","doi-asserted-by":"publisher","DOI":"10.1016\/j.greeac.2025.100268","article-title":"From discovery to delivery: governance of AI in the pharmaceutical industry","volume":"13","author":"Pasas-Farmer","journal-title":"Green Analytical Chemistry"},{"key":"R54","unstructured":"Hajjo R . AI-models [github repository]. GitHub. URL: https:\/\/github.com\/rhajjo\/AI-Models [Accessed 04-01-2026]"}],"container-title":["JMIR Cancer"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/cancer.jmir.org\/2026\/1\/e72839","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:03:17Z","timestamp":1782842597000},"score":1,"resource":{"primary":{"URL":"https:\/\/cancer.jmir.org\/2026\/1\/e72839"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,27]]},"references-count":54,"URL":"https:\/\/doi.org\/10.2196\/72839","relation":{},"ISSN":["2369-1999"],"issn-type":[{"value":"2369-1999","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,27]]},"article-number":"v12i3e72839"}}