{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,7]],"date-time":"2026-09-07T06:08:19Z","timestamp":1788761299520,"version":"build-2803163510"},"reference-count":49,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,7]],"date-time":"2026-09-07T00:00:00Z","timestamp":1788739200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:p>Screening for text-implied structural gaps in policy documents is an important component of public health policy review, particularly when implementation relies on online portals, digital identity verification, remote-care platforms, or self-service processes. However, existing text classifiers, large language model prompting methods, and retrieval-based approaches often provide document-level predictions or general explanations without explicitly representing key elements, relations, and potentially missing links in policy implementation. This study proposes Policy Review Process-Knowledge Graph (PRP-KG), a large language model-enhanced knowledge graph framework for identifying structural-gap review signals across policy clauses, implementation processes, and target populations. PRP-KG segments policy documents into clauses, extracts implementation-related elements using a predefined policy-execution schema, and grounds the extracted entities and relations in source evidence spans. The validated elements are then organized into a three-layer knowledge graph. A graph-consistency feedback mechanism revises missing or schema-inconsistent triples, after which structural-gap patterns identify text-implied gaps in execution specification, accessibility safeguards, and service-support coverage. Experiments on policy-text annotation benchmarks constructed from public sources show that, under the evaluated settings, PRP-KG achieves lower prediction error and more accurate identification of high-priority text-implied review signals than most comparison methods. Controlled analyses indicate comparatively stable performance under the evaluated perturbations and provide evidence-linked outputs that can be inspected by policy reviewers.<\/jats:p>","DOI":"10.3389\/fpubh.2026.1937335","type":"journal-article","created":{"date-parts":[[2026,9,7]],"date-time":"2026-09-07T05:33:58Z","timestamp":1788759238000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["A large language model-enhanced knowledge graph framework for text-implied public health policy gap screening: digital health executability, behavioral accessibility, and service-support coverage"],"prefix":"10.3389","volume":"14","author":[{"given":"Xinyi","family":"Wang","sequence":"first","affiliation":[{"name":"Carey Business School, Johns Hopkins University","place":["Baltimore, MD, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiao","family":"Lu","sequence":"additional","affiliation":[{"name":"School of Public Policy and Administration, Xi'an Jiaotong University","place":["Xi'an, Shaanxi, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,7]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"932","DOI":"10.1093\/jamia\/ocaf017","article-title":"Digital health equity frameworks and key concepts: a scoping review","volume":"32","author":"Kim","year":"2025","journal-title":"J Am Med Inform Assoc."},{"key":"B2","doi-asserted-by":"publisher","first-page":"e0000509","DOI":"10.1371\/journal.pdig.0000509","article-title":"Construction of the digital health equity-focused implementation research conceptual model-bridging the divide between equity-focused digital health and implementation research","volume":"3","author":"Groom","year":"2024","journal-title":"PLOS Digit Health."},{"key":"B3","doi-asserted-by":"publisher","first-page":"1690117","DOI":"10.3389\/fpubh.2025.1690117","article-title":"Evaluation of health equity frameworks in telehealth and digital health: a systematic review and narrative synthesis","volume":"13","author":"Wang","year":"2025","journal-title":"Front Public Health."},{"key":"B4","doi-asserted-by":"publisher","first-page":"1977","DOI":"10.1057\/s41599-025-05924-3","article-title":"Large language models in legal systems: a survey","volume":"12","author":"Dehghani","year":"2025","journal-title":"Humanit Soc Sci Commun."},{"key":"B5","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1093\/jla\/laae003","article-title":"Large legal fictions: profiling legal hallucinations in large language models","volume":"16","author":"Dahl","year":"2024","journal-title":"J Legal Anal."},{"key":"B6","doi-asserted-by":"publisher","first-page":"5364","DOI":"10.18653\/v1\/2024.naacl-long.300","article-title":"GLiNER: generalist model for named entity recognition using bidirectional transformer","author":"Zaratiana","year":"2024","journal-title":"Proceedings of the 2024 conference of the North American chapter of the association for computational linguistics: human language technologies, Vol 1 (long papers)."},{"key":"B7","doi-asserted-by":"publisher","first-page":"549","DOI":"10.3390\/info16070549","article-title":"Large language models in medical chatbots: opportunities, challenges, and the need to address AI risks","volume":"16","author":"Chow","year":"2025","journal-title":"Information"},{"key":"B8","doi-asserted-by":"publisher","first-page":"104082","DOI":"10.1016\/j.ipm.2025.104082","article-title":"An LLM-assisted ETL pipeline to build a high-quality knowledge graph of the Italian legislation","volume":"62","author":"Colombo","year":"2025","journal-title":"Inf Process Manag."},{"key":"B9","doi-asserted-by":"publisher","first-page":"837","DOI":"10.3390\/biomedinformatics4010047","article-title":"Generative pre-trained transformer-empowered healthcare conversations: current trends, challenges, and future directions in large language model-enabled medical chatbots","volume":"4","author":"Chow","year":"2024","journal-title":"BioMedInformatics."},{"key":"B10","doi-asserted-by":"publisher","first-page":"8622","DOI":"10.1109\/TKDE.2024.3469578","article-title":"Large language models on graphs: a comprehensive survey","volume":"36","author":"Jin","year":"2024","journal-title":"IEEE Trans Knowl Data Eng."},{"key":"B11","doi-asserted-by":"publisher","first-page":"132876","DOI":"10.52202\/079017-4224","article-title":"G-retriever: retrieval-augmented generation for textual graph understanding and question answering","volume":"37","author":"He","year":"2024","journal-title":"Adv Neural Inf Process Syst."},{"key":"B12","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1038\/s41746-024-01177-7","article-title":"Recommendations to advance digital health equity: a systematic review of qualitative studies","volume":"7","author":"Wilson","year":"2024","journal-title":"NPJ Digit Med."},{"key":"B13","doi-asserted-by":"publisher","first-page":"e70856","DOI":"10.2196\/70856","article-title":"Digital health technology infrastructure challenges to support health equity in the United States: scoping review","volume":"27","author":"Roy","year":"2025","journal-title":"J Med Internet Res."},{"key":"B14","doi-asserted-by":"publisher","first-page":"44123","DOI":"10.52202\/075280-1915","article-title":"Legalbench: a collaboratively built benchmark for measuring legal reasoning in large language models","volume":"36","author":"Guha","year":"2023","journal-title":"Adv Neural Inf Process Syst."},{"key":"B15","doi-asserted-by":"publisher","first-page":"7933","DOI":"10.18653\/v1\/2024.emnlp-main.452","article-title":"Lawbench: benchmarking legal knowledge of large language models","author":"Fei","year":"2024","journal-title":"Proceedings of the 2024 conference on empirical methods in natural language processing."},{"key":"B16","doi-asserted-by":"publisher","first-page":"e441","DOI":"10.1016\/S2589-7500(24)00111-0","article-title":"From text to treatment: the crucial role of validation for generative large language models in health care","volume":"6","author":"de Hond","year":"2024","journal-title":"Lancet Digit Health."},{"key":"B17","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","year":"2024","journal-title":"NPJ Digit Med."},{"key":"B18","doi-asserted-by":"publisher","first-page":"52","DOI":"10.18653\/v1\/2024.findings-emnlp.4","article-title":"General collaborative framework between large language model and experts for universal information extraction","author":"Bao","year":"2024","journal-title":"Findings of the association for computational linguistics: EMNLP 2024"},{"key":"B19","doi-asserted-by":"publisher","first-page":"3146","DOI":"10.63317\/4zus5294j5vi","article-title":"ChatUIE: exploring chat-based unified information extraction using large language models","author":"Xu","year":"2024","journal-title":"Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (LREC-COLING 2024)"},{"key":"B20","first-page":"9112","article-title":"Self-rag: learning to retrieve, generate, and critique through self-reflection","volume-title":"International conference on learning representations, Vol. 2024","author":"Asai","year":"2024"},{"key":"B21","doi-asserted-by":"publisher","first-page":"10862","DOI":"10.18653\/v1\/2024.acl-long.585","article-title":"Ragtruth: a hallucination corpus for developing trustworthy retrieval-augmented language models","author":"Niu","year":"2024","journal-title":"Proceedings of the 62nd annual meeting of the association for computational linguistics, Vol. 1 (long papers)"},{"key":"B22","doi-asserted-by":"publisher","first-page":"881","DOI":"10.18653\/v1\/2024.emnlp-industry.66","article-title":"Retrieval augmented generation or long-context llms?","author":"Li","year":"2024"},{"key":"B23","doi-asserted-by":"publisher","first-page":"10371","DOI":"10.18653\/v1\/2024.findings-emnlp.607","article-title":"Retrieving, rethinking and revising: the chain-of-verification can improve retrieval augmented generation","author":"He","year":"2024","journal-title":"Findings of the association for computational linguistics: EMNLP 2024"},{"key":"B24","doi-asserted-by":"publisher","first-page":"12758","DOI":"10.18653\/v1\/2024.findings-emnlp.746","article-title":"Graphreader: building graph-based agent to enhance long-context abilities of large language models","author":"Li","year":"2024","journal-title":"Findings of the association for computational linguistics: EMNLP 2024"},{"key":"B25","doi-asserted-by":"publisher","first-page":"59532","DOI":"10.52202\/079017-1902","article-title":"Hipporag: neurobiologically inspired long-term memory for large language models","volume":"37","author":"Guti\u00e9rrez","year":"2024","journal-title":"Adv Neural Inf Process Syst."},{"key":"B26","doi-asserted-by":"publisher","first-page":"666","DOI":"10.3390\/info15110666","article-title":"Construction of legal knowledge graph based on knowledge-enhanced large language models","volume":"15","author":"Li","year":"2024","journal-title":"Information."},{"key":"B27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10506-025-09482-6","article-title":"LegisSearch: navigating legislation with graphs and large language models","volume":"33","author":"Colombo","year":"2025","journal-title":"Artif Intell Law."},{"key":"B28","first-page":"3868","article-title":"Think-on-graph: deep and responsible reasoning of large language model on knowledge graph","volume-title":"International Conference on learning representations, Vol. 2024","author":"Sun","year":"2024"},{"key":"B29","first-page":"14400","article-title":"Reasoning on graphs: faithful and interpretable large language model reasoning","volume-title":"International conference on learning representations, Vol. 2024","author":"Luo","year":"2024"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1102","DOI":"10.1007\/s10489-025-06885-5","article-title":"Knowledge graph-extended retrieval augmented generation for question answering","volume":"55","author":"Linders","year":"2025","journal-title":"Appl Intell."},{"key":"B31","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1038\/s41746-024-01385-1","article-title":"An umbrella review on how digital health intervention co-design is conducted and described","volume":"7","author":"Kilfoy","year":"2024","journal-title":"NPJ Digit Med."},{"key":"B32","doi-asserted-by":"publisher","first-page":"e53855","DOI":"10.2196\/53855","article-title":"Centering digital health equity during technology innovation: protocol for a comprehensive scoping review of evidence-based tools and approaches","volume":"13","author":"Burns","year":"2024","journal-title":"JMIR Res Protoc."},{"key":"B33","doi-asserted-by":"publisher","first-page":"1873","DOI":"10.1038\/s41591-025-03626-6","article-title":"Sociodemographic biases in medical decision making by large language models","volume":"31","author":"Omar","year":"2025","journal-title":"Nat Med."},{"key":"B34","doi-asserted-by":"publisher","first-page":"4171","DOI":"10.18653\/v1\/N19-1423","article-title":"Bert: pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2019","journal-title":"Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, Vol. 1 (long and short papers)"},{"key":"B35","doi-asserted-by":"publisher","first-page":"2898","DOI":"10.18653\/v1\/2020.findings-emnlp.261","article-title":"LEGAL-BERT: the muppets straight out of law school","author":"Chalkidis","year":"2020","journal-title":"Findings of the association for computational linguistics: EMNLP 2020"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/IACIS65746.2025.11210974","article-title":"Longformer-enhanced hierarchical attention for high-quality extractive document summarization","author":"Joshi","year":"2025","journal-title":"2025 2nd international conference on intelligent algorithms for computational intelligence systems (IACIS)"},{"key":"B37","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-93417-4_38","article-title":"Modeling relational data with graph convolutional networks","volume-title":"European semantic web conference","author":"Schlichtkrull","year":"2018"},{"key":"B38","doi-asserted-by":"crossref","DOI":"10.1145\/3366423.3380027","article-title":"Heterogeneous graph transformer","volume-title":"Proceedings of the web conference 2020","author":"Hu","year":"2020"},{"key":"B39","doi-asserted-by":"publisher","first-page":"23424","DOI":"10.1609\/aaai.v39i22.34510","article-title":"Lightprof: a lightweight reasoning framework for large language model on knowledge graph","author":"Ao","year":"2025","journal-title":"Proceedings of the AAAI conference on artificial intelligence, Vol. 39"},{"key":"B40","doi-asserted-by":"publisher","first-page":"12282","DOI":"10.1609\/aaai.v39i12.33338","article-title":"Eperm: an evidence path enhanced reasoning model for knowledge graph question and answering","author":"Long","year":"2025","journal-title":"Proceedings of the AAAI conference on artificial intelligence, Vol. 39"},{"key":"B41","doi-asserted-by":"publisher","first-page":"3344","DOI":"10.1145\/3711896.3736849","article-title":"Atomr: atomic operator-empowered large language models for heterogeneous knowledge reasoning","author":"Xin","year":"2025","journal-title":"Proceedings of the 31st ACM SIGKDD conference on knowledge discovery and data mining V. 2"},{"key":"B42","doi-asserted-by":"crossref","DOI":"10.3233\/FAIA200321","article-title":"Span-based joint entity and relation extraction with transformer pre-training","volume-title":"ECAI 2020: 24th european conference on artificial intelligence\u2013including 10th conference on prestigious applications of artificial intelligence (PAIS 2020) (2006-2013)","author":"Eberts","year":"2020"},{"key":"B43","doi-asserted-by":"publisher","first-page":"1476","DOI":"10.18653\/v1\/2020.acl-main.136","article-title":"A novel cascade binary tagging framework for relational triple extraction","author":"Wei","year":"2020","journal-title":"Proceedings of the 58th annual meeting of the association for computational linguistics"},{"key":"B44","doi-asserted-by":"publisher","first-page":"5755","DOI":"10.18653\/v1\/2022.acl-long.395","article-title":"Unified structure generation for universal information extraction","author":"Lu","year":"2022","journal-title":"Proceedings of the 60th annual meeting of the association for computational linguistics, Vol 1 (long papers)"},{"key":"B45","doi-asserted-by":"publisher","first-page":"13121","DOI":"10.18653\/v1\/2025.acl-long.642","article-title":"Qwen2","author":"Yang","year":"2025"},{"key":"B46","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1038\/s41746-024-01157-x","article-title":"The ethics of ChatGPT in medicine and healthcare: a systematic review on Large Language Models (LLMs)","volume":"7","author":"Haltaufderheide","year":"2024","journal-title":"NPJ Digit Med."},{"key":"B47","doi-asserted-by":"publisher","first-page":"48","DOI":"10.18653\/v1\/D19-5406","article-title":"BillSum: a corpus for automatic summarization of US legislation","author":"Kornilova","year":"2019","journal-title":"Proceedings of the 2nd workshop on new frontiers in summarization"},{"key":"B48","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1001\/jama.2024.21700","article-title":"Testing and evaluation of health care applications of large language models: a systematic review","volume":"333","author":"Bedi","year":"2025","journal-title":"JAMA."},{"key":"B49","doi-asserted-by":"publisher","first-page":"6974","DOI":"10.18653\/v1\/2021.emnlp-main.559","article-title":"Multieurlex-a multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer","author":"Chalkidis","year":"2021","journal-title":"Proceedings of the 2021 conference on empirical methods in natural language processing"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1937335\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,7]],"date-time":"2026-09-07T05:34:00Z","timestamp":1788759240000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1937335\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,7]]},"references-count":49,"alternative-id":["10.3389\/fpubh.2026.1937335"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1937335","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,7]]},"article-number":"1937335"}}