{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:26:48Z","timestamp":1760059608798,"version":"build-2065373602"},"reference-count":55,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2025,6,29]],"date-time":"2025-06-29T00:00:00Z","timestamp":1751155200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>This article focuses on assisting medical professionals in analyzing domain-specific texts and selecting and comparing Named Entity Recognition (NER) frameworks. It details the development and evaluation of a system that utilizes a generic approach alongside the structured Nunamaker methodology. This system empowers medical professionals to train, evaluate, and compare NER models across diverse frameworks, such as Stanford CoreNLP, spaCy, and Hugging Face Transformers, independent of their specific implementations. Additionally, it introduces a concept for modeling a general training and evaluation process. Finally, experiments using various ontologies from the CRAFT corpus are conducted to assess the effectiveness of the current prototype.<\/jats:p>","DOI":"10.3390\/info16070554","type":"journal-article","created":{"date-parts":[[2025,6,30]],"date-time":"2025-06-30T13:06:17Z","timestamp":1751288777000},"page":"554","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A FIT4NER Generic Approach for Framework-Independent Medical Named Entity Recognition"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7344-6869","authenticated-orcid":false,"given":"Florian","family":"Freund","sequence":"first","affiliation":[{"name":"Faculty of Mathematics and Computer Science, University of Hagen, 58097 Hagen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0786-4253","authenticated-orcid":false,"given":"Philippe","family":"Tamla","sequence":"additional","affiliation":[{"name":"Faculty of Mathematics and Computer Science, University of Hagen, 58097 Hagen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-5024-2109","authenticated-orcid":false,"given":"Frederik","family":"Wilde","sequence":"additional","affiliation":[{"name":"Faculty of Mathematics and Computer Science, University of Hagen, 58097 Hagen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8293-2802","authenticated-orcid":false,"given":"Matthias","family":"Hemmje","sequence":"additional","affiliation":[{"name":"Faculty of Mathematics and Computer Science, University of Hagen, 58097 Hagen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Freund, F., Tamla, P., and Hemmje, M. (2023, January 7\u20138). Towards Improving Clinical Practice Guidelines through Named Entity Recognition: Model Development and Evaluation. Proceedings of the 2023 31st Irish Conference on Artificial Intelligence and Cognitive Science (AICS), Letterkenny, Ireland.","DOI":"10.1109\/AICS60730.2023.10470480"},{"key":"ref_2","unstructured":"Steinberg, E., Greenfield, S., Wolman, D.M., Mancher, M., and Graham, R. (2011). Clinical Practice Guidelines We Can Trust, National Academies Press."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kuhlen, R. (2004). Informationsethik: Umgang mit Wissen und Information in Elektronischen R\u00e4umen, UVK Verlag-Gesellschaft.","DOI":"10.1515\/9783110929225.159"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"402","DOI":"10.1162\/dint_a_00105","article-title":"Medical Named Entity Recognition from Un-labelled Medical Records Based on Pre-trained Language Models and Domain Dictionary","volume":"3","author":"Wen","year":"2021","journal-title":"Data Intell."},{"key":"ref_5","unstructured":"Pakhale, K. (2023). Comprehensive Overview of Named Entity Recognition: Models, Domain-Specific Applications and Challenges. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Freund, F., Tamla, P., and Hemmje, M. (2025, January 25\u201326). Survey: Understand the Challenges of Machine Learning Experts Using Named Entity Recognition Tools. Proceedings of the Computer Science, Coppenhagen, Denmark.","DOI":"10.5121\/csit.2024.150208"},{"key":"ref_7","unstructured":"Bielefeld University (2025, April 22). RATIO: Rationalizing Recommendations (RecomRatio). Available online: https:\/\/spp-ratio.de\/projects\/recomratio\/."},{"key":"ref_8","unstructured":"Hemmje, M. (2025, April 22). Chair of Multimedia and Internet Applications. Available online: https:\/\/www.fernuni-hagen.de\/multimedia-internetanwendungen\/en\/."},{"key":"ref_9","unstructured":"FTK (2025, April 22). FTK e.V. Research Institute for Telecommunications and Cooperation. Available online: https:\/\/www.ftk.de\/en."},{"key":"ref_10","unstructured":"Lamberth-Cocca, S., Dimanova, V., Bruchhaus, S., Nawroth, C., Mc Kevitt, P., and Hemmje, M. (2023, January 9\u201310). Towards Robust Named Entity Recognition to Support the Extraction of Emerging Technological Knowledge from Biomedical Literature. Proceedings of the CERC 2023, Barcelona, Spain."},{"key":"ref_11","unstructured":"FTK (2020). Artificial Intelligence for Hospitals, Healthcare & Humanity (AI4H3), FTK e.V. Research Institute for Telecommunications and Cooperation. R&D White Paper."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Vu, B., Wu, Y., Afli, H., Mc Kevitt, P., Walsh, P., Engel, F., Fuchs, M., and Hemmje, M. (2019, January 18\u201321). A Metagenomic Content and Knowledge Management Ecosystem Platform. Proceedings of the 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), San Diego, CA, USA.","DOI":"10.1109\/BIBM47256.2019.9043355"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Coenen, F., Fred, A., Aveiro, D., Dietz, J., Bernardino, J., Masciari, E., and Filipe, J. (2023). CIE: A Cloud-Based Information Extraction System for Named Entity Recognition in AWS, Azure, and Medical Domain. Knowledge Discovery, Knowledge Engineering and Knowledge Management, Springer Nature.","DOI":"10.1007\/978-3-031-43471-6"},{"key":"ref_14","unstructured":"Freund, F., Tamla, P., Reis, T., Hemmje, M., and Kevitt, P.M. (2023, January 9\u201310). FIT4NER\u2014Towards a Framework-Independent Toolkit for Named Entity Recognition. Proceedings of the CERC 2023, Barcelona, Spain."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Tamla, P., Freund, F., and Hemmje, M. (2021). Supporting Named Entity Recognition and Document Classification for Effective Text Retrieval. The Role of Gamification in Software Development Lifecycle, IntechOpen.","DOI":"10.5772\/intechopen.95076"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Freund, F., Tamla, P., Wilde, F., and Hemmje, M. (2025). Making Medical Experts Fit4ner: Transforming Domain Knowledge through Machine Learning-Based Named Entity Recognition. Int. J. Nat. Lang. Comput., 14.","DOI":"10.5121\/ijnlc.2025.14202"},{"key":"ref_17","unstructured":"Wilde, F. (2025). Entwicklung einer Microservice-basierten Abstraktionsebene f\u00fcr das Framework-unabh\u00e4ngige Training von Named Entity Recognition in einem Wissensmanagement-System f\u00fcr den medizinischen Bereich. [Master\u2019s Thesis, FernUniversit\u00e4t in Hagen]."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1080\/07421222.1990.11517898","article-title":"Systems Development in Information Systems Research","volume":"7","author":"Nunamaker","year":"1990","journal-title":"J. Manag. Inf. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"100017","DOI":"10.1016\/j.nlp.2023.100017","article-title":"A Survey on Named Entity Recognition \u2014 Datasets, Tools, and Methodologies","volume":"3","author":"Jehangir","year":"2023","journal-title":"Nat. Lang. Process. J."},{"key":"ref_20","unstructured":"Pagad, N.S., and Pradeep, N. (2022, January 19\u201320). Clinical Named Entity Recognition Methods: An Overview. Proceedings of the International Conference on Innovative Computing and Communications, Delhi, India."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1109\/TKDE.2020.2981314","article-title":"A Survey on Deep Learning for Named Entity Recognition","volume":"34","author":"Li","year":"2022","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_22","unstructured":"Konkol, I.M. (2015). Named Entity Recognition. [Ph.D. Thesis, University of West Bohemia]."},{"key":"ref_23","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv."},{"key":"ref_24","first-page":"9","article-title":"Language Models Are Unsupervised Multitask Learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI Blog"},{"key":"ref_25","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A Robustly Optimized Bert Pretraining Approach. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Murray-Smith, D.J. (2012). Modelling and Simulation of Integrated Systems in Engineering, Woodhead Publishing Limited.","DOI":"10.1533\/9780857096050"},{"key":"ref_27","unstructured":"Raaijmakers, S. (2022). Deep Learning for Natural Language Processing, Simon and Schuster."},{"key":"ref_28","unstructured":"Developers, T. (2021). TensorFlow. Zenodo."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N., Peters, M., Schmitz, M., and Zettlemoyer, L. (2018). AllenNLP: A Deep Semantic Natural Language Processing Platform. arXiv.","DOI":"10.18653\/v1\/W18-2501"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Klein, G., Kim, Y., Deng, Y., Senellart, J., and Rush, A.M. (2017). OpenNMT: Open-Source Toolkit for Neural Machine Translation. arXiv.","DOI":"10.18653\/v1\/P17-4012"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Miller, A.H., Feng, W., Fisch, A., Lu, J., Batra, D., Bordes, A., Parikh, D., and Weston, J. (2018). ParlAI: A Dialog Research Software Platform. arXiv.","DOI":"10.18653\/v1\/D17-2014"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chen, W., Qiu, P., and Cauteruccio, F. (2024). MedNER: A Service-Oriented Framework for Chinese Medical Named-Entity Recognition with Real-World Application. Big Data Cogn. Comput., 8.","DOI":"10.3390\/bdcc8080086"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"van Rijn, J.N., and Hutter, F. (2018, January 19\u201323). Hyperparameter Importance Across Datasets. Proceedings of the 24th ACM SIGKDD International\nConference on Knowledge Discovery & Data Mining, London, UK. KDD \u201918.","DOI":"10.1145\/3219819.3220058"},{"key":"ref_34","unstructured":"Ruder, S. (2017). An Overview of Gradient Descent Optimization Algorithms. arXiv."},{"key":"ref_35","unstructured":"Duan, X., Banchs, R.E., Zhang, M., Li, H., and Kumaran, A. (2016, January 12). Evaluating and Combining Name Entity Recognition Systems. Proceedings of the Sixth Named Entity Workshop, Berlin, Germany."},{"key":"ref_36","unstructured":"Sang, E.F.T.K., and De Meulder, F. (2003). Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. arXiv."},{"key":"ref_37","first-page":"339","article-title":"Named Entity Recognition Approaches","volume":"8","author":"Mansouri","year":"2008","journal-title":"Int. J. Comput. Sci. Netw. Secur."},{"key":"ref_38","first-page":"293","article-title":"Named Entity Recognition (NER) in NLP Techniques, Tools Accuracy and Performance","volume":"2","author":"Naseer","year":"2021","journal-title":"Pak. J. Multidiscip. Res."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., and McClosky, D. (2014, January 23\u201324). The Stanford CoreNLP Natural Language Processing Toolkit. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Baltimore, MD, USA.","DOI":"10.3115\/v1\/P14-5010"},{"key":"ref_40","unstructured":"Honnibal, M., Montani, I., Van Landeghem, S., and Boyd, A. (2020). spaCy: Industrial-strength Natural Language Processing in Python. Zenodo."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Jain, S.M. (2022). Introduction to Transformers for NLP: With the Hugging Face Library and Models to Solve Problems, Apress.","DOI":"10.1007\/978-1-4842-8844-3"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"ElBatanony, A., and Succi, G. (2021, January 17). Towards the No-Code Era: A Vision and Plan for the Future of Software Development. Proceedings of the 1st ACM SIGPLAN International Workshop on Beyond Code: No Code, Chicago, IL, USA. BCNC 2021.","DOI":"10.1145\/3486949.3486965"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Hu, C. (2023). Software Design Patterns. An Introduction to Software Design: Concepts, Principles, Methodologies, and Techniques, Springer International Publishing.","DOI":"10.1007\/978-3-031-28311-6"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Norman, D.A., and Draper, S.W. (1986). User Centered System Design, L. Erlbaum Associates Inc.. New Perspectives on Human-Computer Interaction.","DOI":"10.1201\/b15703"},{"key":"ref_45","unstructured":"Jacobson, L., and Booch, J.R.G. (2021). The Unified Modeling Language Reference Manual, Addison-Wesley Longman Ltd."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Ide, N., and Pustejovsky, J. (2017). The Colorado Richly Annotated Full Text (CRAFT) Corpus: Multi-Model Annotation in the Biomedical Domain. Handbook of Linguistic Annotation, Springer.","DOI":"10.1007\/978-94-024-0881-2"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Basaldella, M., Furrer, L., Tasso, C., and Rinaldi, F. (2017). Entity Recognition in the Biomedical Domain Using a Hybrid Approach. J. Biomed. Semant., 8.","DOI":"10.1186\/s13326-017-0157-6"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Hailu, N.D., Bada, M., Hadgu, A.T., and Hunter, L.E. (2019). Biomedical Concept Recognition Using Deep Neural Sequence Models. bioRxiv.","DOI":"10.1101\/530337"},{"key":"ref_49","unstructured":"Langer, S., Neuhaus, F., and N\u00fcrnberger, A. (2024, January 15\u201319). CEAR: Creating a Knowledge Graph of Chemical Entities and Roles in Scientific Literature. Proceedings of the Joint Ontology Workshops (JOWO)\u2014Episode X, Enschede, The Netherlands."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"D344","DOI":"10.1093\/nar\/gkm791","article-title":"ChEBI: A Database and Ontology for Chemical Entities of Biological Interest","volume":"36","author":"Degtyarenko","year":"2008","journal-title":"Nucleic Acids Res."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Diehl, A.D., Meehan, T.F., Bradford, Y.M., Brush, M.H., Dahdul, W.M., Dougall, D.S., He, Y., Osumi-Sutherland, D., Ruttenberg, A., and Sarntivijai, S. (2016). The Cell Ontology 2016: Enhanced Content, Modularization, and Ontology Interoperability. J. Biomed. Semant., 7.","DOI":"10.1186\/s13326-016-0088-7"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"D415","DOI":"10.1093\/nar\/gkt1173","article-title":"Protein Ontology: A Controlled Structured Network of Protein Entities","volume":"42","author":"Natale","year":"2014","journal-title":"Nucleic Acids Res."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Mungall, C.J., Torniai, C., Gkoutos, G.V., Lewis, S.E., and Haendel, M.A. (2012). Uberon, an Integrative Multi-Species Anatomy Ontology. Genome Biol., 13.","DOI":"10.1186\/gb-2012-13-1-r5"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Bhargava, P., Drozd, A., and Rogers, A. (2021). Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics. arXiv.","DOI":"10.18653\/v1\/2021.insights-1.18"},{"key":"ref_55","unstructured":"Turc, I., Chang, M., Lee, K., and Toutanova, K. (2019). Well-Read Students Learn Better: The Impact of Student Initialization on Knowledge Distillation. arXiv."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/7\/554\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:01:12Z","timestamp":1760032872000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/7\/554"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,29]]},"references-count":55,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["info16070554"],"URL":"https:\/\/doi.org\/10.3390\/info16070554","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2025,6,29]]}}}