{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T06:27:38Z","timestamp":1785479258131,"version":"3.56.0"},"reference-count":51,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,2,16]],"date-time":"2021-02-16T00:00:00Z","timestamp":1613433600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJERPH"],"abstract":"<jats:p>(1) Background: Modern medicine generates a great deal of information that stored in medical databases. Simultaneously, extracting useful knowledge and making scientific decisions for diagnosis and treatment of diseases becomes increasingly necessary. Headache disorders are the most prevalent of all the neurological conditions. Headaches have not only medical but also great socioeconomic significance. The aim of this research is to develop an intelligent system for diagnosing primary headache disorders. (2) Methods: This research applied various mathematical, statistical and artificial intelligence techniques, among which the most important are: Calinski-Harabasz index, Analytical Hierarchy Process, and Weighted Fuzzy C-means Clustering Algorithm. These methods, techniques and methodologies are used to create a hybrid intelligent system for diagnosing primary headache disorders. The proposed intelligent diagnostic system is tested with original real-world data set with different metrics. (3) Results: First at all, nine of 20 attributes \u2013 features from International Headache Society (IHS) criteria are selected, and then only five most important attributes from IHS criteria are selected. The calculation result based on the Calinski\u2013Harabasz index value (178) for the optimal number of clusters is three, and they present three classes of headaches: (i) migraine, (ii) tension-type headaches (TTHs), and (iii) other primary headaches (OPHs). The proposed hybrid intelligent system shows the following quality metrics: Accuracy 75%; Precision 67% for migraine, 74% for TTHs, 86% for OPHs, and Average Precision 77%; Recall 86% for migraine, 73% for TTHs, 67% for OPHs, Average Recall 75%; F1 score 75% for migraine, 74% for TTHs, 75% for OPHs, and Average F1 score 75%. (4) Conclusions: The hybrid intelligent system presents qualitative and respectable experimental results. The implementation of existing diagnostics systems and the development of new diagnostics systems in medicine is necessary in order to help physicians make quality diagnosis and decide the best treatments for the patients.<\/jats:p>","DOI":"10.3390\/ijerph18041890","type":"journal-article","created":{"date-parts":[[2021,2,16]],"date-time":"2021-02-16T05:02:14Z","timestamp":1613451734000},"page":"1890","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["An Application of a Hybrid Intelligent System for Diagnosing Primary Headaches"],"prefix":"10.3390","volume":"18","author":[{"given":"Svetlana","family":"Simi\u0107","sequence":"first","affiliation":[{"name":"Faculty of Medicine, University of Novi Sad, 21000 Novi Sad, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6024-9527","authenticated-orcid":false,"given":"Jos\u00e9 R.","family":"Villar","sequence":"additional","affiliation":[{"name":"Faculty of Geology, Campus de Llamaquique, University of Oviedo, 33005 Oviedo, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2333-8405","authenticated-orcid":false,"given":"Jos\u00e9 Luis","family":"Calvo-Rolle","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering, University of A Coru\u00f1a, 15405 Ferrol-A Coru\u00f1a, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Slobodan R.","family":"Sekuli\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, University of Novi Sad, 21000 Novi Sad, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0692-2124","authenticated-orcid":false,"given":"Svetislav D.","family":"Simi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Technical Sciences, University of Novi Sad, 21000 Novi Sad, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7392-9767","authenticated-orcid":false,"given":"Dragan","family":"Simi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Technical Sciences, University of Novi Sad, 21000 Novi Sad, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/s10194-018-0854-2","article-title":"The epidemiology of headache disorders: A face-to-face interview of participants in HUNT4","volume":"19","author":"Hagen","year":"2018","journal-title":"J. Headache Pain"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1001\/jama.279.5.381","article-title":"Epidemiology of tension-type headache","volume":"279","author":"Schwartz","year":"1998","journal-title":"JAMA"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1055\/s-0038-1646946","article-title":"The global burden of headache","volume":"38","author":"Saylor","year":"2018","journal-title":"Semin. Neurol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1111\/head.12482","article-title":"The prevalence and burden of migraine and severe headache in the United States: Updated statistics from government health surveillance studies","volume":"55","author":"Burch","year":"2015","journal-title":"Headache"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1186\/s10194-019-1062-4","article-title":"Prevalence, burden, and clinical management of migraine in China, Japan, and South Korea: A comprehensive review of the literature","volume":"20","author":"Takeshima","year":"2019","journal-title":"J. Headache Pain"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1007\/s10194-008-0051-9","article-title":"Improving care through health economics analyses: Cost of illness and headache","volume":"9","author":"Mennini","year":"2008","journal-title":"J. Headache Pain"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1111\/head.13301","article-title":"Migraine burden of disease: From the patient\u2019s experience to a socio-economic view","volume":"58","author":"Agosti","year":"2018","journal-title":"Headache"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1007\/s10194-005-0219-5","article-title":"Risk\u2013Benefit and cost\u2013benefit ratio in headache treatment","volume":"6","author":"Pini","year":"2005","journal-title":"J. Headache Pain"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.1468-1331.2005.01190.x","article-title":"Cost of disorder of the brain in Europe","volume":"12","author":"Jonsson","year":"2005","journal-title":"J. Eur. Neurol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"813","DOI":"10.1001\/archinte.159.8.813","article-title":"Burden of migraine in the United States: Disability and economic cost","volume":"159","author":"Hu","year":"1999","journal-title":"Arch. Intern. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"709","DOI":"10.1111\/j.1468-2982.2005.00946.x","article-title":"An introduction to the basic principles of health economics for those involved in the development and delivery of headache care","volume":"25","author":"Kernick","year":"2005","journal-title":"Cephalalgia"},{"key":"ref_12","first-page":"411","article-title":"Costs of acute headache medication use and productivity losses among patients with migraine: Insights from three randomized controlled trials","volume":"3","author":"Porter","year":"2019","journal-title":"Pharm. Open"},{"key":"ref_13","first-page":"670","article-title":"Guideline for primary care management of headache in adults","volume":"8","author":"Becker","year":"2015","journal-title":"Can. Fam. Physician"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Headache Classification Committee of the International Headache Society (2013). The International Classification of Headache Disorders, 3rd edition (beta version). Cephalalgia, 33, 629\u2013808.","DOI":"10.1177\/0333102413485658"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Simi\u0107, S., Rabi-\u017diki\u0107, T., Villar, J.R., Calvo-Rolle, J.L., Simi\u0107, D., and Simi\u0107, S.D. (2020). Impact of individual headache types on the work and work efficiency of headache sufferers. Int. J. Environ. Res. Public Health, 17.","DOI":"10.3390\/ijerph17186918"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1007\/978-3-540-87656-4_39","article-title":"Computer-Assisted diagnosis of primary headaches","volume":"Volume 5271","author":"Slankamenac","year":"2008","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1007\/978-3-540-87881-0_37","article-title":"Rule-Based Fuzzy Logic System for Diagnosing Migraine","volume":"Volume 5138","author":"Slankamenac","year":"2008","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1007\/978-3-319-92639-1_62","article-title":"A hybrid clustering approach for diagnosing medical diseases","volume":"Volume 10870","year":"2018","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1007\/978-3-319-19644-2_40","article-title":"A Hybrid Analytic Hierarchy process for clustering and ranking best location for logistics distribution center","volume":"Volume 9121","author":"Ilin","year":"2015","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1093\/jigpal\/jzx047","article-title":"A hybrid clustering and ranking method for best positioned logistics distribution centre in Balkan Peninsula","volume":"25","author":"Ilin","year":"2017","journal-title":"Log. J. IGPL"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Headache Classification Committee of the International Headache Society (2018). The international classification of headache disorders, 3rd edition. Cephalalgia, 38, 1\u2013211.","DOI":"10.1177\/0333102417738202"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"157","DOI":"10.2478\/s11536-012-0098-5","article-title":"Automatic diagnosis of primary headaches by machine learning methods","volume":"8","author":"Krawczyk","year":"2013","journal-title":"Open Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"592","DOI":"10.1007\/978-3-319-68935-7_64","article-title":"Stochastic and non-stochastic feature selection","volume":"Volume 10585","author":"Correia","year":"2017","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1016\/0021-9681(82)90014-5","article-title":"Cluster analysis to determine headache types","volume":"35","author":"Diehr","year":"1982","journal-title":"J. Chronic Dis."},{"key":"ref_25","unstructured":"Bonaccorso, G. (2018). Machine Learning Algorithms, Packt Publishing. [2nd ed.]."},{"key":"ref_26","unstructured":"Babu\u0161ka, R., van der Veen, P.J., and Kaymak, U. (2002, January 12\u201317). Improved covariance estimation for Gustafson-Kessel clustering. Proceedings of the 2002 IEEE International Conference on Fuzzy Systems, Honolulu, HI, USA."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/0022-2496(77)90033-5","article-title":"Scaling method for priorities in hierarchical structures","volume":"15","author":"Saaty","year":"1977","journal-title":"J. Math. Psychol."},{"key":"ref_28","unstructured":"Saaty, T.L. (1980). Analytic Hierarchy Process, Planning, Priority Setting, Resource Allocation, McGraw Hill Inc."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Everitt, B.S., Landau, S., Leese, M., and Stahl, D. (2011). ClusterAnalysis, Wiley.","DOI":"10.1002\/9780470977811"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1080\/01969727308546046","article-title":"A Fuzzy relative of the ISODATA Process and its use in detecting compact well-separated clusters","volume":"3","author":"Dunn","year":"1973","journal-title":"J. Cybern."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Bezdek, J.C. (1981). Pattern Recognition with Fuzzy Objective Function Algorithms, Kluwer Academic Publishers.","DOI":"10.1007\/978-1-4757-0450-1"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1016\/0304-3959(83)90150-1","article-title":"The headache scale: A new approach to the assessment of headache pain based on pain descriptions","volume":"16","author":"Hunter","year":"1983","journal-title":"Pain"},{"key":"ref_33","first-page":"641","article-title":"Use of k-means clustering in migraine detection by using EEG records under flash stimulation","volume":"6","author":"Alkan","year":"2011","journal-title":"Int. J. Phys. Sci."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4200","DOI":"10.3906\/elk-1612-178","article-title":"An ant colony optimization algorithm-based classification for the diagnosis of primary headaches using a website questionnaire expert system","volume":"25","author":"Celik","year":"2017","journal-title":"Turk. J. Electr. Eng. Comput. Sci."},{"key":"ref_35","unstructured":"Celik, U., Yurtay, N., Koc, E.R., Tepe, N., Gulluoglu, H., and Ertas, M. (2015). Diagnostic accuracy comparison of artificial immune algorithms for primary headaches. Comput. Math. Methods Med., 465192."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Simi\u0107, S., Bankovi\u0107, Z., Villar, J.R., Simi\u0107, D., and Simi\u0107, S.D. (2020). A hybrid fuzzy clustering approach for diagnosing primary headache disorder. Log. J. IGPL.","DOI":"10.1007\/978-3-030-61705-9_62"},{"key":"ref_37","first-page":"71","article-title":"Fuzzy AHP method to determine headache types based on symptoms","volume":"58","author":"Maseleno","year":"2017","journal-title":"Investig. Clin."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1016","DOI":"10.1046\/j.1526-4610.2002.02231.x","article-title":"Application of the 1988 International Headache Society diagnostic criteria in nine Italian headache centers using a computerized structured record","volume":"42","author":"Gallai","year":"2002","journal-title":"Headache"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1007\/s10194-005-0186-x","article-title":"Application of ICHD 2nd edition criteria for primary headaches with the aid of a computerised, structured medical record for the specialist","volume":"6","author":"Sarchielli","year":"2005","journal-title":"J. Headache Pain"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1186\/2193-1801-2-199","article-title":"A computerized expert system for diagnosing primary headache based on International Classification of Headache Disorder (ICHD-II)","volume":"2","author":"Eslami","year":"2013","journal-title":"Springer Plus"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.jocn.2019.08.084","article-title":"Suicidality and its risk factors in tension-type headache patients: A multicenter case-control study","volume":"69","author":"Seo","year":"2019","journal-title":"J. Clin. Neurosci."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"102276","DOI":"10.1016\/j.ctim.2019.102276","article-title":"The features and burden of headaches within a chiropractic clinical population: A cross-sectional analysis","volume":"48","author":"Moore","year":"2020","journal-title":"Complement. Ther. Med."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1016\/j.ajem.2018.07.030","article-title":"The effect of peer influence on the use of CT by emergency physicians for patients with headaches","volume":"37","author":"Huang","year":"2019","journal-title":"Am. J. Emerg. Med."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"105382","DOI":"10.1016\/j.clineuro.2019.105382","article-title":"Comparison of chronic daily headache with and without medication overuse headache using ICHD II R and ICHD 3 beta criteria","volume":"183","author":"Laskar","year":"2019","journal-title":"Clin. Neurol. Neurosurg."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"2443","DOI":"10.1001\/jama.290.18.2443","article-title":"Lost productive time and cost due to common pain conditions in the US workforce","volume":"290","author":"Stewart","year":"2003","journal-title":"JAMA"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1111\/j.1526-4610.2004.446005.x","article-title":"Impact of headache on quality of life in general population survey in France (GRIM 2000 Study)","volume":"44","author":"Duru","year":"2004","journal-title":"Headache"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1111\/j.1468-2982.2004.00660.x","article-title":"Assessment of migraine severity using the MIGSEV scale: Relationship to migraine features and quality of life","volume":"24","author":"Vray","year":"2004","journal-title":"Cephalalgia"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"715","DOI":"10.1016\/j.jpain.2012.03.009","article-title":"The economic costs of pain in the United States","volume":"13","author":"Darrell","year":"2012","journal-title":"J. Pain"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1046\/j.1468-2982.2003.00509.x","article-title":"One-Year prevalence of migraine in Austria: A nation-wide survey","volume":"23","author":"Lampl","year":"2003","journal-title":"Cephalalgia"},{"key":"ref_50","first-page":"84","article-title":"The prevalence and characteristics of migraine among the Belgian working population","volume":"107","author":"Moens","year":"2007","journal-title":"ActaNeurol. Belg."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1046\/j.1468-2982.2003.00568.x","article-title":"The prevalence and disability burden of adult migraine in England and their relationships to age, gender and ethnicity","volume":"23","author":"Steiner","year":"2003","journal-title":"Cephalalgia"}],"container-title":["International Journal of Environmental Research and Public Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/4\/1890\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:24:40Z","timestamp":1760160280000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/4\/1890"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,16]]},"references-count":51,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["ijerph18041890"],"URL":"https:\/\/doi.org\/10.3390\/ijerph18041890","relation":{},"ISSN":["1660-4601"],"issn-type":[{"value":"1660-4601","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,16]]}}}