{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:34:10Z","timestamp":1760146450988,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,11,3]],"date-time":"2024-11-03T00:00:00Z","timestamp":1730592000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Henan Province Key Scientific Research Project Plan for Higher Education Institutions from Henan Provincial Department of Education","award":["24A520060"],"award-info":[{"award-number":["24A520060"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>In the area of consumer health search (CHS), there is an increasing concern about returning topically relevant and understandable health information to the user. Besides being used to rank topically relevant documents, Learning to Rank (LTR) has also been used to promote understandability ranking. Traditionally, features coming from different document fields are joined together, limiting the performance of standard LTR, since field information plays an important role in promoting understandability ranking. In this paper, a novel field-level Learning-to-Rank (f-LTR) approach is proposed, and its application in CHS is investigated by developing thorough experiments on CLEF\u2019 2016\u20132018 eHealth IR data collections. An in-depth analysis of the effects of using f-LTR is provided, with experimental results suggesting that in LTR, title features are more effective than other field features in promoting understandability ranking. Moreover, the fused f-LTR model is compared to existing work, confirming the effectiveness of the methodology.<\/jats:p>","DOI":"10.3390\/info15110695","type":"journal-article","created":{"date-parts":[[2024,11,4]],"date-time":"2024-11-04T10:57:20Z","timestamp":1730717840000},"page":"695","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Improving Consumer Health Search with Field-Level Learning-to-Rank Techniques"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6720-4831","authenticated-orcid":false,"given":"Hua","family":"Yang","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Zhongyuan University of Technology, Zhengzhou 450007, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1323-0249","authenticated-orcid":false,"given":"Teresa","family":"Gon\u00e7alves","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of \u00c9vora, 7000-671 \u00c9vora, Portugal"},{"name":"VISTA Lab, Algoritmi Center, University of \u00c9vora, 7000-671 \u00c9vora, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,3]]},"reference":[{"doi-asserted-by":"crossref","unstructured":"Pugachev, A., Artemova, E., Bondarenko, A., and Braslavski, P. (2023, January 2\u20136). Consumer health question answering using off-the-shelf components. Proceedings of the European Conference on Information Retrieval, Dublin, Ireland.","key":"ref_1","DOI":"10.1007\/978-3-031-28238-6_48"},{"doi-asserted-by":"crossref","unstructured":"Upadhyay, R., Pasi, G., and Viviani, M. (2023, January 18\u201322). A passage retrieval transformer-based re-ranking model for truthful consumer health search. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Turin, Italy.","key":"ref_2","DOI":"10.1007\/978-3-031-43412-9_21"},{"doi-asserted-by":"crossref","unstructured":"Upadhyay, R., Knoth, P., Pasi, G., and Viviani, M. (2023). Explainable online health information truthfulness in Consumer Health Search. Front. Artif. Intell., 6.","key":"ref_3","DOI":"10.3389\/frai.2023.1184851"},{"doi-asserted-by":"crossref","unstructured":"Goeuriot, L., Suominen, H., Kelly, L., Alemany, L.A., Brew-Sam, N., Cotik, V., Filippo, D., Gonzalez Saez, G., Luque, F., and Mulhem, P. (April, January 28). CLEF eHealth evaluation lab 2021. Proceedings of the Advances in Information Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event. Proceedings, Part II 43.","key":"ref_4","DOI":"10.1007\/978-3-030-72240-1_69"},{"doi-asserted-by":"crossref","unstructured":"Zehlike, M., and Castillo, C. (2020, January 20\u201324). Reducing disparate exposure in ranking: A learning to rank approach. Proceedings of the Web Conference 2020, Taipei, Taiwan.","key":"ref_5","DOI":"10.1145\/3366424.3380048"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e030140","DOI":"10.1161\/JAHA.123.030140","article-title":"Evaluating readability, understandability, and actionability of online printable patient education materials for cholesterol management: A systematic review","volume":"13","author":"Bhatt","year":"2024","journal-title":"J. Am. Heart Assoc."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2374373521998847","DOI":"10.1177\/2374373521998847","article-title":"Readability of patient education materials from high-impact medical journals: A 20-year analysis","volume":"8","author":"Rooney","year":"2021","journal-title":"J. Patient Exp."},{"unstructured":"Deidra Bunn, S., and Erickson, K. (2022). Voices from Academia Minimizing the Complexity of Public Health Documents: Making COVID-19 Documents Accessible to Individuals Who Read Below the Third-Grade Level. Assistive Technology Outcomes and Benefits Accessible Public Health Materials During a Pandemic: Lessons Learned from COVID-19, Assistive Technology Outcomes & Benefits (ATOB).","key":"ref_8"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"9780317","DOI":"10.1155\/2017\/9780317","article-title":"Readability assessment of online patient education material on congestive heart failure","volume":"2017","author":"Kher","year":"2017","journal-title":"Adv. Prev. Med."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e37","DOI":"10.1097\/EDE.0000000000000650","article-title":"Readability Assessment of Patient-Centered Outcomes Research Institute Public Abstracts in Relation to Accessibility","volume":"28","author":"Hollada","year":"2017","journal-title":"Epidemiology"},{"doi-asserted-by":"crossref","unstructured":"Antunes, H., and Lopes, C.T. (2020, January 25\u201330). Proposal and comparison of health specific features for the automatic assessment of readability. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Xi\u2019an, China.","key":"ref_11","DOI":"10.1145\/3397271.3401187"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"e36835","DOI":"10.2196\/36835","article-title":"Readability of English, German, and Russian Disease-Related Wikipedia Pages: Automated Computational Analysis","volume":"24","author":"Gordejeva","year":"2022","journal-title":"J. Med. Internet Res."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1561\/1500000016","article-title":"Learning to rank for information retrieval","volume":"3","author":"Liu","year":"2009","journal-title":"Found. Trends\u00ae Inf. Retr."},{"key":"ref_14","first-page":"81","article-title":"From ranknet to lambdarank to lambdamart: An overview","volume":"11","author":"Burges","year":"2010","journal-title":"Learning"},{"doi-asserted-by":"crossref","unstructured":"Joachims, T. (2002, January 23\u201326). Optimizing search engines using clickthrough data. Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Edmonton, AB, Canada.","key":"ref_15","DOI":"10.1145\/775047.775067"},{"doi-asserted-by":"crossref","unstructured":"Miyachi, Y., Ishii, O., and Torigoe, K. (2023). Design, implementation, and evaluation of the computer-aided clinical decision support system based on learning-to-rank: Collaboration between physicians and machine learning in the differential diagnosis process. BMC Med. Inform. Decis. Mak., 23.","key":"ref_16","DOI":"10.1186\/s12911-023-02123-5"},{"key":"ref_17","first-page":"58","article-title":"Significance of machine learning in healthcare: Features, pillars and applications","volume":"3","author":"Javaid","year":"2022","journal-title":"Int. J. Intell. Netw."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"291","DOI":"10.2174\/1389202922666210705124359","article-title":"Machine learning in healthcare","volume":"22","author":"Habehh","year":"2021","journal-title":"Curr. Genom."},{"doi-asserted-by":"crossref","unstructured":"Geng, X., Liu, T.Y., Qin, T., and Li, H. (2007, January 23\u201327). Feature selection for ranking. Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands.","key":"ref_19","DOI":"10.1145\/1277741.1277811"},{"doi-asserted-by":"crossref","unstructured":"Xu, J., and Li, H. (2007, January 23\u201327). Adarank: A boosting algorithm for information retrieval. Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands.","key":"ref_20","DOI":"10.1145\/1277741.1277809"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e30258","DOI":"10.2196\/30258","article-title":"Designing Formulae for Ranking Search Results: Mixed Methods Evaluation Study","volume":"9","author":"Douze","year":"2022","journal-title":"JMIR Hum. Factors"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1017\/S1351324920000029","article-title":"Learning to rank for multi-label text classification: Combining different sources of information","volume":"27","author":"Azarbonyad","year":"2021","journal-title":"Nat. Lang. Eng."},{"doi-asserted-by":"crossref","unstructured":"Ueda, A., Santos, R.L., Macdonald, C., and Ounis, I. (2021, January 11\u201315). Structured Fine-Tuning of Contextual Embeddings for Effective Biomedical Retrieval. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual.","key":"ref_23","DOI":"10.1145\/3404835.3463075"},{"unstructured":"Fox, E.A., and Shaw, J.A. (September, January 31). Combination of multiple searches. Proceedings of the 2nd Text REtrieval Conference (TREC-2), Gaithersburg, MD, USA. NIST Special Publication 500-215.","key":"ref_24"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1023\/A:1009980820262","article-title":"Fusion via a linear combination of scores","volume":"1","author":"Vogt","year":"1999","journal-title":"Inf. Retr."},{"doi-asserted-by":"crossref","unstructured":"Manmatha, R., Rath, T., and Feng, F. (2001, January 9\u201313). Modeling score distributions for combining the outputs of search engines. Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, New Orleans, LA, USA.","key":"ref_26","DOI":"10.1145\/383952.384005"},{"doi-asserted-by":"crossref","unstructured":"Kuzi, S., Shtok, A., and Kurland, O. (2016, January 24\u201328). Query expansion using word embeddings. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, Indianapolis, IN, USA.","key":"ref_27","DOI":"10.1145\/2983323.2983876"},{"doi-asserted-by":"crossref","unstructured":"Xia, X., Lo, D., Wang, X., Zhang, C., and Wang, X. (2014, January 2\u20133). Cross-language bug localization. Proceedings of the 22nd International Conference on Program Comprehension, Hyderabad, India.","key":"ref_28","DOI":"10.1145\/2597008.2597788"},{"doi-asserted-by":"crossref","unstructured":"Ru, X., Ye, X., Sakurai, T., and Zou, Q. (2021). Application of learning to rank in bioinformatics tasks. Briefings Bioinform., 22.","key":"ref_29","DOI":"10.1093\/bib\/bbaa394"},{"key":"ref_30","first-page":"4041","article-title":"An approach of a quantum-inspired document ranking algorithm by using feature selection methodology","volume":"15","author":"Bhagawati","year":"2023","journal-title":"Int. J. Inf. Technol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"102390","DOI":"10.1016\/j.ipm.2020.102390","article-title":"Detecting health misinformation in online health communities: Incorporating behavioral features into machine learning based approaches","volume":"58","author":"Zhao","year":"2021","journal-title":"Inf. Process. Manag."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1938","DOI":"10.1080\/10447318.2022.2089085","article-title":"Machine learning techniques in adaptive and personalized systems for health and wellness","volume":"39","author":"Oyebode","year":"2023","journal-title":"Int. J. Hum. Comput. Interact."},{"doi-asserted-by":"crossref","unstructured":"Henrich, A., and Wegmann, M. (2021, January 22\u201326). Search and evaluation methods for class level information retrieval: Extended use and evaluation of methods applied in expertise retrieval. Proceedings of the 36th Annual ACM Symposium on Applied Computing, Gwangju, Republic of Korea.","key":"ref_33","DOI":"10.1145\/3412841.3442092"},{"doi-asserted-by":"crossref","unstructured":"Macdonald, C., Tonellotto, N., MacAvaney, S., and Ounis, I. (2021, January 1\u20135). PyTerrier: Declarative experimentation in Python from BM25 to dense retrieval. Proceedings of the 30th ACM International Conference on Information & Knowledge Management, Gold Coast, Australia.","key":"ref_34","DOI":"10.1145\/3459637.3482013"},{"unstructured":"Aloteibi, S. (2021). A User-Centred Approach to Information Retrieval. [Ph.D. Thesis, University of Cambridge].","key":"ref_35"},{"doi-asserted-by":"crossref","unstructured":"Santos, P.M., and Teixeira Lopes, C. (2020, January 14\u201317). Generating query suggestions for cross-language and cross-terminology health information retrieval. Proceedings of the Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal. Proceedings, Part II 42.","key":"ref_36","DOI":"10.1007\/978-3-030-45442-5_43"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1843","DOI":"10.1016\/j.procs.2023.10.174","article-title":"How Normalization Strategies Affect the Quality of Rank Aggregation Methods in Recommendation Systems","volume":"225","author":"Boryczka","year":"2023","journal-title":"Procedia Comput. Sci."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1698","DOI":"10.1016\/j.ipm.2019.05.009","article-title":"Query expansion techniques for information retrieval: A survey","volume":"56","author":"Azad","year":"2019","journal-title":"Inf. Process. Manag."},{"unstructured":"Zuccon, G., and Palotti, J. (2018, January 10\u201314). Overview of the CLEF 2018 Consumer Health Search Task. Proceedings of the Working Notes of Conference and Labs of the Evaluation (CLEF) Forum, CEUR Workshop Proceedings, Avignon, France.","key":"ref_39"},{"unstructured":"Nentidis, A., Katsimpras, G., Krithara, A., and Paliouras, G. (2024). Overview of BioASQ tasks 12b and Synergy12 in CLEF2024. Work. Notes CLEF, 2024, Available online: https:\/\/ceur-ws.org\/Vol-3740\/paper-01.pdf.","key":"ref_40"},{"unstructured":"\u015eerbet\u00e7i, O., Wang, X.D., and Leser, U. (2024, January 9\u201312). HU-WBI at BioASQ12B Phase A: Exploring Rank Fusion of Dense Retrievers and Re-rankers. Proceedings of the Conference and Labs of the Evaluation Forum, Grenoble, France.","key":"ref_41"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/11\/695\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:27:42Z","timestamp":1760113662000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/11\/695"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,3]]},"references-count":41,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["info15110695"],"URL":"https:\/\/doi.org\/10.3390\/info15110695","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2024,11,3]]}}}