{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T12:43:28Z","timestamp":1777380208212,"version":"3.51.4"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"11","license":[{"start":{"date-parts":[[2024,11,21]],"date-time":"2024-11-21T00:00:00Z","timestamp":1732147200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2024,11,30]]},"abstract":"<jats:p>Multimodal content contains more deception than unimodal information, causing significant social and economic impacts. Current techniques often focus on a single modality, neglecting knowledge fusion. While most studies have concentrated on English fake news detection, this study explores multimodality for low-resource languages like Hindi. This work introduces the MMHFND model, based on M-CLIP, which uses late fusion for coarse (Fake vs Real) and fine-grained (World vs India vs Politics vs News vs Fact-Check) configurations. We extract deep representations from image and text using image transformer ResNet-50, a BERT-based L3cube-HindRoberta text transformer handling headlines, content, OCR text, and image captions, paired M-CLIP transformers, and an ELA (Error-Level Analysis) image forensic method incorporating EfficientNet B0 to analyze multimodal news in Hindi language based on Devanagari script. M-CLIP integrates cross-modal similarity mapping of images and texts with retrieved multimodal features. The extracted features undergo redundancy reduction before being channeled into the final classifier. The MAM (Modality Attention Mechanism) is introduced, which generates weights for each modality individually. The MMHFND model uses a computed modality divergence score to identify dissonance between modalities and a modified contrastive loss on the score. We thoroughly analyze the HinFakeNews dataset in a multimodal context, achieving accuracy in coarse- and fine-grained configurations. We also undertake an ablation study to evaluate outcomes and explore alternative fusion processes on three different setups. The results show that the MMHFND model effectively detects fake news in Hindi with an accuracy of 0.986, outperforming other existing multimodal approaches.<\/jats:p>","DOI":"10.1145\/3686797","type":"journal-article","created":{"date-parts":[[2024,8,12]],"date-time":"2024-08-12T11:19:25Z","timestamp":1723461565000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["MMHFND: Fusing Modalities for Multimodal Multiclass Hindi Fake News Detection via Contrastive Learning"],"prefix":"10.1145","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4539-7051","authenticated-orcid":false,"given":"Richa","family":"Sharma","sequence":"first","affiliation":[{"name":"CSE, PES University, Bengaluru, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4470-0311","authenticated-orcid":false,"given":"Arti","family":"Arya","sequence":"additional","affiliation":[{"name":"CSE, PES University, Bengaluru, India and CSE, PES University, Bengaluru, India"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,11,21]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313552"},{"key":"e_1_3_2_3_2","first-page":"518","volume-title":"Proceedings of the IEEE International Conference on Data Mining (ICDM\u201919)","author":"Peng Q.","year":"2019","unstructured":"Q. Peng, J. Cao, T. Yang, G. Junbo, and L. Jintao. 2019. Exploiting multidomain visual information for fake news detection. In Proceedings of the IEEE International Conference on Data Mining (ICDM\u201919). IEEE. 518\u2013527. https:\/\/doi.ieeecomputersociety.org\/10.1109\/ICDM.2019.00062"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3161603"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2018.10.495"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395046"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2015.7113322"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2016.2617078"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/RIVF51545.2021.9642125"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3123454"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigMM.2019.00-44"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219903"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3343031.3350850"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3511968"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSS.2023.3244068"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2021.03.037"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3410566.3410599"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIRCA51532.2021.9544661"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10844-022-00764-y"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.3390\/e25040614"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2103.00020"},{"key":"e_1_3_2_22_2","unstructured":"N. Batra. 2021. What Percentage of People can Speak Hindi in India? Check Here. Retrieved August 15 2024 from https:\/\/www.jagranjosh.com\/general-knowledge\/know-what-percentage-of-people-in-india-can-speak-hindi-1694658097-1"},{"key":"e_1_3_2_23_2","unstructured":"F. Carlsson P. Eisen F. Rekathati and M. Sahlgren. 2022. Cross-lingual and multilingual CLIP. In Proceedings of the 13th Conference on Language Resources and Evaluation (LREC\u201922). 6848\u20136854. https:\/\/aclanthology.org\/2022.lrec-1.739"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1905.11946"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","unstructured":"R. Joshi. 2023. L3Cube-HindBERT and DevBERT: Pre-Trained BERT transformer models for Devanagri-based Hindi and Marathi languages. arXiv:2211.11418 (2023). 10.48550\/arXiv.2211.11418","DOI":"10.48550\/arXiv.2211.11418"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00745"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","unstructured":"K. Chugh P. Gupta A. Dhall and R. Subramanian. 2020. Not made for each other\u2014Audio-visual dissonance-based deepfake detection and localization. In Proceedings of the 28th ACM International Conference on Multimedia (MM\u201920). ACM New York NY USA 439\u2013447. 10.1145\/3394171.3413700","DOI":"10.1145\/3394171.3413700"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3589764"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3123454"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313552"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219903"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i10.7230"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.7544\/issn1000-1239.2020.20200413"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2024.3386168"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2021.11.006"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","unstructured":"H. Zhang Q. Fang S. Qian and C. Xu. 2019. Multi-modal knowledge-aware event memory network for social media rumor detection. In Proceedings of the 27th ACM International Conference on Multimedia (MM\u201919). ACM New York NY USA 1942\u20131951. 10.1145\/3343031.3350850","DOI":"10.1145\/3343031.3350850"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00950"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.393"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2103.00020"},{"key":"e_1_3_2_41_2","unstructured":"D. Dong F. Lin G. Li and B. Liu. 2022. Similarity-aware attention network for multimodal fake news detection. In Proceedings of the 3rd International Conference on Big Data and Artificial Intelligence and Software Engineering (ICBASE\u201922) Vol. 3304. CEUR-ws.org\/Vol-3304\/paper10.pdf"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3591106.3592271"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/INCOFT55651.2022.10094358"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1002\/itl2.523"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40747-021-00552-1"},{"issue":"1","key":"e_1_3_2_46_2","first-page":"81","article-title":"IndDeepFake: Mitigating the spread of misinformation in India through a multimodal adversarial network","volume":"12","author":"Singh M. K.","year":"2024","unstructured":"M. K. Singh, J. Ahmed, K. K. Raghuvanshi, and M. A. Alam. 2024. IndDeepFake: Mitigating the spread of misinformation in India through a multimodal adversarial network. International Journal of Intelligent Systems and Applications in Engineering 12, 1S (2024), 81\u201397. https:\/\/ijisae.org\/index.php\/IJISAE\/article\/view\/3397","journal-title":"International Journal of Intelligent Systems and Applications in Engineering"},{"key":"e_1_3_2_47_2","unstructured":"A. Radford J. Wu R. Child D. Luan D. Amodei and I. Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1 8 (2019) 9. https:\/\/api.semanticscholar.org\/CorpusID:160025533"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3487553.3524650"},{"key":"e_1_3_2_49_2","unstructured":"C. Wardle. 2017. Fake news. It\u2019s complicated. First Draft. Retrieved August 15 2024 from https:\/\/firstdraftnews.org\/articles\/fake-news-complicated\/"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2004.11362"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3686797","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3686797","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:05:52Z","timestamp":1750291552000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3686797"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,21]]},"references-count":49,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2024,11,30]]}},"alternative-id":["10.1145\/3686797"],"URL":"https:\/\/doi.org\/10.1145\/3686797","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"value":"2375-4699","type":"print"},{"value":"2375-4702","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,21]]},"assertion":[{"value":"2024-06-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}