{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T18:14:56Z","timestamp":1769710496149,"version":"3.49.0"},"reference-count":21,"publisher":"SAGE Publications","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IFS"],"published-print":{"date-parts":[[2023,7,2]]},"abstract":"<jats:p>Recent advances in technology and devices have caused a data explosion on the Internet and on our home PCs. This data is predominantly obtained in various modalities (text, image, video, etc.) and is essential for e-commerce websites. The products on these websites have both images and descriptions in text form, making them multimodal in nature. Earlier categorization and information retrieval methods focused mostly on a single modality. This study employs multimodal data for classification using neutrosophic fuzzy sets for uncertainty management for information retrieval tasks. This effort utilizes image and text data and, inspired by past techniques of embedding text over an image, attempts to classify the images using neutrosophic classification algorithms. For classification tasks, Neutrosophic Convolutional Neural Networks (NCNNs) are used to learn feature representations of the produced images. We demonstrate how a pipeline based on NCNN can be utilized to learn representations of the innovative fusion method. Traditional convolutional neural networks are vulnerable to unknown noisy conditions in the test phase, and as a result, their performance for the classification of noisy data declines. Comparing our method against individual sources on two large-scale multi-modal categorization datasets yielded good results. In addition, we have compared our method to two well-known multi-modal fusion methodologies, namely early fusion and late fusion.<\/jats:p>","DOI":"10.3233\/jifs-223752","type":"journal-article","created":{"date-parts":[[2023,5,9]],"date-time":"2023-05-09T16:50:30Z","timestamp":1683651030000},"page":"1039-1055","source":"Crossref","is-referenced-by-count":6,"title":["Neutrosophic-CNN-based image and text fusion for multimodal classification"],"prefix":"10.1177","volume":"45","author":[{"given":"Mohd Anas","family":"Wajid","sequence":"first","affiliation":[{"name":"Department of Computer Science and Application, School of Engineering and Technology, Sharda University, Greater Noida, India"}]},{"given":"Aasim","family":"Zafar","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Aligarh Muslim University, Civil Lines, Aligarh, Uttar Pradesh, India"}]},{"given":"Hugo","family":"Terashima-Mar\u00edn","sequence":"additional","affiliation":[{"name":"School of Engineering and Sciences, Tecnol\u00f3gico de Monterrey, Monterrey, Mexico"}]},{"given":"Mohammad Saif","family":"Wajid","sequence":"additional","affiliation":[{"name":"School of Engineering and Sciences, Tecnol\u00f3gico de Monterrey, Monterrey, Mexico"}]}],"member":"179","reference":[{"issue":"1","key":"10.3233\/JIFS-223752_ref3","first-page":"8","article-title":"Multimodal Fusion: A Review, Taxonomy, Open Challenges, Research Roadmap and Future Directions","volume":"45","author":"Wajid","year":"2021","journal-title":"Neutrosophic Sets and Systems"},{"issue":"2","key":"10.3233\/JIFS-223752_ref12","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1109\/TPAMI.2018.2798607","article-title":"Multimodal machine learning: A survey and taxonomy","volume":"41","author":"Baltrusaitis","year":"2018","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"9","key":"10.3233\/JIFS-223752_ref15","doi-asserted-by":"crossref","first-page":"185","DOI":"10.3390\/sym9090185","article-title":"An efficient image segmentation algorithm using neutrosophic graph cut","volume":"9","author":"Guo","year":"2017","journal-title":"Symmetry"},{"issue":"8","key":"10.3233\/JIFS-223752_ref32","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1016\/j.jvcir.2010.08.005","article-title":"Web image concept annotation with better understanding of tags and visual features","volume":"21","author":"Gao","year":"2010","journal-title":"Journal of Visual Communication and Image Representation"},{"issue":"5","key":"10.3233\/JIFS-223752_ref34","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1016\/j.cviu.2012.10.009","article-title":"Multimodal recognition of visual concepts using histograms of textual concepts and selective weighted late fusion scheme","volume":"117","author":"Liu","year":"2013","journal-title":"Computer Vision and Image Understanding"},{"key":"10.3233\/JIFS-223752_ref39","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.knosys.2019.01.019","article-title":"Image\u2013text sentiment analysis via deep multimodal attentive fusion","volume":"167","author":"Huang","year":"2019","journal-title":"Knowledge-Based Systems"},{"issue":"7","key":"10.3233\/JIFS-223752_ref43","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.1109\/TMM.2013.2267205","article-title":"Multimodal saliency and fusion for movie summarization based on aural, visual, and textual attention","volume":"15","author":"Evangelopoulos","year":"2013","journal-title":"IEEE Transactions on Multimedia"},{"issue":"5","key":"10.3233\/JIFS-223752_ref48","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1016\/j.patcog.2008.10.002","article-title":"New neutrosophic approach to image segmentation","volume":"42","author":"Guo","year":"2009","journal-title":"Pattern Recognition"},{"key":"10.3233\/JIFS-223752_ref49","first-page":"80","article-title":"Neutrosophic Supra Topological Applications in Data Mining Process","volume":"27","author":"Jayaparthasarathy","year":"2019","journal-title":"Neutrosophic Sets and Systems"},{"issue":"6","key":"10.3233\/JIFS-223752_ref53","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Communications of the ACM"},{"issue":"1","key":"10.3233\/JIFS-223752_ref55","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10462-009-9124-7","article-title":"Ensemble-based classifiers","volume":"33","author":"Rokach","year":"2010","journal-title":"Artificial Intelligence Review"},{"key":"10.3233\/JIFS-223752_ref59","unstructured":"Zhang X. , Zhao J. and LeCun Y. , Character-level convolutional networks for text classification, Advances in Neural Information Processing Systems 28 (2015)."},{"key":"10.3233\/JIFS-223752_ref64","doi-asserted-by":"crossref","first-page":"106830","DOI":"10.1016\/j.measurement.2019.07.058","article-title":"Dogantekin E Brain tumor detection based on convolutional neural network with neutrosophic expert maximum fuzzy sure entropy","volume":"147","author":"\u00d6zyurt","year":"2019","journal-title":"Measurement"},{"key":"10.3233\/JIFS-223752_ref65","doi-asserted-by":"crossref","first-page":"1603","DOI":"10.1007\/s40815-016-0250-5","article-title":"NCM-Based Raga Classification using musical features","volume":"19","author":"Anitha","year":"2017","journal-title":"International Journal of Fuzzy Systems"},{"issue":"2","key":"10.3233\/JIFS-223752_ref66","doi-asserted-by":"crossref","first-page":"11","DOI":"10.21928\/uhdjst.v6n2y2022.pp11-18","article-title":"COVID-19 Classification based on Neutrosophic Set Transfer Learning Approach","volume":"6","author":"Hamaamin","year":"2022","journal-title":"UHD Journal of Science and Technology"},{"key":"10.3233\/JIFS-223752_ref67","doi-asserted-by":"crossref","unstructured":"Wady S.H. , Yousif R.Z. and Hasan H.R. , A novel intelligent system for brain tumor diagnosis based on a composite neutrosophic-slantlet transform domain for statistical texture feature extraction, BioMed Research International 2020 (2020).","DOI":"10.1155\/2020\/8125392"},{"issue":"1","key":"10.3233\/JIFS-223752_ref69","first-page":"1","article-title":"Violence detection approach based on cloud data and Neutrosophic cognitive maps","volume":"11","author":"Wajid","year":"2022","journal-title":"Journal of Cloud Computing"},{"key":"10.3233\/JIFS-223752_ref70","first-page":"15","article-title":"The Importance of Indeterminate and Unknown Factors in Nourishing Crime: A Case Study of South Africa Using Neutrosophy","volume":"41","author":"Wajid","year":"2021","journal-title":"Neutrosophic Sets and Systems"},{"issue":"2","key":"10.3233\/JIFS-223752_ref72","doi-asserted-by":"crossref","first-page":"86","DOI":"10.54216\/IJNS.200207","article-title":"NEP-\u2019s Implementation & Execution: A Study Conducted Using Neutrosophic PESTEL Analysis","volume":"20","author":"Mohd Yasir","year":"2023","journal-title":"International Journal of Neutrosophic Science"},{"key":"10.3233\/JIFS-223752_ref74","first-page":"111","article-title":"Neutrosophic Sociogram Approach to Neutrosophic Cognitive Maps in Swift Language","volume":"49","author":"Priya","year":"2022","journal-title":"Neutrosophic Sets and Systems"},{"key":"10.3233\/JIFS-223752_ref75","doi-asserted-by":"crossref","first-page":"108780","DOI":"10.1016\/j.asoc.2022.108780","article-title":"COVID-WideNet\u2014 A capsule network for COVID-19 detection","volume":"122","author":"Gupta","year":"2022","journal-title":"Applied Soft Computing"}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/JIFS-223752","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T07:26:58Z","timestamp":1769671618000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/JIFS-223752"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,2]]},"references-count":21,"journal-issue":{"issue":"1"},"URL":"https:\/\/doi.org\/10.3233\/jifs-223752","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,2]]}}}