{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,17]],"date-time":"2026-11-17T19:33:31Z","timestamp":1794944011435,"version":"4.9.1"},"reference-count":38,"publisher":"IOP Publishing","issue":"13","license":[{"start":{"date-parts":[[2022,6,22]],"date-time":"2022-06-22T00:00:00Z","timestamp":1655856000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/publishingsupport.iopscience.iop.org\/iop-standard\/v1"},{"start":{"date-parts":[[2022,6,22]],"date-time":"2022-06-22T00:00:00Z","timestamp":1655856000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"the National Key Research and Development Project","award":["2019YFC0119402"],"award-info":[{"award-number":["2019YFC0119402"]}]},{"name":"Tianjin University Independent Innovation Fund","award":["2021XYZ-0011"],"award-info":[{"award-number":["2021XYZ-0011"]}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Phys. Med. Biol."],"published-print":{"date-parts":[[2022,7,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    <jats:italic>Objective<\/jats:italic>\n                    : Cervical cancer is one of the two biggest killers of women and early detection of cervical precancerous lesions can effectively improve the survival rate of patients. Manual diagnosis by combining colposcopic images and clinical examination results is the main clinical diagnosis method at present. Developing an intelligent diagnosis algorithm based on artificial intelligence is an inevitable trend to solve the objectification of diagnosis and improve the quality and efficiency of diagnosis.\n                    <jats:italic>Approach<\/jats:italic>\n                    : A colposcopic multimodal fusion convolutional neural network (CMF-CNN) was proposed for the classification of cervical lesions. Mask region convolutional neural network was used to detect the cervical region while the encoding network EfficientNet-B3 was introduced to extract the multimodal image features from the acetic image and iodine image. Finally, Squeeze-and-Excitation, Atrous Spatial Pyramid Pooling, and convolution block were also adopted to encode and fuse the patient\u2019s clinical text information.\n                    <jats:italic>Main results<\/jats:italic>\n                    : The experimental results showed that in 7106 cases of colposcopy, the accuracy, macro F1-score, macro-areas under the curve of the proposed model were 92.70%, 92.74%, 98.56%, respectively. They are superior to the mainstream unimodal image classification models.\n                    <jats:italic>Significance<\/jats:italic>\n                    : CMF-CNN proposed in this paper combines multimodal information, which has high performance in the classification of cervical lesions in colposcopy, so it can provide comprehensive diagnostic aid.\n                  <\/jats:p>","DOI":"10.1088\/1361-6560\/ac73d4","type":"journal-article","created":{"date-parts":[[2022,5,26]],"date-time":"2022-05-26T18:24:41Z","timestamp":1653589481000},"page":"135003","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["Colposcopic multimodal fusion for the classification of cervical lesions"],"prefix":"10.1088","volume":"67","author":[{"given":"Yinuo","family":"Fan","sequence":"first","affiliation":[{"name":"Tianjin University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huizhan","family":"Ma","sequence":"additional","affiliation":[{"name":"Tianjin University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuanbin","family":"Fu","sequence":"additional","affiliation":[{"name":"Tianjin University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoyun","family":"Liang","sequence":"additional","affiliation":[{"name":"Tianjin University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8511-7296","authenticated-orcid":true,"given":"Hui","family":"Yu","sequence":"additional","affiliation":[{"name":"Tianjin University"},{"name":"Tianjin University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuzhen","family":"Liu","sequence":"additional","affiliation":[{"name":"Affiliated Hospital of Weifang Medical University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2022,6,22]]},"reference":[{"key":"pmbac73d4bib1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2019.112951","type":"journal-article","article-title":"A fully-automated deep learning pipeline for cervical cancer classification","volume":"141","author":"Alyafeai","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"pmbac73d4bib2","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2019.101785","type":"journal-article","article-title":"Detection of cervical lesion region from colposcopic images based on feature reselection","volume":"57","author":"Bai","year":"2020","journal-title":"Biomed. Signal Process. Control"},{"key":"pmbac73d4bib3","doi-asserted-by":"publisher","first-page":"595","DOI":"10.3390\/pr8050595","type":"journal-article","article-title":"MobileNetV2 ensemble for cervical precancerous lesions classification","volume":"8","author":"Buiu","year":"2020","journal-title":"Processes"},{"key":"pmbac73d4bib4","doi-asserted-by":"publisher","first-page":"1505","DOI":"10.1109\/ISBI.2019.8759303","type":"conference-proceedings","article-title":"Multi-modal fusion learning for cervical dysplasia diagnosis","author":"Chen","year":"2019"},{"key":"pmbac73d4bib5","doi-asserted-by":"publisher","first-page":"794\u22128","DOI":"10.1109\/ITOEC.2018.8740393","type":"conference-proceedings","article-title":"A cervical intraepithelial neoplasia classification method using feature extraction and back propagation neural network","author":"Du","year":"2018"},{"key":"pmbac73d4bib6","doi-asserted-by":"publisher","first-page":"2261","DOI":"10.1109\/CVPR.2017.243","type":"conference-proceedings","article-title":"Densely connected convolutional networks","author":"Gao","year":"2017"},{"key":"pmbac73d4bib7","doi-asserted-by":"publisher","first-page":"408","DOI":"10.3969\/j.issn.1672-8467.2019.03.020","type":"journal-article","article-title":"Advances in the application of deep learning in medical imaging","volume":"46","author":"Gao","year":"2019","journal-title":"Fudan Univ. J. Med. Sci."},{"key":"pmbac73d4bib8","doi-asserted-by":"publisher","first-page":"2980","DOI":"10.48550\/arXiv.1703.06870","type":"conference-proceedings","article-title":"Mask R-CNN","author":"He","year":"2017"},{"key":"pmbac73d4bib9","doi-asserted-by":"publisher","first-page":"770\u22128","DOI":"10.1109\/CVPR.2016.90","type":"conference-proceedings","article-title":"Deep residual learning for image recognition","author":"He","year":"2016"},{"key":"pmbac73d4bib10","doi-asserted-by":"publisher","first-page":"2011","DOI":"10.1109\/TPAMI.2019.2913372","type":"journal-article","article-title":"Squeeze-and-excitation networks","volume":"42","author":"Hu","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"pmbac73d4bib11","doi-asserted-by":"publisher","first-page":"178","DOI":"10.1016\/j.ygyno.2014.12.022","type":"journal-article","article-title":"Use of primary high-risk human papillomavirus testing for cervical cancer screening: Interim clinical guidance","volume":"136","author":"Huh","year":"2015","journal-title":"Gynecol. Oncol."},{"key":"pmbac73d4bib12","first-page":"592","type":"journal-article","article-title":"Standardized colposcopy and the related precautions","volume":"36","author":"Kong","year":"2020","journal-title":"Chin. J. Pract. Gynecol. Obstetrics"},{"key":"pmbac73d4bib13","first-page":"46","type":"book","author":"Kurman","year":"2014","edition":"4th ed."},{"key":"pmbac73d4bib14","doi-asserted-by":"publisher","first-page":"3403","DOI":"10.1109\/TMI.2020.2994778","type":"journal-article","article-title":"Computer-aided cervical cancer diagnosis using time-lapsed colposcopic Images","volume":"39","author":"Li","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"pmbac73d4bib15","doi-asserted-by":"publisher","first-page":"1045","DOI":"10.21037\/atm-21-885","type":"journal-article","article-title":"Computer-aided diagnostic system based on deep learning for classifying colposcopy images","volume":"9","author":"Liu","year":"2021","journal-title":"Ann. Trans. Med."},{"key":"pmbac73d4bib16","doi-asserted-by":"publisher","first-page":"29616","DOI":"10.1109\/ACCESS.2020.2972610","type":"journal-article","article-title":"MDFI: multi-CNN decision feature integration for diagnosis of cervical precancerous lesions","volume":"8","author":"Luo","year":"2020","journal-title":"IEEE Access"},{"key":"pmbac73d4bib17","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1007\/978-3-030-01264-9_8","type":"journal-article","article-title":"ShuffleNet V2: practical guidelines for efficient CNN architecture design","volume":"11218","author":"Ma","year":"2018","journal-title":"Lecture Notes Comput. Sci."},{"key":"pmbac73d4bib18","doi-asserted-by":"publisher","first-page":"1602","DOI":"10.3892\/ol.2019.11214","type":"journal-article","article-title":"Application of deep learning to the classification of uterine cervical squamous epithelial lesion from colposcopy images combined with HPV types","volume":"19","author":"Miyagi","year":"2020","journal-title":"Oncol. Lett."},{"key":"pmbac73d4bib19","doi-asserted-by":"publisher","first-page":"867","DOI":"10.1109\/TMI.2011.2106796","type":"journal-article","article-title":"Domain-specific image analysis for cervical neoplasia detection based on conditional random fields","volume":"30","author":"Park","year":"2011","journal-title":"IEEE Trans. Med. Imaging"},{"key":"pmbac73d4bib20","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1007\/s00138-020-01063-8","type":"journal-article","article-title":"ColpoNet for automated cervical cancer screening using colposcopy images","volume":"31","author":"Saini","year":"2020","journal-title":"Mach. Vis. Appl."},{"key":"pmbac73d4bib21","doi-asserted-by":"publisher","first-page":"4510","DOI":"10.1109\/CVPR.2018.00474","type":"conference-proceedings","article-title":"MobileNetV2: inverted residuals and linear bottlenecks","author":"Sandler","year":"2018"},{"key":"pmbac73d4bib22","doi-asserted-by":"publisher","first-page":"3518","DOI":"10.3892\/ol.2018.7762","type":"journal-article","article-title":"Application of deep learning to the classification of images from colposcopy","volume":"15","author":"Sato","year":"2018","journal-title":"Oncol. Lett."},{"key":"pmbac73d4bib23","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","type":"journal-article","article-title":"Grad-CAM: visual explanations from deep networks via gradient-based localization","volume":"128","author":"Selvaraju","year":"2020","journal-title":"Int. Conf. Comput. Vis."},{"key":"pmbac73d4bib40","article-title":"Very deep convolutional networks for large-scale visual recognition","author":"Simonyan K","year":"2015","type":"preprint"},{"key":"pmbac73d4bib25","doi-asserted-by":"publisher","first-page":"209","DOI":"10.3322\/caac.21660","type":"journal-article","article-title":"Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"71","author":"Sung","year":"2021","journal-title":"CA Cancer J. Clin."},{"key":"pmbac73d4bib26","doi-asserted-by":"publisher","first-page":"2818","DOI":"10.1109\/CVPR.2016.308","type":"conference-proceedings","article-title":"Rethinking the inception architecture for computer vision","author":"Szegedy","year":"2016"},{"key":"pmbac73d4bib27","article-title":"EfficientNet: rethinking model scaling for convolutional neural networks","author":"Tan","year":"2019","type":"conference-proceedings"},{"key":"pmbac73d4bib29","first-page":"468","type":"journal-article","article-title":"Research progress in the application of artificial intelligence in cervical cancer screening","volume":"48","author":"Wang","year":"2021","journal-title":"Chin. J. Clin. Oncol."},{"key":"pmbac73d4bib30","doi-asserted-by":"publisher","first-page":"3484","DOI":"10.3760\/cma.j.cn112137-20200909-02606","type":"journal-article","article-title":"WHO\u2032s global strategy of cervical cancer elimination and the challenges and initiatives in China","volume":"100","author":"Xia","year":"2020","journal-title":"Natl. Med. J. China"},{"key":"pmbac73d4bib31","doi-asserted-by":"publisher","first-page":"5987","DOI":"10.1109\/CVPR.2017.634","type":"conference-proceedings","article-title":"Aggregated residual transformations for deep neural networks","author":"Xie","year":"2017"},{"key":"pmbac73d4bib32","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1016\/j.patcog.2016.09.027","type":"journal-article","article-title":"Multi-feature based benchmark for cervical dysplasia classification evaluation","volume":"63","author":"Xu","year":"2017","journal-title":"Pattern Recognit."},{"key":"pmbac73d4bib33","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1186\/s12916-020-01613-x","type":"journal-article","article-title":"The challenges of colposcopy for cervical cancer screening in LMICs and solutions by artificial intelligence","volume":"18","author":"Xue","year":"2020","journal-title":"BMC Med."},{"key":"pmbac73d4bib34","doi-asserted-by":"publisher","first-page":"406","DOI":"10.1186\/s12916-020-01860-y","type":"journal-article","article-title":"Development and validation of an artificial intelligence system for grading colposcopic impressions and guiding biopsies","volume":"18","author":"Xue","year":"2020","journal-title":"BMC Med."},{"key":"pmbac73d4bib35","doi-asserted-by":"publisher","first-page":"159","DOI":"10.12280\/gjfckx.20200748","type":"journal-article","article-title":"Diagnostic value of colposcopy for detecting cervical intraepithelial neoplasia: a meta-analysis","volume":"48","author":"Xue","year":"2021","journal-title":"J. Int. Obstetrics Gynecol."},{"key":"pmbac73d4bib36","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2021.102700","type":"journal-article","article-title":"Multi-state colposcopy image fusion for cervical precancerous lesion diagnosis using BF-CNN","volume":"68","author":"Yan","year":"2021","journal-title":"Biomed. Signal Process. Control"},{"key":"pmbac73d4bib37","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2020.104352","type":"journal-article","article-title":"MSCI: a multistate dataset for colposcopy image classification of cervical cancer screening","volume":"146","author":"Yu","year":"2021","journal-title":"Int. J. Med. Inform."},{"key":"pmbac73d4bib38","doi-asserted-by":"publisher","first-page":"11639","DOI":"10.1038\/s41598-020-68252-3","type":"journal-article","article-title":"The application of deep learning based diagnostic system to cervical squamous intraepithelial lesions recognition in colposcopy images","volume":"10","author":"Yuan","year":"2020","journal-title":"Sci. Rep."},{"key":"pmbac73d4bib39","doi-asserted-by":"publisher","first-page":"844","DOI":"10.1109\/JBHI.2019.2922682","type":"journal-article","article-title":"Automatic CIN grades prediction of sequential cervigram image using LSTM with multistate CNN features","volume":"24","author":"Yue","year":"2020","journal-title":"IEEE J. Biomed. Health Inform."}],"container-title":["Physics in Medicine &amp; Biology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ac73d4","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ac73d4\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ac73d4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ac73d4\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ac73d4\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ac73d4\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ac73d4\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ac73d4\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T12:09:37Z","timestamp":1783166977000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ac73d4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,22]]},"references-count":38,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2022,6,22]]},"published-print":{"date-parts":[[2022,7,7]]}},"URL":"https:\/\/doi.org\/10.1088\/1361-6560\/ac73d4","relation":{},"ISSN":["0031-9155","1361-6560"],"issn-type":[{"value":"0031-9155","type":"print"},{"value":"1361-6560","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,22]]},"assertion":[{"value":"Colposcopic multimodal fusion for the classification of cervical lesions","name":"article_title","label":"Article Title"},{"value":"Physics in Medicine & Biology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2022 Institute of Physics and Engineering in Medicine. All rights, including for text and data mining, AI training, and similar technologies, are reserved.","name":"copyright_information","label":"Copyright Information"},{"value":"2022-03-16","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-05-26","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-06-22","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}