{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T10:07:44Z","timestamp":1761646064611,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2017,4,21]],"date-time":"2017-04-21T00:00:00Z","timestamp":1492732800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No.61272373","No.61202254","No.71303031"],"award-info":[{"award-number":["No.61272373","No.61202254","No.71303031"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["No.DC13010313","No.DC201502030202"],"award-info":[{"award-number":["No.DC13010313","No.DC201502030202"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Compound figure detection on figures and associated captions is the first step to making medical figures from biomedical literature available for further analysis. The performance of traditional methods is limited to the choice of hand-engineering features and prior domain knowledge. We train multiple convolutional neural networks (CNNs), long short-term memory (LSTM) networks, and gated recurrent unit (GRU) networks on top of pre-trained word vectors to learn textual features from captions and employ deep CNNs to learn visual features from figures. We then identify compound figures by combining textual and visual prediction. Our proposed architecture obtains remarkable performance in three run types\u2014textual, visual and mixed\u2014and achieves better performance in ImageCLEF2015 and ImageCLEF2016.<\/jats:p>","DOI":"10.3390\/info8020048","type":"journal-article","created":{"date-parts":[[2017,4,21]],"date-time":"2017-04-21T10:59:30Z","timestamp":1492772370000},"page":"48","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Assembling Deep Neural Networks for Medical Compound Figure Detection"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7827-3143","authenticated-orcid":false,"given":"Yuhai","family":"Yu","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China"},{"name":"School of Computer Science &amp; Engineering, Dalian Minzu University, Dalian 116600, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7191-2280","authenticated-orcid":false,"given":"Hongfei","family":"Lin","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiana","family":"Meng","sequence":"additional","affiliation":[{"name":"School of Computer Science &amp; Engineering, Dalian Minzu University, Dalian 116600, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3675-1020","authenticated-orcid":false,"given":"Xiaocong","family":"Wei","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Dalian University of Foreign Language, Dalian 116044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhehuan","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,4,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Lu, Z. (2011). PubMed and beyond: A survey of web tools for searching biomedical literature. Database, 2011.","DOI":"10.1093\/database\/baq036"},{"key":"ref_2","unstructured":"M\u00fcller, H., Despont-Gros, C., Hersh, W., Jensen, J., Lovis, C., and Geissbuhler, A. (2006, January 27\u201330). Health care professionals\u2019 image use and search behavior. Proceedings of the Medical Informatics Europe, Maastricht, The Netherlands."},{"key":"ref_3","unstructured":"De Herrera, A.G.S., Kalpathy-Cramer, J., Fushman, D.D., Antani, S., and M\u00fcller, H. (2013, January 23\u201326). Overview of the ImageCLEF 2013 medical tasks. Proceedings of the Working Notes of CLEF, Valencia, Spain."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"168","DOI":"10.5626\/JCSE.2012.6.2.168","article-title":"Design and development of a multimodal biomedical information retrieval system","volume":"6","author":"Antani","year":"2012","journal-title":"J. Comput. Sci. Eng."},{"key":"ref_5","unstructured":"De Herrera, A.G.S., M\u00fcller, H., and Bromuri, S. (2015, January 8\u201311). Overview of the ImageCLEF 2015 medical classification task. Proceedings of the Working Notes of CLEF, Toulouse, France."},{"key":"ref_6","unstructured":"De Herrera, A.G.S., Schaer, R., Bromuri, S., and M\u00fcller, H. (2016, January 5\u20138). Overview of the ImageCLEF 2016 medical task. Proceedings of the Working Notes of CLEF, \u00c9vora, Portugal."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Plataki, M., Tzortzaki, E., Lambiri, I., Giannikaki, E., Ernst, A., and Siafakas, N.M. (2006). Severe airway stenosis associated with Crohn\u2019s disease: Case report. BMC Pulm Med., 6.","DOI":"10.1186\/1471-2466-6-7"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Theegarten, D., Sachse, K., Mentrup, B., Fey, K., Hotzel, H., and Anhenn, O. (2008). Chlamydophila spp. infection in horses with recurrent airway obstruction: Similarities to human chronic obstructive disease. BMC Respir. Res., 9.","DOI":"10.1186\/1465-9921-9-14"},{"key":"ref_9","unstructured":"Pelka, O., and Friedrich, C.M. (2015, January 8\u201311). FHDO biomedical computer science group at medical classification task of ImageCLEF 2015. Proceedings of the Working Notes of CLEF, Toulouse, France."},{"key":"ref_10","unstructured":"Wang, X., Jiang, X., Kolagunda, A., Shatkay, H., and Kambhamettu, C. (2015, January 8\u201311). CIS UDEL Working Notes on ImageCLEF 2015: Compound figure detection task. Proceedings of the Working Notes of CLEF 2015, Toulouse, France."},{"key":"ref_11","unstructured":"De Herrera, A.G.S., Markonis, D., Schaer, R., Eggel, I., and M\u00fcller, H. (2013, January 23\u201326). The medGIFT Group in ImageCLEFmed 2013. Proceedings of the Working Notes of CLEF, Valencia, Spain."},{"key":"ref_12","unstructured":"Csurka, G., Dance, C., Fan, L., Willamowski, J., and Bray, C. (2004, January 11\u201314). Visual Categorization with Bags of Keypoints. Proceedings of the Workshop on Statistical Learning in Computer Vision, European Conference on Computer Vision, Prague, Czech Republic."},{"key":"ref_13","unstructured":"De Herrera, A.G.S., Markonis, D., and M\u00fcller, H. (2012, January 1). Bag-of-Colors for Biomedical Document Image Classification. Proceedings of Medical Content-Based Retrieval for Clinical Decision Support (MCBR-CDS), Nice, France."},{"key":"ref_14","unstructured":"Zhou, X. (2011). Grid-Based Medical Image Retrieval Using Local Features. [Ph.D. Thesis, University of Geneva]."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_16","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_20","first-page":"2493","article-title":"Natural language processing (almost) from scratch","volume":"12","author":"Collobert","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Tai, K.S., Socher, R., and Manning, C.D. (2015). Improved semantic representations from tree-structured long short-term memory networks. arXiv.","DOI":"10.3115\/v1\/P15-1150"},{"key":"ref_23","unstructured":"Gal, Y.A. (2015). Theoretically Grounded Application of Dropout in Recurrent Neural Networks. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Bahdanau, D., and Bengio, Y. (2014). On the Properties of Neural Machine Translation: Encoder\u2013Decoder Approaches. arXiv.","DOI":"10.3115\/v1\/W14-4012"},{"key":"ref_25","unstructured":"Mikolov, T., and Dean, J. (2013, January 5\u201310). Distributed representations of words and phrases and their compositionality. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J.R., Bethard, S., and McClosky, D. (2014, January 22\u201327). The stanford corenlp natural language processing toolkit. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Baltimore, MD, USA.","DOI":"10.3115\/v1\/P14-5010"},{"key":"ref_27","unstructured":"Glorot, X., and Bengio, Y. (2010, January 13\u201315). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS), Sardinia, Italy."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yu, Y., Lin, H., Meng, J., and Zhao, Z. (2016). Visual and Textual Sentiment Analysis of a Microblog Using Deep Convolutional Neural Networks. Algorithms, 9.","DOI":"10.3390\/a9020041"},{"key":"ref_29","unstructured":"Tieleman, T., and Hinton, G. (2017, April 21). Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, Technical Report. Available online: https:\/\/zh.coursera.org\/learn\/neural-networks\/lecture\/YQHki\/rmsprop-divide-the-gradient-by-a-running-average-of-its-recent-magnitude."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Graves, A. (2013). Generating sequences with recurrent neural networks. arXiv.","DOI":"10.1007\/978-3-642-24797-2_3"},{"key":"ref_31","unstructured":"Kingma, D., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_32","first-page":"5403","article-title":"Modality classification for medical images using multiple deep convolutional neural networks","volume":"11","author":"Yu","year":"2015","journal-title":"JCIS"},{"key":"ref_33","unstructured":"Wan, L., Zeiler, M., Zhang, S., Cun, Y.L., and Fergus, R. (2013, January 16\u201321). Regularization of neural networks using dropconnect. Proceedings of the 30th International Conference on Machine Learning (ICML-13), Atlanta, GA, USA."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/8\/2\/48\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:33:07Z","timestamp":1760207587000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/8\/2\/48"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,4,21]]},"references-count":33,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2017,6]]}},"alternative-id":["info8020048"],"URL":"https:\/\/doi.org\/10.3390\/info8020048","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2017,4,21]]}}}