{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,5]],"date-time":"2026-05-05T12:24:17Z","timestamp":1777983857076,"version":"3.51.4"},"reference-count":31,"publisher":"Oxford University Press (OUP)","issue":"16","license":[{"start":{"date-parts":[[2019,1,2]],"date-time":"2019-01-02T00:00:00Z","timestamp":1546387200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2018YFC0910500"],"award-info":[{"award-number":["2018YFC0910500"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61725302"],"award-info":[{"award-number":["61725302"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61671288"],"award-info":[{"award-number":["61671288"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["91530321"],"award-info":[{"award-number":["91530321"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61603161"],"award-info":[{"award-number":["61603161"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003399","name":"Science and Technology Commission of Shanghai Municipality","doi-asserted-by":"publisher","award":["16ZR1448700"],"award-info":[{"award-number":["16ZR1448700"]}],"id":[{"id":"10.13039\/501100003399","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003399","name":"Science and Technology Commission of Shanghai Municipality","doi-asserted-by":"publisher","award":["16JC1404300"],"award-info":[{"award-number":["16JC1404300"]}],"id":[{"id":"10.13039\/501100003399","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003399","name":"Science and Technology Commission of Shanghai Municipality","doi-asserted-by":"publisher","award":["17JC1403500"],"award-info":[{"award-number":["17JC1403500"]}],"id":[{"id":"10.13039\/501100003399","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,8,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>In the post-genomic era, image-based transcriptomics have received huge attention, because the visualization of gene expression distribution is able to reveal spatial and temporal expression pattern, which is significantly important for understanding biological mechanisms. The Berkeley Drosophila Genome Project has collected a large-scale spatial gene expression database for studying Drosophila embryogenesis. Given the expression images, how to annotate them for the study of Drosophila embryonic development is the next urgent task. In order to speed up the labor-intensive labeling work, automatic tools are highly desired. However, conventional image annotation tools are not applicable here, because the labeling is at the gene-level rather than the image-level, where each gene is represented by a bag of multiple related images, showing a multi-instance phenomenon, and the image quality varies by image orientations and experiment batches. Moreover, different local regions of an image correspond to different CV annotation terms, i.e. an image has multiple labels. Designing an accurate annotation tool in such a multi-instance multi-label scenario is a very challenging task.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>To address these challenges, we develop a new annotator for the fruit fly embryonic images, called AnnoFly. Driven by an attention-enhanced RNN model, it can weight images of different qualities, so as to focus on the most informative image patterns. We assess the new model on three standard datasets. The experimental results reveal that the attention-based model provides a transparent approach for identifying the important images for labeling, and it substantially enhances the accuracy compared with the existing annotation methods, including both single-instance and multi-instance learning methods.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>http:\/\/www.csbio.sjtu.edu.cn\/bioinf\/annofly\/<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/bty1064","type":"journal-article","created":{"date-parts":[[2018,12,27]],"date-time":"2018-12-27T12:08:40Z","timestamp":1545912520000},"page":"2834-2842","source":"Crossref","is-referenced-by-count":9,"title":["AnnoFly: annotating <i>Drosophila<\/i> embryonic images based on an attention-enhanced RNN model"],"prefix":"10.1093","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5720-773X","authenticated-orcid":false,"given":"Yang","family":"Yang","sequence":"first","affiliation":[{"name":"Center for Brain-Like Computing and Machine Intelligence, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China"},{"name":"Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mingyu","family":"Zhou","sequence":"additional","affiliation":[{"name":"Center for Brain-Like Computing and Machine Intelligence, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qingwei","family":"Fang","sequence":"additional","affiliation":[{"name":"School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4029-3325","authenticated-orcid":false,"given":"Hong-Bin","family":"Shen","sequence":"additional","affiliation":[{"name":"Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai, China"},{"name":"Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2019,1,2]]},"reference":[{"key":"2023062708582621000_bty1064-B1","doi-asserted-by":"crossref","first-page":"1127","DOI":"10.1038\/nmeth.2657","article-title":"Image-based transcriptomics in thousands of single human cells at single-molecule resolution","volume":"10","author":"Battich","year":"2013","journal-title":"Nat. Methods"},{"key":"2023062708582621000_bty1064-B2","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cviu.2007.09.014","article-title":"Speeded-up robust features (SURF)","volume":"110","author":"Bay","year":"2008","journal-title":"Comput. Vis. Image Underst"},{"key":"2023062708582621000_bty1064-B3","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.ymeth.2014.04.006","article-title":"Developments in in situ hybridisation","volume":"70","author":"Cassidy","year":"2014","journal-title":"Methods"},{"key":"2023062708582621000_bty1064-B4","first-page":"1884","volume-title":"Deep MIML network","author":"Feng","year":"2017"},{"key":"2023062708582621000_bty1064-B5","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1038\/msb.2009.102","article-title":"Systematic image-driven analysis of the spatial Drosophila embryonic expression landscape","volume":"6","author":"Frise","year":"2010","journal-title":"Mol. Syst. Biol"},{"key":"2023062708582621000_bty1064-B6","article-title":"A theoretically grounded application of dropout in recurrent neural networks","author":"Gal","year":"2015","journal-title":"arXiv, 1512, 05287"},{"key":"2023062708582621000_bty1064-B7","first-page":"5401","article-title":"Neural Turing Machines","volume":"1410","author":"Graves","year":"2014","journal-title":"arXiv: Neural Evol. Comput"},{"key":"2023062708582621000_bty1064-B8","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1109\/CVPR.2016.90","volume-title":"Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"He","year":"2016"},{"key":"2023062708582621000_bty1064-B9","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"2023062708582621000_bty1064-B10","first-page":"407","volume-title":"Drosophila gene expression pattern annotation using sparse features and term-term interactions","author":"Ji","year":"2009"},{"key":"2023062708582621000_bty1064-B11","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"arXiv, 1412, 6980"},{"key":"2023062708582621000_bty1064-B12","doi-asserted-by":"crossref","first-page":"3319","DOI":"10.1093\/bioinformatics\/btr567","article-title":"FlyExpress: visual mining of spatiotemporal patterns for genes and publications in Drosophila embryogenesis","volume":"27","author":"Kumar","year":"2011","journal-title":"Bioinformatics"},{"key":"2023062708582621000_bty1064-B13","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1038\/nature13395","article-title":"Genome-scale functional characterization of Drosophila developmental enhancers in vivo","volume":"512","author":"Kvon","year":"2014","journal-title":"Nature"},{"key":"2023062708582621000_bty1064-B14","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1109\/TCBB.2011.73","article-title":"Drosophila gene expression pattern annotation through multi-instance multi-label learning","volume":"9","author":"Li","year":"2012","journal-title":"IEEE\/ACM Trans. Comput. Biol. Bioinform"},{"key":"2023062708582621000_bty1064-B15","first-page":"02002.","article-title":"Focal loss for dense object detection","volume":"1708","author":"Lin","year":"2017","journal-title":"arXiv"},{"key":"2023062708582621000_bty1064-B16","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis"},{"key":"2023062708582621000_bty1064-B17","doi-asserted-by":"crossref","first-page":"i27","DOI":"10.1093\/bioinformatics\/btt206","article-title":"Automated annotation of gene expression image sequences via non-parametric factor analysis and conditional random fields","volume":"29","author":"Pruteanu-Malinici","year":"2013","journal-title":"Bioinformatics"},{"key":"2023062708582621000_bty1064-B18","doi-asserted-by":"crossref","first-page":"1003227.","DOI":"10.1371\/journal.pcbi.1003227","article-title":"GINI: from ISH images to gene interaction networks","volume":"9","author":"Puniyani","year":"2013","journal-title":"PLoS Comput. Biol"},{"key":"2023062708582621000_bty1064-B19","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis"},{"key":"2023062708582621000_bty1064-B20","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res"},{"key":"2023062708582621000_bty1064-B21","doi-asserted-by":"crossref","first-page":"350.","DOI":"10.1186\/1471-2105-14-350","article-title":"Image-level and group-level models for Drosophila gene expression pattern annotation","volume":"14","author":"Sun","year":"2013","journal-title":"BMC Bioinformatics"},{"key":"2023062708582621000_bty1064-B22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/gb-2002-3-12-research0088","article-title":"Systematic determination of patterns of gene expression during Drosophila embryogenesis","volume":"3","author":"Tomancak","year":"2002","journal-title":"Genome Biol"},{"key":"2023062708582621000_bty1064-B23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/gb-2007-8-7-r145","article-title":"Global analysis of patterns of gene expression during Drosophila embryogenesis","volume":"8","author":"Tomancak","year":"2007","journal-title":"Genome Biol"},{"key":"2023062708582621000_bty1064-B24","article-title":"Order Matters: Sequence to Sequence for Sets","author":"Vinyals","year":"2015","journal-title":"arXiv, 1511, 06391"},{"key":"2023062708582621000_bty1064-B25","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.proghi.2015.12.001","article-title":"In situ hybridisation: technologies and their application to understanding disease","volume":"50","author":"Warford","year":"2016","journal-title":"Prog. Histochem. Cyto"},{"key":"2023062708582621000_bty1064-B26","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1038\/nprot.2009.55","article-title":"Determination of gene expression patterns using high-throughput RNA in situ hybridization to whole-mount Drosophila embryos","volume":"4","author":"Weiszmann","year":"2009","journal-title":"Nat. Protoc"},{"key":"2023062708582621000_bty1064-B27","doi-asserted-by":"crossref","first-page":"4290","DOI":"10.1073\/pnas.1521171113","article-title":"Stability-driven nonnegative matrix factorization to interpret spatial gene expression and build local gene networks","volume":"113","author":"Wu","year":"2016","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023062708582621000_bty1064-B28","doi-asserted-by":"crossref","first-page":"107.","DOI":"10.1186\/1471-2105-13-107","article-title":"Learning sparse representations for fruit-fly gene expression pattern image annotation and retrieval","volume":"13","author":"Yuan","year":"2012","journal-title":"BMC Bioinformatics"},{"key":"2023062708582621000_bty1064-B29","first-page":"579","volume-title":"Deep convolutional neural networks for multi-instance multi-task learning","author":"Zeng","year":"2015"},{"key":"2023062708582621000_bty1064-B30","doi-asserted-by":"crossref","first-page":"1475","DOI":"10.1145\/2783258.2783304","article-title":"Deep model based transfer and multi-task learning for biological image analysis","author":"Zhang","year":"2015","journal-title":"the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining"},{"key":"2023062708582621000_bty1064-B31","doi-asserted-by":"crossref","first-page":"2291","DOI":"10.1016\/j.artint.2011.10.002","article-title":"Multi-instance multi-label learning","volume":"176","author":"Zhou","year":"2012","journal-title":"Artif. Intell"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/16\/2834\/50719203\/bty1064.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/16\/2834\/50719203\/bty1064.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,27]],"date-time":"2023-06-27T09:03:20Z","timestamp":1687856600000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/35\/16\/2834\/5270662"}},"subtitle":[],"editor":[{"given":"Jonathan","family":"Wren","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2019,1,2]]},"references-count":31,"journal-issue":{"issue":"16","published-print":{"date-parts":[[2019,8,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/bty1064","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,8,15]]},"published":{"date-parts":[[2019,1,2]]}}}