{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T02:38:10Z","timestamp":1787020690333,"version":"build-2736575974"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,3,16]],"date-time":"2024-03-16T00:00:00Z","timestamp":1710547200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,16]],"date-time":"2024-03-16T00:00:00Z","timestamp":1710547200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100020595","name":"National Science and Technology Council","doi-asserted-by":"publisher","award":["107-2923-E-007-004"],"award-info":[{"award-number":["107-2923-E-007-004"]}],"id":[{"id":"10.13039\/100020595","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100020595","name":"National Science and Technology Council","doi-asserted-by":"publisher","award":["109-2636-B-007-002"],"award-info":[{"award-number":["109-2636-B-007-002"]}],"id":[{"id":"10.13039\/100020595","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Behav Brain Funct"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Fruit fly courtship behaviors composed of a series of actions have always been an important model for behavioral research. While most related studies have focused only on total courtship behaviors, specific courtship elements have often been underestimated. Identifying these courtship element details is extremely labor intensive and would largely benefit from an automatic recognition system. To address this issue, in this study, we established a vision-based fly courtship behavior recognition system. The system based on the proposed image processing methods can precisely distinguish body parts such as the head, thorax, and abdomen and automatically recognize specific courtship elements, including orientation, singing, attempted copulation, copulation and tapping, which was not detectable in previous studies. This system, which has high identity tracking accuracy (99.99%) and high behavioral element recognition rates (&gt;\u200997.35%), can ensure correct identification even when flies completely overlap. Using this newly developed system, we investigated the total courtship time, and proportion, and transition of courtship elements in flies across different ages and found that male flies adjusted their courtship strategy in response to their physical condition. We also identified differences in courtship patterns between males with and without successful copulation. Our study therefore demonstrated how image processing methods can be applied to automatically recognize complex animal behaviors. The newly developed system will largely help us investigate the details of fly courtship in future research.<\/jats:p>","DOI":"10.1186\/s12993-024-00231-4","type":"journal-article","created":{"date-parts":[[2024,3,16]],"date-time":"2024-03-16T06:01:54Z","timestamp":1710568914000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["An automatic system for recognizing fly courtship patterns via an image processing method"],"prefix":"10.1186","volume":"20","author":[{"given":"Ching-Hsin","family":"Chen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu-Chiao","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sheng-Hao","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tsung-Han","family":"Kuo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hung-Yin","family":"Tsai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,3,16]]},"reference":[{"key":"231_CR1","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.1328","author":"K Branson","year":"2009","unstructured":"Branson K, Robie AA, Bender J, Perona P, Dickinson MH. High-throughput ethomics in large groups of Drosophila. Nat Methods. 2009. https:\/\/doi.org\/10.1038\/nmeth.1328.","journal-title":"Nat Methods"},{"key":"231_CR2","doi-asserted-by":"publisher","DOI":"10.1111\/gbb.12598","author":"DB Brenman-Suttner","year":"2020","unstructured":"Brenman-Suttner DB, Yost RT, Frame AK, Robinson JW, Moehring AJ, Simon AF. Social behavior and aging: a fly model. Genes Brain Behav. 2020. https:\/\/doi.org\/10.1111\/gbb.12598.","journal-title":"Genes Brain Behav"},{"key":"231_CR3","doi-asserted-by":"publisher","DOI":"10.3390\/s150819369","author":"R Chao","year":"2015","unstructured":"Chao R, Mac\u00eda-V\u00e1zquez G, Zalama E, G\u00f3mez-Garc\u00eda JB, Per\u00e1n JR. Automated tracking of drosophila specimens. Sensors (Switzerland). 2015. https:\/\/doi.org\/10.3390\/s150819369.","journal-title":"Sensors (Switzerland)"},{"key":"231_CR4","doi-asserted-by":"publisher","DOI":"10.1093\/jisesa\/ieab079","author":"CH Chen","year":"2021","unstructured":"Chen CH, Chiang AS, Tsai HY. Three-dimensional tracking of multiple small insects by a single camera. J Insect Sci. 2021. https:\/\/doi.org\/10.1093\/jisesa\/ieab079.","journal-title":"J Insect Sci"},{"key":"231_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2015.07.025","author":"EJ Clowney","year":"2015","unstructured":"Clowney EJ, Iguchi S, Bussell JJ, Scheer E, Ruta V. Multimodal chemosensory circuits controlling male courtship in Drosophila. Neuron. 2015. https:\/\/doi.org\/10.1016\/j.neuron.2015.07.025.","journal-title":"Neuron"},{"key":"231_CR6","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.1310","author":"H Dankert","year":"2009","unstructured":"Dankert H, Wang L, Hoopfer ED, Anderson DJ, Perona P. Automated monitoring and analysis of social behavior in Drosophila. Nat Methods. 2009. https:\/\/doi.org\/10.1038\/nmeth.1310.","journal-title":"Nat Methods"},{"key":"231_CR7","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1146\/annurev.genet.34.1.205","volume":"34","author":"RJ Greenspan","year":"2000","unstructured":"Greenspan RJ, Ferveur J-F. Courtship in DROSOPHILA. Annu Rev Genet. 2000;34:205\u201332. https:\/\/doi.org\/10.1146\/annurev.genet.34.1.205.","journal-title":"Annu Rev Genet"},{"key":"231_CR8","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.172060","author":"M Grillet","year":"2018","unstructured":"Grillet M, Ferveur JF, Everaerts C. Behavioural elements and sensory cues involved in sexual isolation between Drosophila melanogaster strains. R Soc Open Sci. 2018. https:\/\/doi.org\/10.1098\/rsos.172060.","journal-title":"R Soc Open Sci"},{"key":"231_CR9","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1177\/0956797614567339","volume":"26","author":"JK Hartshorne","year":"2015","unstructured":"Hartshorne JK, Germine LT. When does cognitive functioning peak? The asynchronous rise and fall of different cognitive abilities across the life span. Psychol Sci. 2015;26:451\u20137. https:\/\/doi.org\/10.1177\/0956797614567339.","journal-title":"Psychol Sci"},{"key":"231_CR10","doi-asserted-by":"publisher","DOI":"10.3109\/01677063.2012.729626","author":"A Iyengar","year":"2012","unstructured":"Iyengar A, Imoehl J, Ueda A, Nirschl J, Wu CF. Automated quantification of locomotion, social interaction, and mate preference in Drosophila mutants. J Neurogenet. 2012. https:\/\/doi.org\/10.3109\/01677063.2012.729626.","journal-title":"J Neurogenet"},{"key":"231_CR11","doi-asserted-by":"publisher","first-page":"67972","DOI":"10.1109\/ACCESS.2019.2917000","volume":"7","author":"Z Jiang","year":"2019","unstructured":"Jiang Z, Chazot PL, Celebi ME, Crookes D, Jiang R. Social behavioral phenotyping of Drosophila with a 2D\u20133D hybrid CNN framework. IEEE Access. 2019;7:67972\u201382. https:\/\/doi.org\/10.1109\/ACCESS.2019.2917000.","journal-title":"IEEE Access"},{"key":"231_CR12","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1038\/nmeth.2281","volume":"10","author":"M Kabra","year":"2013","unstructured":"Kabra M, Robie AA, Rivera-Alba M, Branson S, Branson K. JAABA: interactive machine learning for automatic annotation of animal behavior. Nat Methods. 2013;10:64\u20137. https:\/\/doi.org\/10.1038\/nmeth.2281.","journal-title":"Nat Methods"},{"key":"231_CR13","doi-asserted-by":"publisher","DOI":"10.1111\/j.1558-5646.1982.tb05003.x","author":"M Kirkpatrick","year":"1982","unstructured":"Kirkpatrick M. Sexual selection and the evolution of female choice. Evolution (N Y). 1982. https:\/\/doi.org\/10.1111\/j.1558-5646.1982.tb05003.x.","journal-title":"Evolution (N Y)"},{"key":"231_CR14","doi-asserted-by":"publisher","DOI":"10.1088\/1478-3975\/aa5c50","author":"U Klibaite","year":"2017","unstructured":"Klibaite U, Berman GJ, Cande J, Stern DL, Shaevitz JW. An unsupervised method for quantifying the behavior of paired animals. Phys Biol. 2017. https:\/\/doi.org\/10.1088\/1478-3975\/aa5c50.","journal-title":"Phys Biol"},{"key":"231_CR15","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2022.01.024","author":"J Ning","year":"2022","unstructured":"Ning J, Li Z, Zhang X, Wang J, Chen D, Liu Q, et al. Behavioral signatures of structured feature detection during courtship in Drosophila. Curr Biol. 2022. https:\/\/doi.org\/10.1016\/j.cub.2022.01.024.","journal-title":"Curr Biol"},{"key":"231_CR16","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2018.06.038","author":"A O\u2019Sullivan","year":"2018","unstructured":"O\u2019Sullivan A, Lindsay T, Prudnikova A, Erdi B, Dickinson M, von Philipsborn AC. Multifunctional wing motor control of song and flight. Curr Biol. 2018. https:\/\/doi.org\/10.1016\/j.cub.2018.06.038.","journal-title":"Curr Biol"},{"key":"231_CR17","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-022-01426-1","author":"TD Pereira","year":"2022","unstructured":"Pereira TD, Tabris N, Matsliah A, Turner DM, Li J, Ravindranath S, et al. SLEAP: a deep learning system for multi-animal pose tracking. Nat Methods. 2022. https:\/\/doi.org\/10.1038\/s41592-022-01426-1.","journal-title":"Nat Methods"},{"key":"231_CR18","doi-asserted-by":"publisher","DOI":"10.3389\/fnbeh.2021.809665","author":"S Qu","year":"2022","unstructured":"Qu S, Zhu Q, Zhou H, Gao Y, Wei Y, Ma Y, et al. EasyFlyTracker: a simple video tracking python package for analyzing adult Drosophila locomotor and sleep activity to facilitate revealing the effect of psychiatric drugs. Front Behav Neurosci. 2022. https:\/\/doi.org\/10.3389\/fnbeh.2021.809665.","journal-title":"Front Behav Neurosci"},{"key":"231_CR19","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1242\/jeb.141457","volume":"220","author":"P Ramdya","year":"2017","unstructured":"Ramdya P, Schneider J, Levine JD. The neurogenetics of group behavior in Drosophila melanogaster. J Exp Biol. 2017;220:35\u201341. https:\/\/doi.org\/10.1242\/jeb.141457.","journal-title":"J Exp Biol"},{"key":"231_CR20","doi-asserted-by":"publisher","DOI":"10.4161\/fly.24110","author":"A Reza","year":"2013","unstructured":"Reza A, Mhatre SD, Morrison JC, Utreja S, Saunders AJ, Breen DE, et al. Automated analysis of courtship suppression learning and memory in Drosophila melanogaster. Fly (Austin). 2013. https:\/\/doi.org\/10.4161\/fly.24110.","journal-title":"Fly (Austin)"},{"key":"231_CR21","unstructured":"Ross Mckinney. Visual and chemosensory pathways associated with male courtship decisions in Drosophila melanogaster."},{"key":"231_CR22","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-40952-5","author":"KM Scaplen","year":"2019","unstructured":"Scaplen KM, Mei NJ, Bounds HA, Song SL, Azanchi R, Kaun KR. Automated real-time quantification of group locomotor activity in Drosophila melanogaster. Sci Rep. 2019. https:\/\/doi.org\/10.1038\/s41598-019-40952-5.","journal-title":"Sci Rep"},{"key":"231_CR23","doi-asserted-by":"publisher","DOI":"10.1186\/1687-5281-2013-64","author":"C Schusterreiter","year":"2013","unstructured":"Schusterreiter C, Grossmann W. A two-fly tracker that solves occlusions by dynamic programming: computational analysis of Drosophila courtship behaviour. EURASIP J Image Video Process. 2013. https:\/\/doi.org\/10.1186\/1687-5281-2013-64.","journal-title":"EURASIP J Image Video Process"},{"key":"231_CR24","doi-asserted-by":"publisher","DOI":"10.1093\/gerona\/glab048","author":"D Tavoian","year":"2021","unstructured":"Tavoian D, Lozier NR, de Lacalle S. Age of peak performance differs by functional task in mice tracked over 2 years. J Gerontol A Biol Sci Med Sci. 2021. https:\/\/doi.org\/10.1093\/gerona\/glab048.","journal-title":"J Gerontol A Biol Sci Med Sci"},{"key":"231_CR25","doi-asserted-by":"publisher","DOI":"10.2307\/2409128","author":"CE Taylor","year":"1988","unstructured":"Taylor CE, Kekic V. Sexual selection in a natural population of Drosophila melanogaster. Evolution (N Y). 1988. https:\/\/doi.org\/10.2307\/2409128.","journal-title":"Evolution (N Y)"},{"key":"231_CR26","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0034784","author":"HY Tsai","year":"2012","unstructured":"Tsai HY, Huang YW. Image tracking study on courtship behavior of drosophila. PLoS ONE. 2012. https:\/\/doi.org\/10.1371\/journal.pone.0034784.","journal-title":"PLoS ONE"},{"key":"231_CR27","doi-asserted-by":"publisher","DOI":"10.1007\/s11222-007-9033-z","author":"U von Luxburg","year":"2007","unstructured":"von Luxburg U. A tutorial on spectral clustering. Stat Comput. 2007. https:\/\/doi.org\/10.1007\/s11222-007-9033-z.","journal-title":"Stat Comput"},{"key":"231_CR28","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2011.01.011","author":"AC von Philipsborn","year":"2011","unstructured":"von Philipsborn AC, Liu T, Yu JY, Masser C, Bidaye SS, Dickson BJ. Neuronal control of Drosophila courtship song. Neuron. 2011. https:\/\/doi.org\/10.1016\/j.neuron.2011.01.011.","journal-title":"Neuron"},{"key":"231_CR29","doi-asserted-by":"publisher","DOI":"10.1016\/0022-5193(75)90111-3","author":"A Zahavi","year":"1975","unstructured":"Zahavi A. Mate selection-a selection for a handicap. J Theor Biol. 1975. https:\/\/doi.org\/10.1016\/0022-5193(75)90111-3.","journal-title":"J Theor Biol"}],"container-title":["Behavioral and Brain Functions"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12993-024-00231-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12993-024-00231-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12993-024-00231-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,16]],"date-time":"2024-03-16T06:04:14Z","timestamp":1710569054000},"score":1,"resource":{"primary":{"URL":"https:\/\/behavioralandbrainfunctions.biomedcentral.com\/articles\/10.1186\/s12993-024-00231-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,16]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["231"],"URL":"https:\/\/doi.org\/10.1186\/s12993-024-00231-4","relation":{},"ISSN":["1744-9081"],"issn-type":[{"value":"1744-9081","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,16]]},"assertion":[{"value":"14 November 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 March 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Drosophila are currently NOT subject to any ethical restrictions in Taiwan.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"5"}}