{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T19:14:00Z","timestamp":1773688440955,"version":"3.50.1"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"35","license":[{"start":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T00:00:00Z","timestamp":1727049600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T00:00:00Z","timestamp":1727049600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"CASDAR","award":["18ART1831"],"award-info":[{"award-number":["18ART1831"]}]},{"name":"Mines Paris - PSL"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In this paper, we address an essential problem related to object detection and image processing: detecting objects potentially nested in other ones. This problem exists particularly in the beekeeping sector: detecting varroa parasites on bees. Indeed, beekeepers must ensure the level of infestation of their apiaries by the varroa parasite which settles on the backs of bees. As far as we know, there is no yet a published approach to deal with nested object detection using only one neural network trained on two  different datasets. We propose an approach that fills this gap. Therefore, we improve the accuracy and the efficiency of bee and varroa detection task. Our work is based on deep learning, more precisely Mask R-CNN neural network. Instead of segmenting detected objects (bees), we segment internal objects (varroas). We add a branch to Faster R-CNN to segment internal objects. We extract relevant features for internal object segmentation and suggest efficient method for training the neural network on two different datasets. Our experiments are based on a set of images of bee frames, containing annotated bees and varroa mites. Due to differences in occurrence rates, two different sets were created. After carrying out experiments, we ended up with a single neural network capable of detecting two nested objects without decreasing accuracy compared to two separate neural networks. Our approach, compared to traditional separate neural networks, improves varroa detection accuracy by 1.9%, reduces infestation level prediction error by 0.22%, and reduces execution time by 28% and model memory by 23%. In our approach, we extract Res4 (a layer of the ResNet neural network) features for varroa segmentation, which improves detection accuracy by 11% compared to standard FPN extraction. Thus, we suggest a new approach that detects nested objects more accurately than two separate network approaches.<\/jats:p>","DOI":"10.1007\/s00521-024-10393-x","type":"journal-article","created":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T09:01:53Z","timestamp":1727082113000},"page":"22587-22609","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Nested object detection using mask R-CNN: application to bee and varroa detection"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1139-2789","authenticated-orcid":false,"given":"Yassine","family":"Kriouile","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Corinne","family":"Ancourt","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katarzyna","family":"Wegrzyn-Wolska","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lamine","family":"Bougueroua","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,9,23]]},"reference":[{"key":"10393_CR1","unstructured":"Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, vol 28"},{"key":"10393_CR2","doi-asserted-by":"publisher","unstructured":"He K, Gkioxari G, Doll\u00e1r P, Girshick R (2017) Mask R-CNN. In: 2017 IEEE international conference on computer vision (ICCV), pp 2980\u20132988. https:\/\/doi.org\/10.1109\/ICCV.2017.322","DOI":"10.1109\/ICCV.2017.322"},{"issue":"8","key":"10393_CR3","doi-asserted-by":"publisher","first-page":"717","DOI":"10.3390\/insects12080717","volume":"12","author":"T Sipos","year":"2021","unstructured":"Sipos T, Donk\u00f3 T, J\u00f3cs\u00e1k I, Keszthelyi S (2021) Study of morphological features in pre-imaginal honey bee impaired by varroa destructor by means of computer tomography. Insects 12(8):717. https:\/\/doi.org\/10.3390\/insects12080717","journal-title":"Insects"},{"key":"10393_CR4","doi-asserted-by":"crossref","unstructured":"Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector. Lecture notes in computer science","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"10393_CR5","doi-asserted-by":"crossref","unstructured":"Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: IEEE conference on computer vision and pattern recognition (CVPR)","DOI":"10.1109\/CVPR.2016.91"},{"key":"10393_CR6","doi-asserted-by":"crossref","unstructured":"Lin T-Y, Goyal P, Girshick R, He K, Doll\u00e1r P (2017) Focal loss for dense object detection. In: IEEE international conference on computer vision (ICCV)","DOI":"10.1109\/ICCV.2017.324"},{"key":"10393_CR7","doi-asserted-by":"crossref","unstructured":"Law H, Deng J (2018) Cornernet: detecting objects as paired keypoints. In: European conference on computer vision (ECCV)","DOI":"10.1007\/978-3-030-01264-9_45"},{"key":"10393_CR8","doi-asserted-by":"publisher","unstructured":"Zhou X, Wang D, Krahenbuhl P (2019) Objects as points. arXiv. https:\/\/doi.org\/10.48550\/ARXIV.1904.07850","DOI":"10.48550\/ARXIV.1904.07850"},{"key":"10393_CR9","doi-asserted-by":"crossref","unstructured":"Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. https:\/\/arxiv.org\/abs\/1311.2524","DOI":"10.1109\/CVPR.2014.81"},{"key":"10393_CR10","doi-asserted-by":"publisher","unstructured":"Girshick R (2015) Fast R-CNN. In: 2015 IEEE international conference on computer vision (ICCV), pp 1440\u20131448. https:\/\/doi.org\/10.1109\/ICCV.2015.169","DOI":"10.1109\/ICCV.2015.169"},{"key":"10393_CR11","unstructured":"Dai J, Li Y, He K, Sun J (2016) R-FCN: Object detection via region-based fully convolutional networks. In: International conference on neural information processing systems"},{"key":"10393_CR12","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Lu, Polosukhin I (2017) Attention is all you need. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf"},{"key":"10393_CR13","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N (2021) An image is worth 16x16 words: transformers for image recognition at scale. In: International conference on learning representations. https:\/\/openreview.net\/forum?id=YicbFdNTTy"},{"key":"10393_CR14","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1007\/978-3-030-58452-8_13","volume-title":"Computer vision\u2014ECCV 2020","author":"N Carion","year":"2020","unstructured":"Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. In: Vedaldi A, Bischof H, Brox T, Frahm J-M (eds) Computer vision\u2014ECCV 2020. Springer, Cham, pp 213\u2013229"},{"key":"10393_CR15","doi-asserted-by":"publisher","unstructured":"Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp. 3431\u20133440. https:\/\/doi.org\/10.1109\/CVPR.2015.7298965","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"10393_CR16","doi-asserted-by":"publisher","unstructured":"David E, Netanyahu N (2016) DeepPainter: painter classification using deep convolutional autoencoders, pp 20\u201328. https:\/\/doi.org\/10.1007\/978-3-319-44781-0_3","DOI":"10.1007\/978-3-319-44781-0_3"},{"key":"10393_CR17","doi-asserted-by":"crossref","unstructured":"Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention (MICCAI) 2015. LNCS, vol 9351, pp. 234\u2013241. Springer. (available on arXiv:1505.04597 [cs.CV]). http:\/\/lmb.informatik.uni-freiburg.de\/Publications\/2015\/RFB15a","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"10393_CR18","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770\u2013778.https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"10393_CR19","unstructured":"Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7\u20139, 2015, Conference track proceedings. http:\/\/arxiv.org\/abs\/1409.1556"},{"key":"10393_CR20","doi-asserted-by":"crossref","unstructured":"Lin T-Y, Doll\u00e1r P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: IEEE conference on computer vision and pattern recognition (CVPR)","DOI":"10.1109\/CVPR.2017.106"},{"key":"10393_CR21","unstructured":"Hood M. Colony Collapse Disorder. https:\/\/www.britannica.com\/science\/colony-collapse-disorder Accessed 2024-04-04"},{"key":"10393_CR22","doi-asserted-by":"publisher","unstructured":"Elizondo V, Briceno JC, Travieso CM, Alonso JB (2013) Video monitoring of a mite in Honeybee Cells. Adv Mater Res 664:1107\u20131113. https:\/\/doi.org\/10.4028\/www.scientific.net\/AMR.664.1107 Accessed 04 Mar 2020","DOI":"10.4028\/www.scientific.net\/AMR.664.1107"},{"key":"10393_CR23","doi-asserted-by":"publisher","unstructured":"Ramirez M, Prendas JP, Travieso CM, Calderon R, Salas O (2012) Detection of the mite Varroa destructor in honey bee cells by video sequence processing. In: 2012 IEEE 16th international conference on intelligent engineering systems (INES). IEEE, Lisbon, Portugal, pp 103\u2013108. https:\/\/doi.org\/10.1109\/INES.2012.6249811","DOI":"10.1109\/INES.2012.6249811"},{"key":"10393_CR24","unstructured":"Bassignana D, Bassignana CF, Cuttini D, Francini G, Ghirardi M, Laurino D, Martino F, Porporato M, Varesio A (2018) Automatic varroa counting bee varroa scanner. In: Conference: Eurbee 2018. https:\/\/www.researchgate.net\/publication\/328192722_Automatic_Varroa_Counting_Bee_Varroa_Scanner"},{"key":"10393_CR25","unstructured":"Marc C (2024) De L\u2019intelligence Artificielle Pour Sauver les Abeilles. https:\/\/actu.epfl.ch\/news\/de-l-intelligence-artificielle-pour-sauver-les-abe\/ Accessed 04 Apr 2024"},{"key":"10393_CR26","unstructured":"Dupleix A, Mui S, Pfister F, Reutenauer V (2018) Application smartphone pour le comptage automatis\u00e9 du varroa, 2"},{"key":"10393_CR27","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1007\/978-3-319-93000-8_52","volume-title":"Image analysis and recognition","author":"S Schurischuster","year":"2018","unstructured":"Schurischuster S, Remeseiro B, Radeva P, Kampel M (2018) A Preliminary study of image analysis for parasite detection on honey bees. In: Campilho A, Karray F, ter Haar Romeny B (eds) Image analysis and recognition, vol 10882. Springer, Cham, pp 465\u2013473. https:\/\/doi.org\/10.1007\/978-3-319-93000-8_52"},{"key":"10393_CR28","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2019.104898","volume":"164","author":"K Bjerge","year":"2019","unstructured":"Bjerge K, Frigaard CE, Mikkelsen PH, Nielsen TH, Misbih M, Kryger P (2019) A computer vision system to monitor the infestation level of Varroa destructor in a honeybee colony. Comput Electron Agric 164:104898. https:\/\/doi.org\/10.1016\/j.compag.2019.104898","journal-title":"Comput Electron Agric"},{"key":"10393_CR29","doi-asserted-by":"publisher","unstructured":"Chazette L, Becker M, Szczerbicka H (2016) Basic algorithms for bee hive monitoring and laser-based mite control. In: 2016 IEEE symposium series on computational intelligence (SSCI). IEEE, Athens, Greece, pp 1\u20138. https:\/\/doi.org\/10.1109\/SSCI.2016.7850001","DOI":"10.1109\/SSCI.2016.7850001"},{"issue":"22","key":"10393_CR30","doi-asserted-by":"publisher","first-page":"11078","DOI":"10.3390\/app112211078","volume":"11","author":"D Mrozek","year":"2021","unstructured":"Mrozek D, Gorny R, Wachowicz A, Malysiak-Mrozek B (2021) Edge-based detection of varroosis in beehives with IoT devices with embedded and TPU-accelerated machine learning. Appl Sci 11(22):11078. https:\/\/doi.org\/10.3390\/app112211078","journal-title":"Appl Sci"},{"issue":"3","key":"10393_CR31","doi-asserted-by":"publisher","first-page":"506","DOI":"10.3390\/signals3030030","volume":"3","author":"G Voudiotis","year":"2022","unstructured":"Voudiotis G, Moraiti A, Kontogiannis S (2022) Deep learning beehive monitoring system for early detection of the varroa mite. Signals 3(3):506\u2013523. https:\/\/doi.org\/10.3390\/signals3030030","journal-title":"Signals"},{"issue":"8","key":"10393_CR32","doi-asserted-by":"publisher","first-page":"2764","DOI":"10.3390\/s21082764","volume":"21","author":"S Bilik","year":"2021","unstructured":"Bilik S, Kratochvila L, Ligocki A, Bostik O, Zemcik T, Hybl M, Horak K, Zalud L (2021) Visual diagnosis of the varroa destructor parasitic mite in honeybees using object detector techniques. Sensors 21(8):2764. https:\/\/doi.org\/10.3390\/s21082764","journal-title":"Sensors"},{"key":"10393_CR33","doi-asserted-by":"publisher","unstructured":"Kriouile Y, Ancourt C, Wegrzyn-Wolska K, Bougueroua L (2022) Generating proposals from corners in RPN to detect bees in dense scenes. In: Farinella GM, Radeva P, Bouatouch K (eds) Proceedings of the 17th international joint conference on computer vision, imaging and computer graphics theory and applications, VISIGRAPP 2022, vol 5: VISAPP, pp 339\u2013350. Scitepress. https:\/\/doi.org\/10.5220\/0010815000003124","DOI":"10.5220\/0010815000003124"},{"key":"10393_CR34","unstructured":"Tiu E (2024) Metrics to evaluate your semantic segmentation model. https:\/\/towardsdatascience.com\/metrics-to-evaluate-your-semantic-segmentation-model-6bcb99639aa2 Accessed 04 Apr 2024"},{"key":"10393_CR35","unstructured":"Groupement de D\u00e9fense Sanitaire Apicole du Var: M\u00e9thodes D\u2019\u00e9valuation de L\u2019infestation Par Varroa Destructor Au Cours de L\u2019ann\u00e9e et Interpr\u00e9tation Diagnostic. http:\/\/gdsa83.fr\/evaluation-varroa\/ Accessed 04 Apr 2024"},{"key":"10393_CR36","doi-asserted-by":"publisher","unstructured":"Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv. https:\/\/doi.org\/10.48550\/ARXIV.1704.04861","DOI":"10.48550\/ARXIV.1704.04861"},{"key":"10393_CR37","unstructured":"Tensorflow: image segmentation. https:\/\/www.tensorflow.org\/tutorials\/images\/segmentation Accessed 04 Apr 2024"},{"key":"10393_CR38","unstructured":"ITSAP: ITSAP. https:\/\/itsap.asso.fr\/ Accessed 04 Apr 2024"},{"key":"10393_CR39","unstructured":"Reid B. Varroa Mite (Varroa Destructor). http:\/\/www.columbia.edu\/itc\/cerc\/danoff-burg\/invasion_bio\/inv_spp_summ\/varroa_destructor.html#Taxonomy Accessed 04 Apr 2024"},{"issue":"1","key":"10393_CR40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3896\/IBRA.1.52.1.09","volume":"52","author":"V Dietemann","year":"2013","unstructured":"Dietemann V, Nazzi F, Martin SJ, Anderson DL, Locke B, Delaplane KS, Wauquiez Q, Tannahill C, Frey E, Ziegelmann B, Rosenkranz P, Ellis JD (2013) Standard methods for varroa research. J Apic Res 52(1):1\u201354. https:\/\/doi.org\/10.3896\/IBRA.1.52.1.09","journal-title":"J Apic Res"},{"key":"10393_CR41","unstructured":"Pierred: Le Comptage des Varroas. Pourquoi? Quand? Comment? https:\/\/asapistra.fr\/?q=node\/1241 Accessed 04 Apr 2024"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-10393-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-024-10393-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-10393-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,25]],"date-time":"2024-11-25T12:08:10Z","timestamp":1732536490000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-024-10393-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,23]]},"references-count":41,"journal-issue":{"issue":"35","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["10393"],"URL":"https:\/\/doi.org\/10.1007\/s00521-024-10393-x","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,23]]},"assertion":[{"value":"13 May 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 August 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 September 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}