{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T12:20:46Z","timestamp":1767183646112,"version":"build-2065373602"},"reference-count":51,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2023,9,15]],"date-time":"2023-09-15T00:00:00Z","timestamp":1694736000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"UK Department for Environment, Food and Rural Affairs (DeFRA) Fisheries Industry Science Partnership (FISP)","award":["ECM_63777","81917"],"award-info":[{"award-number":["ECM_63777","81917"]}]},{"name":"EuropeanMarine and Fisheries Fund and Welsh Government","award":["ECM_63777","81917"],"award-info":[{"award-number":["ECM_63777","81917"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This work investigates the application of Computer Vision to the problem of the automated counting and measuring of crabs and lobsters onboard fishing boats. The aim is to provide catch count and measurement data for these key commercial crustacean species. This can provide vital input data for stock assessment models, to enable the sustainable management of these species. The hardware system is required to be low-cost, have low-power usage, be waterproof, available (given current chip shortages), and able to avoid over-heating. The selected hardware is based on a Raspberry Pi 3A+ contained in a custom waterproof housing. This hardware places challenging limitations on the options for processing the incoming video, with many popular deep learning frameworks (even light-weight versions) unable to load or run given the limited computational resources. The problem can be broken into several steps: (1) Identifying the portions of the video that contain each individual animal; (2) Selecting a set of representative frames for each animal, e.g, lobsters must be viewed from the top and underside; (3) Detecting the animal within the frame so that the image can be cropped to the region of interest; (4) Detecting keypoints on each animal; and (5) Inferring measurements from the keypoint data. In this work, we develop a pipeline that addresses these steps, including a key novel solution to frame selection in video streams that uses classification, temporal segmentation, smoothing techniques and frame quality estimation. The developed pipeline is able to operate on the target low-power hardware and the experiments show that, given sufficient training data, reasonable performance is achieved.<\/jats:p>","DOI":"10.3390\/s23187897","type":"journal-article","created":{"date-parts":[[2023,9,15]],"date-time":"2023-09-15T04:06:13Z","timestamp":1694750773000},"page":"7897","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Automated Video-Based Capture of Crustacean Fisheries Data Using Low-Power Hardware"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3454-807X","authenticated-orcid":false,"given":"Sebastian","family":"Gregory Dal To\u00e9","sequence":"first","affiliation":[{"name":"Department of Computer Science, Aberystwyth University, Aberystwyth SY23 3DB, Ceredigion, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marie","family":"Neal","sequence":"additional","affiliation":[{"name":"Ystumtec Ltd., Pant-Y-Chwarel, Ystumtuen, Aberystwyth SY23 3AF, Ceredigion, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4263-8435","authenticated-orcid":false,"given":"Natalie","family":"Hold","sequence":"additional","affiliation":[{"name":"School of Ocean Sciences, Bangor University, Bangor LL57 2DG, Gwynedd, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0569-9853","authenticated-orcid":false,"given":"Charlotte","family":"Heney","sequence":"additional","affiliation":[{"name":"School of Ocean Sciences, Bangor University, Bangor LL57 2DG, Gwynedd, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7668-0845","authenticated-orcid":false,"given":"Rebecca","family":"Turner","sequence":"additional","affiliation":[{"name":"School of Ocean Sciences, Bangor University, Bangor LL57 2DG, Gwynedd, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Emer","family":"McCoy","sequence":"additional","affiliation":[{"name":"School of Ocean Sciences, Bangor University, Bangor LL57 2DG, Gwynedd, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Muhammad","family":"Iftikhar","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Aberystwyth University, Aberystwyth SY23 3DB, Ceredigion, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7570-1192","authenticated-orcid":false,"given":"Bernard","family":"Tiddeman","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Aberystwyth University, Aberystwyth SY23 3DB, Ceredigion, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,15]]},"reference":[{"key":"ref_1","unstructured":"European Parliament and the Council of the European Union (2023, August 02). CFP. Regulation (EU) No 1380\/2013 of the European Parliament and of the Council of 11 December 2013 on the Common Fisheries Policy, Amending Council Regulations (EC) No 1954\/2003 and (EC) No 1224\/2009 and Repealing Council Regulations (EC) No 2371\/2002 (2013). Available online: https:\/\/eur-lex.europa.eu\/eli\/reg\/2013\/1380\/oj."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2218","DOI":"10.1073\/pnas.1909726116","article-title":"Effective fisheries management instrumental in improving fish stock status","volume":"117","author":"Hilborn","year":"2020","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1040","DOI":"10.1093\/icesjms\/fsu034","article-title":"Reflections on the success of traditional fisheries management","volume":"71","author":"Hilborn","year":"2014","journal-title":"ICES J. Mar. Sci."},{"key":"ref_4","unstructured":"Food and Agriculture Organization of the United Nations (2023, August 02). The State of World Fisheries and Aquaculture 2022. Towards Blue Transformation. Available online: https:\/\/www.fao.org\/3\/cc0461en\/cc0461en.pdf."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1283","DOI":"10.1007\/s00227-012-1909-6","article-title":"What catch data can tell us about the status of global fisheries","volume":"159","author":"Froese","year":"2012","journal-title":"Mar. Biol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1002\/fee.2431","article-title":"The global rise of crustacean fisheries","volume":"20","author":"Boenish","year":"2022","journal-title":"Front. Ecol. Environ."},{"key":"ref_7","unstructured":"Marine Management Organisation (2023, August 02). UK Sea FIsheries Statistics 2020, Available online: https:\/\/assets.publishing.service.gov.uk\/government\/uploads\/system\/uploads\/attachment_data\/file\/1020837\/UK_Sea_Fisheries_Statistics_2020_-_AC_checked.pdf."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1111\/faf.12642","article-title":"Global crustacean stock assessment modelling: Reconciling available data and complexity","volume":"23","author":"Hodgdon","year":"2022","journal-title":"Fish Fish."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1093\/icesjms\/fss185","article-title":"Review of integrated size-structured models for stock assessment of hard-to-age crustacean and mollusc species","volume":"70","author":"Punt","year":"2013","journal-title":"ICES J. Mar. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/BF00409591","article-title":"Molt cycle alterations in behavior, feeding and diel rhythms of a decapod crustacean, the spiny lobster Panulirus argus","volume":"68","author":"Lipcius","year":"1982","journal-title":"Mar. Biol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/S0165-7836(02)00240-0","article-title":"Space-time variation in catchability of southern rock lobster Jasus edwardsii in Tasmania explained by environmental, physiological and density-dependent processes","volume":"61","author":"Ziegler","year":"2003","journal-title":"Fish. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1111\/faf.12425","article-title":"Electronic monitoring in fisheries: Lessons from global experiences and future opportunities","volume":"21","author":"Mortensen","year":"2020","journal-title":"Fish Fish."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1811","DOI":"10.1093\/icesjms\/fsv030","article-title":"Video capture of crustacean fisheries data as an alternative to on-board observers","volume":"72","author":"Hold","year":"2015","journal-title":"ICES J. Mar. Sci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"105339","DOI":"10.1016\/j.compag.2020.105339","article-title":"Real-time robust detector for underwater live crabs based on deep learning","volume":"172","author":"Cao","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18\u201323). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"107522","DOI":"10.1016\/j.compag.2022.107522","article-title":"Real-time detection of underwater river crab based on multi-scale pyramid fusion image enhancement and MobileCenterNet model","volume":"204","author":"Ji","year":"2023","journal-title":"Comput. Electron. Agric."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chen, X., Zhang, Y., Li, D., and Duan, Q. (2022). Chinese Mitten Crab Detection and Gender Classification Method Based on Gmnet-Yolov4. SSRN.","DOI":"10.2139\/ssrn.4314563"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"102115","DOI":"10.1016\/j.aquaeng.2020.102115","article-title":"An improved YOLOv3 algorithm to detect molting in swimming crabs against a complex background","volume":"91","author":"Tang","year":"2020","journal-title":"Aquac. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Hu, K., Shi, C., Gao, G., and Zhu, J. (2021, January 10\u201312). A Detection Systems For Molting Scylla Paramamosain Based On YOLO v4. Proceedings of the 2021 3rd International Academic Exchange Conference on Science and Technology Innovation (IAECST), Guangzhou, China.","DOI":"10.1109\/IAECST54258.2021.9695773"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1093542","DOI":"10.3389\/fmars.2023.1093542","article-title":"A Part-based Deep Learning Network for identifying individual crabs using abdomen images","volume":"10","author":"Wu","year":"2023","journal-title":"Front. Mar. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1308","DOI":"10.1093\/icesjms\/fsz223","article-title":"Automatic detection of Western rock lobster using synthetic data","volume":"77","author":"Mahmood","year":"2019","journal-title":"ICES J. Mar. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Chelouati, N., Fares, F., Bouslimani, Y., and Ghribi, M. (2021, January 28\u201329). Lobster detection using an Embedded 2D Vision System with a FANUC industrual robot. Proceedings of the 2021 IEEE International Symposium on Robotic and Sensors Environments (ROSE), Virtually.","DOI":"10.1109\/ROSE52750.2021.9611755"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Chelouati, N., Bouslimani, Y., and Ghribi, M. (2023). Lobster Position Estimation Using YOLOv7 for Potential Guidance of FANUC Robotic Arm in American Lobster Processing. Designs, 7.","DOI":"10.3390\/designs7030070"},{"key":"ref_24","unstructured":"Wang, D., Holmes, M., Vinson, R., and Seibel, G. (August, January 29). Machine Vision Guided Robotics for Blue Crab Disassembly\u2014Deep Learning Based Crab Morphology Segmentation. Proceedings of the ASABE Annual International Meeting, Detroit, MI, USA."},{"key":"ref_25","first-page":"043103","article-title":"Convolutional neural network guided blue crab knuckle detection for autonomous crab meat picking machine","volume":"57","author":"Wang","year":"2018","journal-title":"Opt. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"10719","DOI":"10.1007\/s00521-022-07007-9","article-title":"CDNet: A real-time and robust crosswalk detection network on Jetson nano based on YOLOv5","volume":"34","author":"Zhang","year":"2022","journal-title":"Neural Comput. Appl."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"108001","DOI":"10.1016\/j.comnet.2021.108001","article-title":"Aerial imagery pile burn detection using deep learning: The FLAME dataset","volume":"193","author":"Shamsoshoara","year":"2021","journal-title":"Comput. Netw."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2023","DOI":"10.1109\/TUFFC.2021.3068190","article-title":"Mini-COVIDNet: Efficient Lightweight Deep Neural Network for Ultrasound Based Point-of-Care Detection of COVID-19","volume":"68","author":"Awasthi","year":"2021","journal-title":"IEEE Trans. Ultrason. Ferroelectr. Freq. Control"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1007\/s11554-023-01271-1","article-title":"Efficient convolutional neural networks on Raspberry Pi for image classification","volume":"20","author":"Ju","year":"2023","journal-title":"J. Real-Time Image Process."},{"key":"ref_30","unstructured":"Wolf, W. (1996, January 9). Key frame selection by motion analysis. Proceedings of the 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings, Atlanta, GA, USA."},{"key":"ref_31","unstructured":"Elgammal, A., Harwood, D., and Davis, L.S. (July, January 26). Non-parametric Model for Background Subtraction. Proceedings of the European Conference on Computer Vision, Dublin, Ireland."},{"key":"ref_32","unstructured":"Viola, P.A., and Jones, M.J. (2001, January 8\u201314). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, Kauai, HI, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Labuguen, R.T., Volante, E.J.P., Causo, A., Bayot, R., Peren, G., Macaraig, R.M., Libatique, N.J.C., and Tangonan, G.L. (2012, January 23\u201325). Automated fish fry counting and schooling behavior analysis using computer vision. Proceedings of the 2012 IEEE 8th International Colloquium on Signal Processing and its Applications, Malacca, Malaysia.","DOI":"10.1109\/CSPA.2012.6194729"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"34","DOI":"10.5815\/ijigsp.2018.09.05","article-title":"A Video based Vehicle Detection, Counting and Classification System","volume":"10","author":"Memon","year":"2018","journal-title":"Int. J. Image Graph. Signal Process. (IJIGSP)"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"4","DOI":"10.4031\/MTSJ.50.1.1","article-title":"Progress in the automated identification, measurement, and counting of fish in underwater image sequences","volume":"50","author":"Shortis","year":"2016","journal-title":"Mar. Technol. Soc. J."},{"key":"ref_36","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_39","unstructured":"Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (November, January 27). Searching for mobilenetv3. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_40","unstructured":"Saleh, A., Jones, D., Jerry, D., and Azghadi, M.R. (2022). A lightweight Transformer-based model for fish landmark detection. arXiv."},{"key":"ref_41","unstructured":"Tan, M., and Le, Q.V. (2020). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., Albanie, S., Sun, G., and Wu, E. (2019). Squeeze-and-Excitation Networks. arXiv.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Ren, J., Shen, X., Lin, Z., and M\u011bch, R. (2020, January 1\u20135). Best Frame Selection in a Short Video. Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093615"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Griffin, B.A., and Corso, J.J. (2019, January 9). BubbleNets: Learning to Select the Guidance Frame in Video Object Segmentation by Deep Sorting Frames. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00912"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Chang, H., Yu, F., Wang, J., Ashley, D., and Finkelstein, A. (2016). Automatic Triage for a Photo Series. ACM Trans. Graph., 35.","DOI":"10.1145\/2897824.2925908"},{"key":"ref_46","unstructured":"Jocher, G., Chaurasia, A., Stoken, A., Borovec, J., NanoCode012, Kwon, Y., Michael, K., Fang, J., and Imyhxy (2022). ultralytics\/yolov5: v7.0\u2014YOLOv5 SOTA Realtime Instance Segmentation. Zenodo."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Dutta, A., and Zisserman, A. (2019, January 21\u201325). The VIA Annotation Software for Images, Audio and Video. Proceedings of the 27th ACM International Conference on Multimedia, Association for Computing Machinery, New York, NY, USA.","DOI":"10.1145\/3343031.3350535"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Padilla, R., Passos, W.L., Dias, T.L.B., Netto, S.L., and da Silva, E.A.B. (2021). A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit. Electronics, 10.","DOI":"10.3390\/electronics10030279"},{"key":"ref_49","unstructured":"Polino, A., Pascanu, R., and Alistarh, D. (2018). Model Compression via Distillation and Quantization. arXiv."},{"key":"ref_50","unstructured":"tensorflow.org (2023, September 11). Post-Training Quantization. Available online: https:\/\/www.tensorflow.org\/lite\/performance\/post_training_quantization."},{"key":"ref_51","unstructured":"(2023, September 11). seafish.co.uk. Available online: https:\/\/seafish.co.uk."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/18\/7897\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:51:25Z","timestamp":1760129485000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/18\/7897"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,15]]},"references-count":51,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["s23187897"],"URL":"https:\/\/doi.org\/10.3390\/s23187897","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,9,15]]}}}