{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T17:04:47Z","timestamp":1774285487568,"version":"3.50.1"},"reference-count":23,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2026,1,27]],"date-time":"2026-01-27T00:00:00Z","timestamp":1769472000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"name":"FCT","award":["UIDB\/00319\/2020"],"award-info":[{"award-number":["UIDB\/00319\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,1,27]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>The use of multimodal systems brings great prospects in solving complex problems unsolvable by the usual unimodal approaches. Therefore, a multimodal video and radar system are proposed to design a multimodal machine-learning problem on edge devices. The system\u2019s architecture uses Docker containers to capture knowledge under models and processes, allowing the system to be easily managed. Furthermore, the importance of object detection is enhanced in the proposed system, as the identification and localization of objects in different data modalities are critical components of several multimodal machine-learning tasks. Hence, it is presented as an overall description of the architecture and a discussion of the data pipeline of this system. It is approached by the challenge of data alignment using homographic transformations with video camera and radar data, as well as using the system calibration to reach the data fusion and consequent predictions. It highlights the advantages of multimodal systems in dealing with complex and dynamic environments and provides a general approach to multimodal machine-learning problems on edge devices.<\/jats:p>","DOI":"10.1093\/jigpal\/jzaf043","type":"journal-article","created":{"date-parts":[[2025,6,9]],"date-time":"2025-06-09T07:20:58Z","timestamp":1749453658000},"source":"Crossref","is-referenced-by-count":0,"title":["Multimodal video and radar system for edge devices object detection"],"prefix":"10.1093","volume":"34","author":[{"given":"Filipa","family":"Ferraz","sequence":"first","affiliation":[{"name":"ALGORITMI Research Centre\/LASI, University of Minho , 4710-057, Braga ,","place":["Portugal"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eduardo","family":"Coelho","sequence":"additional","affiliation":[{"name":"ALGORITMI Research Centre\/LASI, University of Minho , 4710-057, Braga ,","place":["Portugal"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hugo","family":"Peixoto","sequence":"additional","affiliation":[{"name":"ALGORITMI Research Centre\/LASI, University of Minho , 4710-057, Braga ,","place":["Portugal"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dalila","family":"Dur\u00e7es","sequence":"additional","affiliation":[{"name":"ALGORITMI Research Centre\/LASI, University of Minho , 4710-057, Braga ,","place":["Portugal"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pedro","family":"Melo-Pinto","sequence":"additional","affiliation":[{"name":"CITAB, University of Tr\u00e1s-os-Montes and Alto Douro , 5000-801, Vila Real ,","place":["Portugal"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Victor","family":"Alves","sequence":"additional","affiliation":[{"name":"ALGORITMI Research Centre\/LASI, University of Minho , 4710-057, Braga ,","place":["Portugal"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Louren\u00c7o","family":"Bandeira","sequence":"additional","affiliation":[{"name":"Schr\u00e9der Hyperion , 2775-737, Carcavelos ,","place":["Portugal"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jos\u00e9","family":"Machado","sequence":"additional","affiliation":[{"name":"ALGORITMI Research Centre\/LASI, University of Minho , 4710-057, Braga ,","place":["Portugal"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paulo","family":"Novais","sequence":"additional","affiliation":[{"name":"ALGORITMI Research Centre\/LASI, University of Minho , 4710-057, Braga ,","place":["Portugal"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2026,3,21]]},"reference":[{"key":"2026032312184543400_ref1","doi-asserted-by":"crossref","first-page":"2280","DOI":"10.1109\/TMM.2022.3145663","article-title":"Multi-modality sensing and data fusion for multi-vehicle detection","volume":"25","author":"Roy","year":"2023","journal-title":"IEEE Transactions on Multimedia"},{"key":"2026032312184543400_ref2","doi-asserted-by":"crossref","first-page":"6625","DOI":"10.1109\/JSEN.2015.2464232","article-title":"Data fusion of acoustics, infrared, and marine radar for avian study","volume":"15","author":"Mirzaei","year":"2015","journal-title":"IEEE Sensors Journal"},{"key":"2026032312184543400_ref3","doi-asserted-by":"crossref","first-page":"3047","DOI":"10.1109\/ITSC48978.2021.9564951","volume-title":"2021 IEEE International Intelligent Transportation Systems Conference (ITSC)","author":"Xu","year":"2021"},{"key":"2026032312184543400_ref4","first-page":"1","volume-title":"In: 2019 International Joint Conference on Neural Networks (IJCNN)","author":"Liu","year":"2019"},{"key":"2026032312184543400_ref5","first-page":"012044","article-title":"Research on time synchronization and spatial calibration algorithms of radar and camera","volume":"2428","author":"Song","year":"2023","journal-title":"Journal of Physics: Conference Series"},{"key":"2026032312184543400_ref6","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1007\/s11263-019-01247-4","article-title":"Deep learning for generic object detection: a survey","volume":"128","author":"Liu","year":"2020","journal-title":"International Journal of Computer Vision"},{"key":"2026032312184543400_ref7","first-page":"615","volume-title":"Proceedings of the 2017 SIAM International Conference on Data Mining (SDM)","author":"Nassar","year":"2017"},{"key":"2026032312184543400_ref8","doi-asserted-by":"crossref","first-page":"8554","DOI":"10.3390\/s22218554","article-title":"Integrated video and acoustic emission data fusion for intelligent decision making in material surface inspection system","volume":"22, 21","author":"Chernov","year":"2022","journal-title":"Sensors"},{"key":"2026032312184543400_ref9","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1007\/s11431-021-1927-9","article-title":"Deep multimodal learning for municipal solid waste sorting","volume":"65","author":"Gang","year":"2022","journal-title":"Science China Technological Sciences"},{"key":"2026032312184543400_ref10","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1016\/j.patrec.2021.08.028","article-title":"Deep multi-scale and multi-modal fusion for 3D object detection","volume":"151","author":"Guo","year":"2021","journal-title":"Pattern Recognition Letters"},{"key":"2026032312184543400_ref11","doi-asserted-by":"publisher","DOI":"10.1016\/j.measurement.2022.111001","article-title":"Multi-scale multi-modal fusion for object detection in autonomous driving based on selective kernel","volume":"194","author":"Gao","year":"2022","journal-title":"Measurement"},{"key":"2026032312184543400_ref12","first-page":"2","article-title":"Deep multimodal fusion of image and non-image data in disease diagnosis and prognosis: a review. Progress","volume":"5, 2","author":"Cui","year":"2023","journal-title":"Biomedical Engineering"},{"key":"2026032312184543400_ref13","doi-asserted-by":"crossref","first-page":"2887","DOI":"10.1007\/s11042-020-08836-3","article-title":"Deep learning-based late fusion of multimodal information for emotion classification of music video","volume":"80","author":"Pandeya","year":"2021","journal-title":"Multimedia Tools and Applications"},{"key":"2026032312184543400_ref14","year":"2024"},{"key":"2026032312184543400_ref15","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1016\/j.scs.2017.12.034","article-title":"The IoT for smart sustainable cities of the future: an analytical framework for sensor-based big data applications for environmental sustainability","volume":"38","author":"Bibri","year":"2018","journal-title":"Sustainable Cities and Society"},{"key":"2026032312184543400_ref16","author":"MQTT: The Standard for IoT Messaging","year":"2024"},{"key":"2026032312184543400_ref17","year":"2024"},{"key":"2026032312184543400_ref18","doi-asserted-by":"crossref","first-page":"9614","DOI":"10.1109\/ACCESS.2018.2799173","article-title":"Automatic calculation of a transformation matrix between two frames","volume":"6","author":"Cashbaugh","year":"2018","journal-title":"IEEE Access"},{"key":"2026032312184543400_ref19","doi-asserted-by":"crossref","first-page":"1389","DOI":"10.1364\/JOSAA.25.001389","article-title":"Homography-based method for calibrating an omnidirectional vision system","volume":"25","author":"Zhang","year":"2008","journal-title":"Journal of the Optical Society of America A"},{"key":"2026032312184543400_ref20","doi-asserted-by":"crossref","first-page":"650","DOI":"10.1109\/WACV.2011.5711566","volume-title":"In: 2011 IEEE Workshop on Applications of Computer Vision (WACV) Proceedings","author":"Schroeder","year":"2011"},{"key":"2026032312184543400_ref21","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/j.patrec.2020.03.033","article-title":"The equivalence of two definitions of compatible homography matrices","volume":"135","author":"Chojnacki","year":"2020","journal-title":"Pattern Recognition Letters"},{"key":"2026032312184543400_ref22","first-page":"185","article-title":"Homographic p-norms: metrics of homographic image transformation","volume":"39","author":"Je","year":"2015","journal-title":"Signal Processing: Image Communication"},{"key":"2026032312184543400_ref23","doi-asserted-by":"publisher","first-page":"673","DOI":"10.1007\/978-3-031-40725-3_57","volume-title":"Hybrid Artificial Intelligent Systems: 18th International Conference (HAIS 2023) Proceedings","author":"Coelho","year":"2023"}],"container-title":["Logic Journal of the IGPL"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jigpal\/article-pdf\/34\/1\/jzaf043\/67483279\/jzaf043.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jigpal\/article-pdf\/34\/1\/jzaf043\/67483279\/jzaf043.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T16:18:58Z","timestamp":1774282738000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jigpal\/article\/doi\/10.1093\/jigpal\/jzaf043\/8537639"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,27]]},"references-count":23,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1,27]]}},"URL":"https:\/\/doi.org\/10.1093\/jigpal\/jzaf043","relation":{},"ISSN":["1367-0751","1368-9894"],"issn-type":[{"value":"1367-0751","type":"print"},{"value":"1368-9894","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,2]]},"published":{"date-parts":[[2026,1,27]]},"article-number":"jzaf043"}}