{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T01:44:45Z","timestamp":1792028685552,"version":"4.4.0"},"reference-count":33,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T00:00:00Z","timestamp":1786060800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Hacettepe University Scientific Research Projects (BAP) Coordination Unit","award":["FHD-2025-22361"],"award-info":[{"award-number":["FHD-2025-22361"]}]},{"award":["FHD-2025-22361"],"award-info":[{"award-number":["FHD-2025-22361"]}],"id":[{"id":"https:\/\/ror.org\/04kwvgz42","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Reliable multisensor perception is a key requirement for practical waste sorting, yet many low-cost sensor configurations cannot rely on hardware synchronization or carefully controlled acquisition. We present a pilot-scale multisensor waste-sorting testbed that combines an unsynchronized RGB camera with a long-wave infrared (LWIR) camera for object classification on a continuously moving conveyor, and introduce ThermalRGBTrash, a new paired RGB\u2013LWIR video dataset for this task. To enable fusion under asynchronous acquisition, we develop a fully software-based registration pipeline that combines SuperPoint\u2013SuperGlue matching with an adaptive sliding-window strategy designed to recover from long-wave infrared sensor artifacts, including non-uniformity correction events. Across 281,439 matched frame pairs from 19 paired videos, the registration pipeline achieves a mean spatial alignment error of 2.27 pixels and matches 99.98% of attempted frame pairs. We then detect and segment objects with Mask R-CNN, track them across the conveyor, and classify each tracklet using frozen DINOv2 self-supervised Vision Transformer (ViT-L\/14) features with a lightweight multilayer perceptron head. RGB and LWIR representations are combined through late fusion. On 550 tracklets under video-disjoint 10-fold cross-validation, the fused pipeline reaches a macro F1 score of 0.924, outperforming RGB alone (0.886) and LWIR alone (0.856). On a mixed-class test set of 351 tracklets reserved exclusively for final evaluation, fusion reaches a macro F1 score of 0.947. The fusion advantage persists across multiple backbone and pretraining choices, while ablation studies support the chosen temporal sampling and pooling design. These results show that accurate RGB\u2013LWIR object classification is achievable without synchronization hardware, and establish ThermalRGBTrash as a benchmark for future work on practical multisensor perception in conveyor-based waste sorting.<\/jats:p>","DOI":"10.3390\/s26165017","type":"journal-article","created":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T10:12:01Z","timestamp":1786097521000},"page":"5017","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Software-Only Registration and Cross-Spectral Classification of Unsynchronized RGB\u2013LWIR Video: A Multisensor Benchmark for Conveyor-Based Waste Sorting"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9081-4264","authenticated-orcid":false,"given":"Burak","family":"Akdemir","sequence":"first","affiliation":[{"name":"Department of Electrical and Electronics Engineering, Hacettepe University, Beytepe, Ankara 06800, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8868-1132","authenticated-orcid":false,"given":"Seniha Esen","family":"Yuksel","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronics Engineering, Hacettepe University, Beytepe, Ankara 06800, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.wasman.2017.09.019","article-title":"Multi-material classification of dry recyclables from municipal solid waste based on thermal imaging","volume":"70","author":"Gundupalli","year":"2017","journal-title":"Waste Manag."},{"key":"ref_2","unstructured":"Gust, N., Meyke, N., Malkomes, T., and Ziolkowski, M. (2022). ThermalSort: Active Thermography-Based Sorting of Black Plastics. Proceedings of the Sensor-Based Sorting and Control. RWTH Aachen, Fraunhofer IZFP."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"498","DOI":"10.1038\/s41597-026-07053-1","article-title":"Multimodal and Hyperspectral Dataset for Segmentation of Bulky Waste using VIS, IR, NIR, and Terahertz Imaging","volume":"13","author":"Bihler","year":"2026","journal-title":"Sci. Data"},{"key":"ref_4","first-page":"104760","article-title":"Low-value recyclable waste identification based on NIR feature analysis and RGB-NIR fusion","volume":"132","author":"Ji","year":"2023","journal-title":"Infrared Phys. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.wasman.2021.12.021","article-title":"RGB-D fusion models for construction and demolition waste detection","volume":"139","author":"Li","year":"2022","journal-title":"Waste Manag."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Casao, S., Pe\u00f1a, F., Sabater, A., Castill\u00f3n, R., Su\u00e1rez, D., Montijano, E., and Murillo, A.C. (2024, January 14\u201318). SpectralWaste Dataset: Multimodal Data for Waste Sorting Automation. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/IROS58592.2024.10801797"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Tran, K.B., Carballo, A., and Takeda, K. (2025). T360Fusion: Temporal 360 Multimodal Fusion for 3D Object Detection via Transformers. Sensors, 25.","DOI":"10.3390\/s25164902"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wei, X., Li, Z., Wang, Y., and Zhu, S. (2025). RTMF-Net: A Dual-Modal Feature-Aware Fusion Network for Dense Forest Object Detection. Sensors, 25.","DOI":"10.3390\/s25185631"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kafka, O., Rankl, C., and Moser, D. (2025). Beyond RGB: Early Stage Fusion of Thermal and Visual Modalities for Robust Maritime Perception. Electronics, 14.","DOI":"10.3390\/electronics14234746"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"102492","DOI":"10.1016\/j.inffus.2024.102492","article-title":"RGBT Tracking: A Comprehensive Review","volume":"110","author":"Feng","year":"2024","journal-title":"Inf. Fusion"},{"key":"ref_11","unstructured":"Thomas, M., Ablett, T., and Kelly, J. (2025). Learning Cross-Spectral Point Features with Task-Oriented Training. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"DeTone, D., Malisiewicz, T., and Rabinovich, A. (2018, January 18\u201322). SuperPoint: Self-Supervised Interest Point Detection and Description. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00060"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Sarlin, P.E., DeTone, D., Malisiewicz, T., and Rabinovich, A. (2020, January 13\u201319). SuperGlue: Learning Feature Matching With Graph Neural Networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00499"},{"key":"ref_14","unstructured":"Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., and El-Nouby, A. (2024). DINOv2: Learning Robust Visual Features without Supervision. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1016\/j.wasman.2025.02.032","article-title":"Generalization abilities of foundation models in waste classification","volume":"198","author":"Cuingnet","year":"2025","journal-title":"Waste Manag."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yuan, M., Cui, B., Zhao, T., Wang, J., Fu, S., Yang, X., and Wei, X. (2025). UniRGB-IR: A Unified Framework for Visible-Infrared Semantic Tasks via Adapter Tuning. Proceedings of the 33rd ACM International Conference on Multimedia, ACM.","DOI":"10.1145\/3746027.3754806"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhang, M., Wang, Y., Guo, J., Li, Y., Gao, X., and Zhang, J. (2024). IRSAM: Advancing Segment Anything Model for Infrared Small Target Detection. Proceedings of the Computer Vision\u2014ECCV 2024, Springer.","DOI":"10.1007\/978-3-031-72855-6_14"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1340","DOI":"10.1109\/TIP.2025.3541562","article-title":"MaeFuse: Transferring Omni Features with Pretrained Masked Autoencoders for Infrared and Visible Image Fusion via Guided Training","volume":"34","author":"Li","year":"2025","journal-title":"IEEE Trans. Image Process."},{"key":"ref_19","unstructured":"Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2022, January 25\u201329). LoRA: Low-Rank Adaptation of Large Language Models. Proceedings of the International Conference on Learning Representations (ICLR), Virtual."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.wasman.2016.09.015","article-title":"A review on automated sorting of source-separated municipal solid waste for recycling","volume":"60","author":"Gundupalli","year":"2017","journal-title":"Waste Manag."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Cao, J., Pang, J., Weng, X., Khirodkar, R., and Kitani, K. (2023, January 17\u201324). Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00934"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. (2016, January 25\u201328). Simple online and realtime tracking. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7533003"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature Pyramid Networks for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_26","unstructured":"Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., and Girshick, R. (2026, July 26). Detectron2. Available online: https:\/\/github.com\/facebookresearch\/detectron2."},{"key":"ref_27","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 3\u20137). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Proceedings of the International Conference on Learning Representations, ICLR 2021, Virtual."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"He, K., Chen, X., Xie, S., Li, Y., Doll\u00e1r, P., and Girshick, R. (2022, January 21\u201324). Masked Autoencoders Are Scalable Vision Learners. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"ref_30","unstructured":"Loshchilov, I., and Hutter, F. (2019, January 6\u20139). Decoupled Weight Decay Regularization. Proceedings of the International Conference on Learning Representations (ICLR), ICLR 2019, New Orleans, LA, USA."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"Sokolova","year":"2009","journal-title":"Inf. Process. Manag."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1007\/BF02295996","article-title":"Note on the sampling error of the difference between correlated proportions or percentages","volume":"12","author":"McNemar","year":"1947","journal-title":"Psychometrika"},{"key":"ref_33","first-page":"65","article-title":"A Simple Sequentially Rejective Multiple Test Procedure","volume":"6","author":"Holm","year":"1979","journal-title":"Scand. J. Stat."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/16\/5017\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T10:14:04Z","timestamp":1786097644000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/16\/5017"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,7]]},"references-count":33,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["s26165017"],"URL":"https:\/\/doi.org\/10.3390\/s26165017","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,7]]}}}