{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T23:17:57Z","timestamp":1778541477863,"version":"3.51.4"},"reference-count":27,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,12,1]],"date-time":"2022-12-01T00:00:00Z","timestamp":1669852800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Road infrastructure is one of the most vital assets of any country. Keeping the road infrastructure clean and unpolluted is important for ensuring road safety and reducing environmental risk. However, roadside litter picking is an extremely laborious, expensive, monotonous and hazardous task. Automating the process would save taxpayers money and reduce the risk for road users and the maintenance crew. This work presents LitterBot, an autonomous robotic system capable of detecting, localizing and classifying common roadside litter. We use a learning-based object detection and segmentation algorithm trained on the TACO dataset for identifying and classifying garbage. We develop a robust modular manipulation framework by using soft robotic grippers and a real-time visual-servoing strategy. This enables the manipulator to pick up objects of variable sizes and shapes even in dynamic environments. The robot achieves greater than 80% classified picking and binning success rates for all experiments; which was validated on a wide variety of test litter objects in static single and cluttered configurations and with dynamically moving test objects. Our results showcase how a deep model trained on an online dataset can be deployed in real-world applications with high accuracy by the appropriate design of a control framework around it.<\/jats:p>","DOI":"10.3389\/frobt.2022.1064853","type":"journal-article","created":{"date-parts":[[2022,12,1]],"date-time":"2022-12-01T19:27:34Z","timestamp":1669922854000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Autonomous detection and sorting of litter using deep learning and soft robotic grippers"],"prefix":"10.3389","volume":"9","author":[{"given":"Elijah","family":"Almanzor","sequence":"first","affiliation":[]},{"given":"Nzebo Richard","family":"Anvo","sequence":"additional","affiliation":[]},{"given":"Thomas George","family":"Thuruthel","sequence":"additional","affiliation":[]},{"given":"Fumiya","family":"Iida","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2022,12,1]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"382","DOI":"10.1109\/tce.2018.2859629","article-title":"Deep learning based robot for automatically picking up garbage on the grass","volume":"64","author":"Bai","year":"2018","journal-title":"IEEE Trans. Consum. Electron."},{"key":"B2","volume-title":"Roadside litter: Who should pick up the bill?","year":"2018"},{"key":"B3","volume-title":"The cost of cleaning up after car litterbugs: Our \u00a34.8m a year bill to remove rubbish from beside motorways and a-roads","year":"2016"},{"key":"B4","doi-asserted-by":"publisher","first-page":"18809","DOI":"10.1073\/pnas.1003250107","article-title":"Universal robotic gripper based on the jamming of granular material","volume":"107","author":"Brown","year":"2010","journal-title":"Proc. Natl. Acad. Sci. U. S. A."},{"key":"B5","doi-asserted-by":"publisher","first-page":"70","DOI":"10.3389\/frobt.2016.00070","article-title":"Fin ray\u00ae effect inspired soft robotic gripper: From the robosoft grand challenge toward optimization","volume":"3","author":"Crooks","year":"2016","journal-title":"Front. Robot. AI"},{"key":"B6","volume-title":"Crackdown on rubbish drivers! selfish motorists are told to stop using roads as a dump","year":"2021"},{"key":"B7","doi-asserted-by":"publisher","first-page":"631","DOI":"10.1177\/0950017018817307","article-title":"Can dirty work be satisfying? A mixed method study of workers doing dirty jobs","volume":"33","author":"Deery","year":"2019","journal-title":"Work, Employ. Soc."},{"key":"B8","volume-title":"Detectron2 at facebook research","year":"2021"},{"key":"B9","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/j.wasman.2017.09.019","article-title":"Multi-material classification of dry recyclables from municipal solid waste based on thermal imaging","volume":"70","author":"Gundupalli","year":"2017","journal-title":"Waste Manag."},{"key":"B10","volume-title":"2020 national litter study","year":"2020"},{"key":"B11","volume-title":"The issues of roadside litter: A review paper","author":"Karimi","year":"2021"},{"key":"B12","volume-title":"What a waste 2.0","author":"Kaza","year":"2018"},{"key":"B13","first-page":"1","article-title":"Robot based indoor autonomous trash detection algorithm using ultrasonic sensors","author":"Kulkarni","year":"2013"},{"key":"B14","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1016\/j.wasman.2020.01.011","article-title":"The toxicity and valorization options of cigarette butts","volume":"104","author":"Kurmus","year":"2020","journal-title":"Waste Manag."},{"key":"B15","first-page":"408","article-title":"Garbage collection and sorting with a mobile manipulator using deep learning and whole-body control","author":"Liu","year":"2021"},{"key":"B16","first-page":"1","article-title":"Zenrobotics recycler\u2013robotic sorting using machine learning","author":"Lukka","year":"2014"},{"key":"B17","doi-asserted-by":"publisher","first-page":"2163","DOI":"10.1002\/adfm.201303288","article-title":"Pneumatic networks for soft robotics that actuate rapidly","volume":"24","author":"Mosadegh","year":"2014","journal-title":"Adv. Funct. Mater."},{"key":"B18","volume-title":"Driving down roadside litter levels in national spring clean","year":"2022"},{"key":"B19","doi-asserted-by":"publisher","first-page":"2411","DOI":"10.3390\/ijerph16132411","article-title":"Solutions and integrated strategies for the control and mitigation of plastic and microplastic pollution","volume":"16","author":"Prata","year":"2019","journal-title":"Int. J. Environ. Res. Public Health"},{"key":"B20","volume-title":"Taco: Trash annotations in context for litter detection","author":"Proen\u00e7a","year":"2020"},{"key":"B21","doi-asserted-by":"publisher","first-page":"2595","DOI":"10.3390\/s21082595","article-title":"Deep learning based pavement inspection using self-reconfigurable robot","volume":"21","author":"Ramalingam","year":"2021","journal-title":"Sensors"},{"key":"B22","first-page":"1149","article-title":"Robotic pick-and-toss facilitates urban waste sorting","author":"Raptopoulos","year":"2020"},{"key":"B23","first-page":"1","article-title":"A universal gripper using optical sensing to acquire tactile information and membrane deformation","author":"Sakuma","year":"2018"},{"key":"B24","article-title":"Is the UK the litter lout of Europe?","author":"Smith","year":"2022"},{"key":"B25","volume-title":"The prevalence of motor vehicle crashes involving road debris","author":"Tefft","year":"2016"},{"key":"B26","doi-asserted-by":"publisher","first-page":"2045","DOI":"10.1038\/s41598-021-81499-8","article-title":"Microplastics in fish and fishmeal: An emerging environmental challenge?","volume":"11","author":"Thiele","year":"2021","journal-title":"Sci. Rep."},{"key":"B27","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1016\/j.conbuildmat.2014.09.109","article-title":"Upgrading the quality of mixed recycled aggregates from construction and demolition waste by using near-infrared sorting technology","volume":"75","author":"Vegas","year":"2015","journal-title":"Constr. Build. Mater."}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2022.1064853\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,1]],"date-time":"2022-12-01T19:27:41Z","timestamp":1669922861000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2022.1064853\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,1]]},"references-count":27,"alternative-id":["10.3389\/frobt.2022.1064853"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2022.1064853","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,1]]},"article-number":"1064853"}}