{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T14:46:01Z","timestamp":1767192361807,"version":"3.41.2"},"reference-count":73,"publisher":"American Association for the Advancement of Science (AAAS)","issue":"95","content-domain":{"domain":["www.science.org"],"crossmark-restriction":true},"short-container-title":["Sci. Robot."],"published-print":{"date-parts":[[2024,10,30]]},"abstract":"<jats:p>Rendezvous with sperm whales for biological observations is made challenging by their prolonged dive patterns. Here, we propose an algorithmic framework that codevelops multiagent reinforcement learning\u2013based routing (autonomy module) and synthetic aperture radar\u2013based very high frequency (VHF) signal\u2013based bearing estimation (sensing module) for maximizing rendezvous opportunities of autonomous robots with sperm whales. The sensing module is compatible with low-energy VHF tags commonly used for tracking wildlife. The autonomy module leverages in situ noisy bearing measurements of whale vocalizations, VHF tags, and whale dive behaviors to enable time-critical rendezvous of a robot team with multiple whales in simulation. We conducted experiments at sea in the native habitat of sperm whales using an \u201cengineered whale\u201d\u2014a speedboat equipped with a VHF-emitting tag, emulating five distinct whale tracks, with different whale motions. The sensing module shows a median bearing error of 10.55\u00b0 to the tag. Using bearing measurements to the engineered whale from an acoustic sensor and our sensing module, our autonomy module gives an aggregate rendezvous success rate of 81.31% for a 500-meter rendezvous distance using three robots in postprocessing. A second class of fielded experiments that used acoustic-only bearing measurements to three untagged sperm whales showed an aggregate rendezvous success rate of 68.68% for a 1000-meter rendezvous distance using two robots in postprocessing. We further validated these algorithms with several ablation studies using a sperm whale visual encounter dataset collected by marine biologists.<\/jats:p>","DOI":"10.1126\/scirobotics.adn7299","type":"journal-article","created":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T18:00:05Z","timestamp":1730311205000},"update-policy":"https:\/\/doi.org\/10.34133\/aaas_crossmark","source":"Crossref","is-referenced-by-count":3,"title":["Reinforcement learning\u2013based framework for whale rendezvous via autonomous sensing robots"],"prefix":"10.1126","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1101-3336","authenticated-orcid":true,"given":"Ninad","family":"Jadhav","sequence":"first","affiliation":[{"name":"Project CETI, New York, NY, USA."},{"name":"John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA."}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2288-8808","authenticated-orcid":true,"given":"Sushmita","family":"Bhattacharya","sequence":"additional","affiliation":[{"name":"Project CETI, New York, NY, USA."},{"name":"John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA."}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1286-7034","authenticated-orcid":true,"given":"Daniel","family":"Vogt","sequence":"additional","affiliation":[{"name":"Project CETI, New York, NY, USA."},{"name":"John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA."}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-5138-5309","authenticated-orcid":true,"given":"Yaniv","family":"Aluma","sequence":"additional","affiliation":[{"name":"Project CETI, New York, NY, USA."}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0869-6490","authenticated-orcid":true,"given":"Pernille","family":"T\u00f8nnesen","sequence":"additional","affiliation":[{"name":"Project CETI, New York, NY, USA."},{"name":"Zoophysiology, Department of Biology, Aarhus University, 8000 Aarhus, Denmark."}]},{"given":"Akarsh","family":"Prabhakara","sequence":"additional","affiliation":[{"name":"College of Engineering, Carnegie Mellon University, Pittsburgh, PA, USA."}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5398-5347","authenticated-orcid":true,"given":"Swarun","family":"Kumar","sequence":"additional","affiliation":[{"name":"College of Engineering, Carnegie Mellon University, Pittsburgh, PA, USA."}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6854-044X","authenticated-orcid":true,"given":"Shane","family":"Gero","sequence":"additional","affiliation":[{"name":"Project CETI, New York, NY, USA."},{"name":"Department of Biology, Carleton University, Ottawa, Ontario, Canada."}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7969-038X","authenticated-orcid":true,"given":"Robert J.","family":"Wood","sequence":"additional","affiliation":[{"name":"Project CETI, New York, NY, USA."},{"name":"John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA."}]},{"given":"Stephanie","family":"Gil","sequence":"additional","affiliation":[{"name":"Project CETI, New York, NY, USA."},{"name":"John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA."}]}],"member":"221","reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.anbehav.2015.01.008"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10764-012-9598-z"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-47221-8"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.150372"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","unstructured":"L. Marino \u201cCetacean cognition\u201d in The Oxford Handbook of Animal Studies L. Kalof Ed. (Oxford Univ. Press 2017) pp. 227\u2013239.","DOI":"10.1093\/oxfordhb\/9780199927142.013.22"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1098\/rspb.2002.2239"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2201692119"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","unstructured":"M. Cantor S. Gero H. Whitehead L. Rendell \u201cSperm whale: The largest toothed creature on Earth\u201d in Ethology and Behavioral Ecology of Odontocetes C. W. Clark E. C. Garland Eds. (Springer 2019) pp. 261\u2013280.","DOI":"10.1007\/978-3-030-16663-2_12"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/JOE.2002.808212"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2656.2006.01101.x"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00265-018-2570-y"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1098\/rsbl.2020.0134"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1121\/1.4949478"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tree.2010.08.002"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.adg7314"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2022.104393"},{"key":"e_1_3_2_18_2","unstructured":"R. Payne \u201cI spent my life saving the whales. Now they might save us \u201d Time 5 June 2023; https:\/\/time.com\/6284884\/whale-scientist-last-please-save-the-species\/."},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","unstructured":"J. Andreas G. Begu\u0161 M. M. Bronstein R. Diamant D. Delaney S. Gero S. Goldwasser D. F. Gruber S. de Haas P. Malkin R. Payne G. Petri D. Rus P. Sharma D. Tchernov P. T\u00f8nnesen A. Torralba D. Vogt R. J. Wood Cetacean translation initiative: a roadmap to deciphering the communication of sperm whales. arXiv:2104.08614 [cs.SD] (2021).","DOI":"10.1016\/j.isci.2022.104393"},{"key":"e_1_3_2_20_2","unstructured":"The Dominica Sperm Whale Project; www.thespermwhaleproject.org."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1111\/mms.12086"},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","unstructured":"A. Torabi M. W. Shafer G. S. Vega K. M. Rothfus \u201cUAV-RT: An SDR based aerial platform for wildlife tracking\u201d in 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall) (IEEE 2018) pp. 1\u20136.","DOI":"10.1109\/VTCFall.2018.8690711"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21857"},{"key":"e_1_3_2_24_2","doi-asserted-by":"crossref","unstructured":"H. Bayram N. Stefas K. S. Engin V. Isler \u201cTracking wildlife with multiple UAVs: System design safety and field experiments\u201d in 2017 International Symposium on Multi-Robot and Multi-Agent Systems (MRS) (IEEE 2017) pp. 97\u2013103.","DOI":"10.1109\/MRS.2017.8250937"},{"key":"e_1_3_2_25_2","doi-asserted-by":"crossref","unstructured":"H. V. Nguyen F. Chen J. A. Chesser H. Rezatofighi D. C. Ranasinghe \u201cLavapilot: Lightweight UAV trajectory planner with situational awareness for embedded autonomy to track and locate radio-tags\u201d in 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE 2020) pp. 2488\u20132495.","DOI":"10.1109\/IROS45743.2020.9341615"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1071\/WR10211"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","unstructured":"O. M. Cliff R. C. Fitch S. Sukkarieh D. Saunders R. Heinsohn \u201cOnline localization of radio-tagged wildlife with an autonomous aerial robot system\u201d in Proceedings of Robotics: Science and Systems (RSS Foundation 2015) 10.15607\/RSS.2015.XI.042.","DOI":"10.15607\/RSS.2015.XI.042"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1111\/2041-210X.13261"},{"key":"e_1_3_2_29_2","doi-asserted-by":"crossref","unstructured":"F. K\u00f6rner R. Speck A. G\u00f6ktogan S. Sukkarieh \u201cAutonomous airborne wildlife tracking using radio signal strength\u201d in 2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IEEE 2010) pp. 107\u2013112.","DOI":"10.1109\/IROS.2010.5654385"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3268631"},{"key":"e_1_3_2_31_2","unstructured":"S. Goldwasser D. F. Gruber A. T. Kalai O. Paradise \u201cA theory of unsupervised translation motivated by understanding animal communication\u201d in Advances in Neural Information Processing Systems 36 (NeurIPS 2023) A. Oh T. Naumann A. Globerson K. Saenko M. Hardt S. Levine Eds. (Curran Associates 2023) pp. 37286\u201337320."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1111\/2041-210X.13935"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2021.1003814"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2023.3347128"},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","unstructured":"D. Garces S. Bhattacharya S. Gil D. P. Bertsekas \u201cMultiagent reinforcement learning for autonomous routing and pickup problem with adaptation to variable demand\u201d in 2023 IEEE International Conference on Robotics and Automation (ICRA) (IEEE 2023) pp. 3524\u20133531.","DOI":"10.1109\/ICRA48891.2023.10161067"},{"key":"e_1_3_2_36_2","doi-asserted-by":"crossref","unstructured":"D. Garces S. Bhattacharya D. Bertsekas S. Gil \u201cApproximate multiagent reinforcement learning for on-demand urban mobility problem on a large map \u201d in 2024 IEEE International Conference on Robotics and Automation (ICRA) (IEEE 2024) pp. 6843\u20136849.","DOI":"10.1109\/ICRA57147.2024.10611063"},{"key":"e_1_3_2_37_2","unstructured":"S. Bhattacharya S. Kailas S. Badyal S. Gil D. P. Bertsekas \u201cMultiagent rollout and policy iteration for pomdp with application to multi-robot repair problems\u201d in Proceedings of the 2020 Conference on Robot Learning J. Kober F. Ramos C. Tomlin Eds. (MLResearchPress 2021) pp. 1814\u20131828."},{"key":"e_1_3_2_38_2","unstructured":"D. Bertsekas Lessons from AlphaZero for Optimal Model Predictive and Adaptive Control (Athena Scientific 2022)."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2978451"},{"key":"e_1_3_2_40_2","unstructured":"D. Silver J. Veness \u201cMonte-Carlo planning in large POMDPs \u201d Advances in Neural Information Processing Systems 23 (NeurIPS 2010) J. Lafferty C. Williams J. Shawe-Taylor R. Zemel A. Culotta Eds. (Curran Associates 2010)."},{"key":"e_1_3_2_41_2","unstructured":"D. Garces S. Gil \u201cSurge routing: Event-informed multiagent reinforcement learning for autonomous rideshare\u201d in AAMAS \u201924:Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (ACM 2024) pp. 641\u2013650."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364914567793"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1177\/02783649221097989"},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","unstructured":"S. Kumar S. Gil D. Katabi D. Rus \u201cAccurate indoor localization with zero start-up cost\u201d in MobiCom \u201914: Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (ACM 2014) pp. 483\u2013494.","DOI":"10.1145\/2639108.2639142"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-017-9621-5"},{"key":"e_1_3_2_46_2","doi-asserted-by":"crossref","unstructured":"W. Wang N. Jadhav P. A. Vohs N. Hughes M. Mazumder S. Gil \u201cActive rendezvous for multi-robot pose graph optimization using sensing over Wi-Fi\u201d in Robotics Research: The 19th International Symposium ISR T. Asfour E. Yoshida J. Park H. Christensen O. Khatib Eds. vol. 20 of Springer Proceedings in Advanced Robotics (Springer 2022) pp. 832\u2013849.","DOI":"10.1007\/978-3-030-95459-8_51"},{"key":"e_1_3_2_47_2","unstructured":"D. E. Davies \u201cCircular arrays\u201d in The Handbook of Antenna Design Volume 2 A. W. Rudge K. Milne A. D. Olver P. Knight Eds. vol. 28 of Electromagnetic Waves Series (London Peregrinus on behalf of the Institution of Electrical Engineers 1983) chap. 12."},{"key":"e_1_3_2_48_2","unstructured":"D. Tse P. Viswanath Fundamentals of Wireless Communication (Cambridge Univ. Press 2012)."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1121\/10.0010546"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1121\/1.2935394"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.221376"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/s42991-021-00221-3"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2024.3379899"},{"key":"e_1_3_2_54_2","doi-asserted-by":"crossref","unstructured":"E. Dubrovinskaya R. Diamant P. Casari \u201cAnchorless underwater acoustic localization\u201d in 2017 14th Workshop on Positioning Navigation and Communications (WPNC) (IEEE 2017) pp. 1\u20136.","DOI":"10.1109\/WPNC.2017.8250051"},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","unstructured":"B. B. Werger M. J. Matari\u0107 \u201cBroadcast of local eligibility for multi-target observation\u201d in Distributed Autonomous Robotic Systems 4 L. E. Parker G. Bekey J. Barhen Eds. (Springer 2000) pp. 347\u2013356.","DOI":"10.1007\/978-4-431-67919-6_33"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.6.1.80"},{"key":"e_1_3_2_57_2","unstructured":"V. Furnon L. Perron Or-Tools Routing Library; https:\/\/developers.google.com\/optimization\/routing\/."},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2016.7378438"},{"key":"e_1_3_2_59_2","doi-asserted-by":"crossref","unstructured":"H. Bayram N. Stefas V. Isler \u201cAerial radio-based telemetry for tracking wildlife\u201d in 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE 2018) pp. 4723\u20134728.","DOI":"10.1109\/IROS.2018.8594503"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2024.3366013"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.add8662"},{"key":"e_1_3_2_62_2","doi-asserted-by":"crossref","unstructured":"E. A. Wan R. Van Der Merwe \u201cThe unscented Kalman filter for nonlinear estimation\u201d in Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing Communications and Control Symposium (Cat. No. 00EX373) (IEEE 2000) pp. 153\u2013158.","DOI":"10.1109\/ASSPCC.2000.882463"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/MAP.2005.1589932"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/79.526899"},{"key":"e_1_3_2_65_2","unstructured":"P. Stoica R. L. Moses Spectral Analysis of Signals (Prentice Hall 2005)."},{"key":"e_1_3_2_66_2","unstructured":"Mavros https:\/\/github.com\/mavlink\/mavros."},{"key":"e_1_3_2_67_2","unstructured":"SoapySDR Project https:\/\/github.com\/pothosware\/SoapySDR\/wiki."},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1748-7692.2006.00060.x"},{"key":"e_1_3_2_69_2","first-page":"1","article-title":"Change of form in the sperm whale accompanied with growth","volume":"17","author":"Nishiwaki M.","year":"1963","unstructured":"M. Nishiwaki, S. Ohsumi, Y. Maeda, Change of form in the sperm whale accompanied with growth. Sci. Rep. Whales Res. Inst. Tokyo 17, 1\u201314 (1963).","journal-title":"Sci. Rep. Whales Res. Inst. Tokyo"},{"key":"e_1_3_2_70_2","first-page":"1","article-title":"Sperm whales of the southeast Pacific part III: Morphometry","volume":"53","author":"Clarke R.","year":"1972","unstructured":"R. Clarke, O. Paliza, Sperm whales of the southeast Pacific part III: Morphometry. Hvalrad. Skrift. Sci. Results Mar. Biol. Res. 53, 1\u2013106 (1972).","journal-title":"Hvalrad. Skrift. Sci. Results Mar. Biol. Res."},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1469-7998.1991.tb04807.x"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1121\/1.4808713"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1121\/1.4818896"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-30731-8"}],"container-title":["Science Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.science.org\/doi\/pdf\/10.1126\/scirobotics.adn7299","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,9]],"date-time":"2024-12-09T21:36:19Z","timestamp":1733780179000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.adn7299"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,30]]},"references-count":73,"journal-issue":{"issue":"95","published-print":{"date-parts":[[2024,10,30]]}},"alternative-id":["10.1126\/scirobotics.adn7299"],"URL":"https:\/\/doi.org\/10.1126\/scirobotics.adn7299","relation":{},"ISSN":["2470-9476"],"issn-type":[{"type":"electronic","value":"2470-9476"}],"subject":[],"published":{"date-parts":[[2024,10,30]]},"assertion":[{"value":"2023-12-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-07","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"eadn7299"}}