{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T22:20:54Z","timestamp":1778624454073,"version":"3.51.4"},"reference-count":55,"publisher":"The Royal Society","issue":"1869","license":[{"start":{"date-parts":[[2022,12,13]],"date-time":"2022-12-13T00:00:00Z","timestamp":1670889600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/royalsociety.org\/journals\/ethics-policies\/data-sharing-mining\/"}],"funder":[{"DOI":"10.13039\/501100000267","name":"Arts and Humanities Research Council","doi-asserted-by":"crossref","award":["AH\/N006011\/1"],"award-info":[{"award-number":["AH\/N006011\/1"]}],"id":[{"id":"10.13039\/501100000267","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"crossref","award":["EP\/N019423\/1"],"award-info":[{"award-number":["EP\/N019423\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["royalsocietypublishing.org"],"crossmark-restriction":true},"short-container-title":["Phil. Trans. R. Soc. B"],"published-print":{"date-parts":[[2023,1,30]]},"abstract":"<jats:p>\n                    It is often assumed that the brain builds 3D coordinate frames, in retinal coordinates (with binocular disparity giving the third dimension), head-centred, body-centred and world-centred coordinates. This paper questions that assumption and begins to sketch an alternative based on, essentially, a set of reflexes. A \u2018policy network\u2019 is a term used in reinforcement learning to describe the set of actions that are generated by an agent depending on its current state. This is an untypical starting point for describing 3D vision, but a policy network can serve as a useful representation both for the 3D layout of a scene and the location of the observer within it. It avoids 3D reconstruction of the type used in computer vision but is similar to recent representations for navigation generated through reinforcement learning. A policy network for saccades (pure rotations of the camera\/eye) is a logical starting point for understanding (i) an ego-centric representation of space (e.g. Marr\u2019s (Marr 1982\n                    <jats:italic>Vision: a computational investigation into the human representation and processing of visual information<\/jats:italic>\n                    ) 2\n                    <jats:inline-formula>\n                      <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                        <mml:mrow>\n                          <mml:mfrac>\n                            <mml:mn>1<\/mml:mn>\n                            <mml:mn>2<\/mml:mn>\n                          <\/mml:mfrac>\n                        <\/mml:mrow>\n                      <\/mml:math>\n                    <\/jats:inline-formula>\n                    -D sketch) and (ii) a hierarchical, compositional representation for navigation. The potential neural implementation of policy networks is straightforward; a network with a large range of sensory and task-related inputs such as the cerebellum would be capable of implementing this input\/output function. This is not the case for 3D coordinate transformations in the brain: no neurally implementable proposals have yet been put forward that could carry out a transformation of a visual scene from retinal to world-based coordinates. Hence, if the representation underlying 3D vision can be described as a policy network (in which the actions are either saccades or head translations), this would be a significant step towards a neurally plausible model of 3D vision.\n                  <\/jats:p>\n                  <jats:p>This article is part of the theme issue \u2018New approaches to 3D vision\u2019.<\/jats:p>","DOI":"10.1098\/rstb.2021.0448","type":"journal-article","created":{"date-parts":[[2022,12,13]],"date-time":"2022-12-13T07:00:53Z","timestamp":1670914853000},"update-policy":"https:\/\/doi.org\/10.1098\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Understanding 3D vision as a policy network"],"prefix":"10.1098","volume":"378","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8674-2763","authenticated-orcid":true,"given":"Andrew","family":"Glennerster","sequence":"first","affiliation":[{"name":"School of Psychology and Clinical Language Sciences, University of Reading, RG6 6AL Reading, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"175","published-online":{"date-parts":[[2022,12,13]]},"reference":[{"key":"e_1_3_4_2_2","doi-asserted-by":"publisher","DOI":"10.1162\/089892998562861"},{"key":"e_1_3_4_3_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008821210922"},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0010-0277(98)00026-2"},{"key":"e_1_3_4_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(96)00090-9"},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.1364\/JOSAA.8.000377"},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2008.10.008"},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2015.0265"},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","DOI":"10.1242\/jeb.187971"},{"key":"e_1_3_4_10_2","doi-asserted-by":"crossref","unstructured":"Arulkumaran K Deisenroth MP Brundage M Bharath AA. 2017 A brief survey of deep reinforcement learning. (http:\/\/arxiv.org\/abs\/1708.05866)","DOI":"10.1109\/MSP.2017.2743240"},{"key":"e_1_3_4_11_2","doi-asserted-by":"crossref","unstructured":"Gandhi D Pinto L Gupta A. 2017 Learning to fly by crashing. In 2017 IEEE\/RSJ Int. Conf. on Intelligent Robots and Systems (IROS) 24\u201328th September 2017 Vancouver BC Canada pp. 3948\u20133955. New York NY: IEEE.","DOI":"10.1109\/IROS.2017.8206247"},{"key":"e_1_3_4_12_2","unstructured":"Mirowski P et al. 2018 Learning to navigate in cities without a map. Advances in neural information processing systems vol. 31 (eds S Bengio H Wallach H Larochelle K Grauman N Cesa-Bianchi N R Garnett). Red Hook NY: Curran Associates."},{"key":"e_1_3_4_13_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-018-0102-6"},{"key":"e_1_3_4_14_2","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1969.sp008820"},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/0025-5564(71)90051-4"},{"key":"e_1_3_4_16_2","unstructured":"Krizhevsky A Sutskever I Hinton GE. 2012 Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems vol. 25 (eds F Pereira CJ Burges L Bottou KQ Weinberger). Red Hook NY: Curran Associates."},{"key":"e_1_3_4_17_2","first-page":"303","article-title":"Multi-modal representation of space in the posterior parietal cortex and its use in planning movements","volume":"20","author":"Andersen RA","year":"1997","unstructured":"Andersen RA, Snyder LH, Bradley DC, Xing J. 1997 Multi-modal representation of space in the posterior parietal cortex and its use in planning movements. ARN 20, 303-330. (doi:10.1146\/annurev.neuro.20.1.303)","journal-title":"ARN"},{"key":"e_1_3_4_18_2","volume-title":"The hippocampus as a cognitive map","author":"O\u2019Keefe J","year":"1978","unstructured":"O\u2019Keefe J, Nadel L. 1978 The hippocampus as a cognitive map. Oxford, UK: Oxford University Press."},{"key":"e_1_3_4_19_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature03721"},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.114.2.340"},{"key":"e_1_3_4_21_2","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.23-22-08135.2003"},{"key":"e_1_3_4_22_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-48379-8"},{"key":"e_1_3_4_23_2","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.18-04-01583.1998"},{"key":"e_1_3_4_24_2","doi-asserted-by":"publisher","DOI":"10.1152\/jn.2001.86.4.2112"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2010.0184"},{"key":"e_1_3_4_26_2","doi-asserted-by":"publisher","DOI":"10.1038\/nrn914"},{"key":"e_1_3_4_27_2","doi-asserted-by":"publisher","DOI":"10.1093\/cercor\/bhn177"},{"key":"e_1_3_4_28_2","doi-asserted-by":"publisher","DOI":"10.1038\/331679a0"},{"key":"e_1_3_4_29_2","volume-title":"Multiple view geometry in computer vision","author":"Hartley R","year":"2000","unstructured":"Hartley R, Zisserman A. 2000 Multiple view geometry in computer vision. UK, Cambridge: Cambridge University Press."},{"key":"e_1_3_4_30_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511811685"},{"key":"e_1_3_4_31_2","doi-asserted-by":"publisher","DOI":"10.1068\/p2935"},{"key":"e_1_3_4_32_2","doi-asserted-by":"publisher","DOI":"10.1364\/JOSAA.4.002006"},{"key":"e_1_3_4_33_2","volume-title":"Visual processing: computational, psychophysical and cognitive research","author":"Watt RJ","year":"1988","unstructured":"Watt RJ. 1988 Visual processing: computational, psychophysical and cognitive research. Hove, UK: Lawrence Erlbaum Associates."},{"key":"e_1_3_4_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2006.10.005"},{"key":"e_1_3_4_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.visres.2020.05.009"},{"key":"e_1_3_4_36_2","doi-asserted-by":"crossref","unstructured":"Zhu Y Mottaghi R Kolve E Lim JJ Gupta A Fei-Fei L Farhadi A. 2017 Target-driven visual navigation in indoor scenes using deep reinforcement learning. In Robotics and automation (ICRA) 2017 May 29\u2013June 3 Marina Bay Sands Singapore pp. 3357\u20133364. New York NY: IEEE.","DOI":"10.1109\/ICRA.2017.7989381"},{"key":"e_1_3_4_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0042-6989(00)00300-X"},{"key":"e_1_3_4_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/5.664279"},{"key":"e_1_3_4_39_2","volume-title":"Vision: a computational investigation into the human representation and processing of visual information","author":"Marr D","year":"1982","unstructured":"Marr D. 1982 Vision: a computational investigation into the human representation and processing of visual information. New York, NY: W.H. Freemann and Company."},{"key":"e_1_3_4_40_2","doi-asserted-by":"publisher","DOI":"10.1098\/rspb.1978.0020"},{"key":"e_1_3_4_41_2","doi-asserted-by":"publisher","DOI":"10.1068\/p3041"},{"key":"e_1_3_4_42_2","first-page":"1","article-title":"Cue combination for 3D location judgements","volume":"10","author":"Svarverud E","year":"2010","unstructured":"Svarverud E, Gilson SJ, Glennerster A. 2010 Cue combination for 3D location judgements. J. Vis. 10, 1-13. (doi:10.1167\/10.1.5)","journal-title":"J. Vis."},{"key":"e_1_3_4_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0042-6989(98)00139-4"},{"key":"e_1_3_4_44_2","unstructured":"Glennerster A Read JC. 2018 A single coordinate framework for optic flow and binocular disparity. (http:\/\/arxiv.org\/abs\/1808.03875)"},{"key":"e_1_3_4_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(87)90191-X"},{"key":"e_1_3_4_46_2","doi-asserted-by":"publisher","DOI":"10.1364\/JOSA.70.000772"},{"key":"e_1_3_4_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0042-6989(98)00324-1"},{"key":"e_1_3_4_48_2","doi-asserted-by":"publisher","DOI":"10.1167\/4.5.3"},{"key":"e_1_3_4_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.visres.2006.07.011"},{"key":"e_1_3_4_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-03061-1_10"},{"key":"e_1_3_4_51_2","doi-asserted-by":"crossref","unstructured":"Muryy A Glennerster A. 2018 Pointing errors in non-metric virtual environments. In Spatial Cognition XI. Spatial Cognition 2018. Lecture Notes in Computer Science vol. 11034 (eds S Creem-Regehr J Sch\u00f6ning A Klippel A). Cham Switzerland: Springer.","DOI":"10.1101\/273532"},{"key":"e_1_3_4_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2015.02.037"},{"key":"e_1_3_4_53_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2015.07.006"},{"key":"e_1_3_4_54_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aar6170"},{"key":"e_1_3_4_55_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2013.08.013"},{"key":"e_1_3_4_56_2","volume-title":"Consciousness explained","author":"Dennett DC","year":"1991","unstructured":"Dennett DC. 1991 Consciousness explained. Boston, MA: Little, Brown and Co."}],"container-title":["Philosophical Transactions of the Royal Society B: Biological Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/royalsocietypublishing.org\/doi\/pdf\/10.1098\/rstb.2021.0448","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/royalsocietypublishing.org\/doi\/full-xml\/10.1098\/rstb.2021.0448","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/royalsocietypublishing.org\/doi\/pdf\/10.1098\/rstb.2021.0448","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T21:24:06Z","timestamp":1778621046000},"score":1,"resource":{"primary":{"URL":"http:\/\/royalsocietypublishing.org\/rstb\/article\/42672"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,13]]},"references-count":55,"journal-issue":{"issue":"1869","published-print":{"date-parts":[[2023,1,30]]}},"alternative-id":["10.1098\/rstb.2021.0448"],"URL":"https:\/\/doi.org\/10.1098\/rstb.2021.0448","relation":{"has-preprint":[{"id-type":"doi","id":"10.31234\/osf.io\/43v57","asserted-by":"object"}]},"ISSN":["0962-8436","1471-2970"],"issn-type":[{"value":"0962-8436","type":"print"},{"value":"1471-2970","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,13]]},"assertion":[{"value":"2022-04-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-08-02","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-12-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"20210448"}}