{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T05:29:47Z","timestamp":1761629387470,"version":"build-2065373602"},"reference-count":70,"publisher":"Annual Reviews","issue":"1","license":[{"start":{"date-parts":[[2024,7,10]],"date-time":"2024-07-10T00:00:00Z","timestamp":1720569600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,7,10]]},"abstract":"<jats:p>This article makes the case that a powerful new discipline, which we term perception engineering, is steadily emerging. It follows from a progression of ideas that involve creating illusions, from historical paintings and film to modern video games and virtual reality. Rather than creating physical artifacts such as bridges, airplanes, or computers, perception engineers create illusory perceptual experiences. The scope is defined over any agent that interacts with the physical world, including both biological organisms (humans and animals) and engineered systems (robots and autonomous systems). The key idea is that an agent, called a producer, alters the environment with the intent to alter the perceptual experience of another agent, called a receiver. Most importantly, the article introduces a precise mathematical formulation of this process, based on the von Neumann\u2013Morgenstern notion of information, to help scope and define the discipline. This formulation is then applied to the cases of engineered and biological agents, with discussion of its implications for existing fields such as virtual reality, robotics, and even social media. Finally, open challenges and opportunities for involvement are identified.<\/jats:p>","DOI":"10.1146\/annurev-control-062323-102456","type":"journal-article","created":{"date-parts":[[2023,11,6]],"date-time":"2023-11-06T15:23:02Z","timestamp":1699284182000},"page":"409-436","source":"Crossref","is-referenced-by-count":3,"title":["From Virtual Reality to the Emerging Discipline of Perception Engineering"],"prefix":"10.1146","volume":"7","author":[{"given":"Steven M.","family":"LaValle","sequence":"first","affiliation":[{"name":"Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; email: steven.lavalle@oulu.fi"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Evan G.","family":"Center","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; email: steven.lavalle@oulu.fi"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Timo","family":"Ojala","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; email: steven.lavalle@oulu.fi"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matti","family":"Pouke","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; email: steven.lavalle@oulu.fi"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nicoletta","family":"Prencipe","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; email: steven.lavalle@oulu.fi"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Basak","family":"Sakcak","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; email: steven.lavalle@oulu.fi"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Markku","family":"Suomalainen","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; email: steven.lavalle@oulu.fi"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kalle G.","family":"Timperi","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; email: steven.lavalle@oulu.fi"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vadim","family":"Weinstein","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland; email: steven.lavalle@oulu.fi"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"22","reference":[{"key":"B1","first-page":"81","article-title":"A separate reality: an update on place illusion and plausibility in virtual reality","volume":"3","year":"2022","journal-title":"Front. Virtual Real."},{"volume-title":"\u00dcber das Organ der Seele","year":"1796","key":"B2"},{"key":"B3","first-page":"19","article-title":"Through the looking glass: philosophical toys and digital visual effects","volume":"4","year":"2010","journal-title":"Projections"},{"issue":"5","key":"B4","doi-asserted-by":"crossref","first-page":"2073","DOI":"10.1109\/TVCG.2020.2973063","article-title":"Animals in virtual environments","volume":"26","year":"2020","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"volume-title":"Theory of Games and Economic Behavior","year":"1944","key":"B5"},{"volume-title":"Planning Algorithms","year":"2006","key":"B6"},{"key":"B7","first-page":"256","article-title":"The limits of learning and planning: minimal sufficient information transition systems","volume-title":"Algorithmic Foundations of Robotics XV","year":"2023"},{"issue":"4","key":"B8","first-page":"253","article-title":"Sensing and filtering: a fresh perspective based on preimages and information spaces","volume":"1","year":"2012","journal-title":"Found. Trends Robot."},{"key":"B9","doi-asserted-by":"crossref","first-page":"846982","DOI":"10.3389\/fnbot.2022.846982","article-title":"An enactivist-inspired mathematical model of cognition","volume":"16","year":"2022","journal-title":"Front. Neurorobot."},{"volume-title":"Probabilistic Robotics","year":"2005","key":"B10"},{"issue":"6","key":"B11","first-page":"1","article-title":"A survey of presence and related concepts","volume":"50","year":"2017","journal-title":"ACM Comput. Surv."},{"issue":"5098","key":"B12","first-page":"1627","article-title":"Combining sensory information: mandatory fusion within, but not between, senses","volume":"298","year":"2002","journal-title":"Science"},{"issue":"2","key":"B13","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1016\/j.automatica.2008.08.008","article-title":"Temporal logic motion planning for dynamic mobile robots","volume":"45","year":"2009","journal-title":"Automatica"},{"key":"B14","first-page":"445","article-title":"Illusion and dazzle: adversarial optical channel exploits against lidars for automotive applications","volume-title":"Cryptographic Hardware and Embedded Systems\u2014CHES 2017","year":"2017"},{"key":"B15","doi-asserted-by":"crossref","first-page":"11458","DOI":"10.1109\/IROS45743.2020.9341344","article-title":"Virtual reality for robots","volume-title":"2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","year":"2020"},{"key":"B16","first-page":"3","article-title":"WALNUT: waging doubt on the integrity of MEMS accelerometers with acoustic injection attacks","volume-title":"2017 IEEE European Symposium on Security and Privacy","year":"2017"},{"key":"B17","doi-asserted-by":"crossref","first-page":"10327","DOI":"10.1109\/ICRA40945.2020.9196761","article-title":"Reality as a simulation of reality: robot illusions, fundamental limits, and a physical demonstration","volume-title":"2020 IEEE International Conference on Robotics and Automation (ICRA)","year":"2020"},{"volume-title":"Virtual Reality","year":"2023","key":"B18"},{"issue":"1\u20133","key":"B19","first-page":"70","article-title":"A free energy principle for the brain","volume":"100","year":"2006","journal-title":"J. Physiol. Paris"},{"issue":"1038","key":"B20","first-page":"181","article-title":"Perceptions as hypotheses","volume":"290","year":"1980","journal-title":"Philos. Trans. R. Soc. B"},{"issue":"1","key":"B21","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1038\/4580","article-title":"Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects","volume":"2","year":"1999","journal-title":"Nat. Neurosci."},{"volume":"1","volume-title":"Elemente der Psychophysik","year":"1888","key":"B22"},{"volume-title":"Psychophysics: A Practical Introduction","year":"2016","key":"B23"},{"issue":"17","key":"B24","doi-asserted-by":"crossref","first-page":"2503","DOI":"10.1016\/0042-6989(95)00016-X","article-title":"Minireview: adaptive psychophysical procedures","volume":"35","year":"1995","journal-title":"Vis. Res."},{"issue":"10","key":"B25","doi-asserted-by":"crossref","first-page":"1154","DOI":"10.1016\/j.visres.2008.07.012","article-title":"Top-down and bottom-up mechanisms in biasing competition in the human brain","volume":"49","year":"2009","journal-title":"Vis. Res."},{"issue":"3","key":"B26","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1145\/566654.566630","article-title":"Physiological measures of presence in stressful virtual environments","volume":"21","year":"2002","journal-title":"ACM Trans. Graph."},{"key":"B27","first-page":"103","article-title":"Virtual reality for spatial navigation","volume-title":"Virtual Reality in Behavioral Neuroscience: New Insights and Methods","year":"2023"},{"issue":"13","key":"B28","doi-asserted-by":"crossref","first-page":"R545","DOI":"10.1016\/j.cub.2015.04.053","article-title":"Striking individual differences in color perception uncovered by `the dress' photograph","volume":"25","year":"2015","journal-title":"Curr. Biol."},{"key":"B29","first-page":"161","article-title":"Experimentelle Studien \u00fcber das Sehen von Bewegung [Experimental studies on the perception of motion]","volume":"61","year":"1912","journal-title":"Z. Psychol."},{"volume-title":"Curves and Surfaces in Geometric Modeling","year":"2000","key":"B30"},{"key":"B31","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1017\/S0140525X01000115","article-title":"A sensorimotor account of vision and visual consciousness","volume":"24","year":"2001","journal-title":"Behav. Brain Sci."},{"key":"B32","doi-asserted-by":"crossref","first-page":"1125","DOI":"10.3389\/fpsyg.2017.01125","article-title":"Model of illusions and virtual reality","volume":"8","year":"2017","journal-title":"Front. Psychol."},{"issue":"3","key":"B33","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1017\/S0140525X12000477","article-title":"Whatever next? Predictive brains, situated agents, and the future of cognitive science","volume":"36","year":"2013","journal-title":"Behav. Brain Sci."},{"issue":"1","key":"B34","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1111\/nyas.14321","article-title":"Evaluating the neurophysiological evidence for predictive processing as a model of perception","volume":"1464","year":"2020","journal-title":"Ann. N.Y. Acad. Sci."},{"issue":"4","key":"B35","doi-asserted-by":"crossref","first-page":"695","DOI":"10.1016\/j.neuron.2012.10.038","article-title":"Canonical microcircuits for predictive coding","volume":"76","year":"2012","journal-title":"Neuron"},{"issue":"12","key":"B36","doi-asserted-by":"crossref","first-page":"706","DOI":"10.1016\/j.tins.2013.09.004","article-title":"Reflections on agranular architecture: predictive coding in the motor cortex","volume":"36","year":"2013","journal-title":"Trends Neurosci."},{"issue":"10","key":"B37","first-page":"3839","article-title":"Immersion and coherence: research agenda and early results","volume":"27","year":"2020","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"issue":"4","key":"B38","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1162\/1054746041944849","article-title":"How colorful was your day? Why questionnaires cannot assess presence in virtual environments","volume":"13","year":"2004","journal-title":"Presence"},{"issue":"5","key":"B39","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1162\/105474600566989","article-title":"Using presence questionnaires in reality","volume":"9","year":"2000","journal-title":"Presence"},{"issue":"3","key":"B40","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1207\/s15327108ijap0303_3","article-title":"Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness","volume":"3","year":"1993","journal-title":"Int. J. Aviat. Psychol."},{"key":"B41","first-page":"531","article-title":"Motion sickness symptomatology and origins","volume-title":"Handbook of Virtual Environments","year":"2015"},{"issue":"19","key":"B42","doi-asserted-by":"crossref","first-page":"1783","DOI":"10.1080\/10447318.2020.1828535","article-title":"Identifying causes of and solutions for cybersickness in immersive technology: reformulation of a research and development agenda","volume":"36","year":"2020","journal-title":"Int. J. Hum.-Comput. Interact."},{"issue":"2","key":"B43","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1139\/y90-044","article-title":"Motion sickness: a synthesis and evaluation of the sensory conflict theory","volume":"68","year":"1990","journal-title":"Can. J. Physiol. Pharmacol."},{"issue":"3","key":"B44","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1167\/8.3.33","article-title":"Vergence\u2013accommodation conflicts hinder visual performance and cause visual fatigue","volume":"8","year":"2008","journal-title":"J. Vis."},{"key":"B45","first-page":"647","article-title":"Visually induced motion sickness: causes, characteristics, and countermeasures","volume-title":"Handbook of Virtual Environments","year":"2015"},{"issue":"10","key":"B46","doi-asserted-by":"crossref","first-page":"1096","DOI":"10.3390\/medicina57101096","article-title":"Critical flicker fusion frequency: a narrative review","volume":"57","year":"2021","journal-title":"Medicina"},{"issue":"4","key":"B47","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1093\/scan\/nsr025","article-title":"The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions","volume":"7","year":"2012","journal-title":"Soc. Cogn. Affect. Neurosci."},{"key":"B48","first-page":"217","article-title":"Gaze-directed volume rendering","volume-title":"I3D '90: Proceedings of the 1990 Symposium on Interactive 3D Graphics","year":"1990"},{"issue":"5","key":"B49","doi-asserted-by":"crossref","first-page":"2072","DOI":"10.1109\/TVCG.2019.2898741","article-title":"Temporal resolution multiplexing: exploiting the limitations of spatio-temporal vision for more efficient VR rendering","volume":"25","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B50","first-page":"7","article-title":"Post-rendering 3D warping","volume-title":"I3D '97: Proceedings of the 1997 Symposium on Interactive 3D graphics","year":"1997"},{"issue":"1","key":"B51","first-page":"17","article-title":"Estimation of detection thresholds for redirected walking techniques","volume":"16","year":"2009","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"issue":"4","key":"B52","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1162\/1054746041944803","article-title":"Transformed social interaction: decoupling representation from behavior and form in collaborative virtual environments","volume":"13","year":"2004","journal-title":"Presence"},{"key":"B53","first-page":"119","article-title":"What can fish brains tell us about visual perception?","volume":"8","year":"2014","journal-title":"Front. Neural Circuits"},{"issue":"1","key":"B54","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1093\/cz\/zow070","article-title":"Virtual reality systems for rodents","volume":"63","year":"2017","journal-title":"Curr. Zool."},{"issue":"22","key":"B55","doi-asserted-by":"crossref","first-page":"3523","DOI":"10.1016\/j.cub.2018.09.014","article-title":"Biological motion as an innate perceptual mechanism driving social affiliation","volume":"28","year":"2018","journal-title":"Curr. Biol."},{"issue":"10","key":"B56","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1038\/nmeth.4399","article-title":"Virtual reality for freely moving animals","volume":"14","year":"2017","journal-title":"Nat. Methods"},{"issue":"1","key":"B57","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1038\/nn.3884","article-title":"Impaired spatial selectivity and intact phase precession in two-dimensional virtual reality","volume":"18","year":"2015","journal-title":"Nat. Neurosci."},{"issue":"1","key":"B58","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/j.bbr.2003.10.026","article-title":"Navigation in virtual environment by the macaque monkey","volume":"153","year":"2004","journal-title":"Behav. Brain Res."},{"issue":"6","key":"B59","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1145\/129888.129892","article-title":"The CAVE: audio visual experience automatic virtual environment","volume":"35","year":"1992","journal-title":"Commun. ACM"},{"key":"B60","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.neuron.2019.04.034","article-title":"A cellular-resolution atlas of the larval zebrafish brain","volume":"103","year":"2019","journal-title":"Neuron"},{"key":"B61","first-page":"86","article-title":"Visually driven chaining of elementary swim patterns into a goal-directed motor sequence: a virtual reality study of zebrafish prey capture","volume":"7","year":"2013","journal-title":"Front. Neural Circuits"},{"issue":"9","key":"B62","doi-asserted-by":"crossref","first-page":"e24666","DOI":"10.1371\/journal.pone.0024666","article-title":"An image-free opto-mechanical system for creating virtual environments and imaging neuronal activity in freely moving Caenorhabditis elegans","volume":"6","year":"2011","journal-title":"PLOS ONE"},{"issue":"7763","key":"B63","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1038\/s41586-019-1352-7","article-title":"Whole-animal connectomes of both Caenorhabditis elegans sexes","volume":"571","year":"2019","journal-title":"Nature"},{"volume-title":"Control of a micro-organism as a prototype micro-robot","year":"1991","key":"B64"},{"key":"B65","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1111\/j.1475-4975.1987.tb00532.x","article-title":"Why paramecia don't have mental representations","volume":"10","year":"1986","journal-title":"Midwest Stud. Philos."},{"issue":"4","key":"B66","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1109\/56.800","article-title":"An exploration of sensorless manipulation","volume":"4","year":"1988","journal-title":"IEEE Trans. Robot. Autom."},{"issue":"4","key":"B67","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1109\/MM.2022.3161018","article-title":"ILLIXR: an open testbed to enable extended reality systems research","volume":"42","year":"2022","journal-title":"IEEE Micro"},{"volume-title":"Enactivist Interventions: Rethinking the Mind","year":"2017","key":"B68"},{"key":"B69","doi-asserted-by":"crossref","first-page":"643276","DOI":"10.3389\/fpsyg.2021.643276","article-title":"Cognition without neural representation: dynamics of a complex system","volume":"12","year":"2022","journal-title":"Front. Psychol."},{"volume-title":"Radicalizing Enactivism: Basic Minds Without Content","year":"2012","key":"B70"}],"container-title":["Annual Review of Control, Robotics, and Autonomous Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.annualreviews.org\/content\/journals\/10.1146\/annurev-control-062323-102456?crawler=true&mimetype=application\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,3]],"date-time":"2025-10-03T04:11:11Z","timestamp":1759464671000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.annualreviews.org\/content\/journals\/10.1146\/annurev-control-062323-102456"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,10]]},"references-count":70,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,7,10]]}},"URL":"https:\/\/doi.org\/10.1146\/annurev-control-062323-102456","relation":{},"ISSN":["2573-5144"],"issn-type":[{"type":"electronic","value":"2573-5144"}],"subject":[],"published":{"date-parts":[[2024,7,10]]}}}