{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T11:31:30Z","timestamp":1784287890928,"version":"3.55.0"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,8,12]],"date-time":"2020-08-12T00:00:00Z","timestamp":1597190400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2020,8,31]]},"abstract":"<jats:p>Designing a camera motion controller that has the capacity to move a virtual camera automatically in relation with contents of a 3D animation, in a cinematographic and principled way, is a complex and challenging task. Many cinematographic rules exist, yet practice shows there are significant stylistic variations in how these can be applied.<\/jats:p>\n          <jats:p>In this paper, we propose an example-driven camera controller which can extract camera behaviors from an example film clip and re-apply the extracted behaviors to a 3D animation, through learning from a collection of camera motions. Our first technical contribution is the design of a low-dimensional cinematic feature space that captures the essence of a film's cinematic characteristics (camera angle and distance, screen composition and character configurations) and which is coupled with a neural network to automatically extract these cinematic characteristics from real film clips. Our second technical contribution is the design of a cascaded deep-learning architecture trained to (i) recognize a variety of camera motion behaviors from the extracted cinematic features, and (ii) predict the future motion of a virtual camera given a character 3D animation. We propose to rely on a Mixture of Experts (MoE) gating+prediction mechanism to ensure that distinct camera behaviors can be learned while ensuring generalization.<\/jats:p>\n          <jats:p>We demonstrate the features of our approach through experiments that highlight (i) the quality of our cinematic feature extractor (ii) the capacity to learn a range of behaviors through the gating mechanism, and (iii) the ability to generate a variety of camera motions by applying different behaviors extracted from film clips. Such an example-driven approach offers a high level of controllability which opens new possibilities toward a deeper understanding of cinematographic style and enhanced possibilities in exploiting real film data in virtual environments.<\/jats:p>","DOI":"10.1145\/3386569.3392427","type":"journal-article","created":{"date-parts":[[2020,8,12]],"date-time":"2020-08-12T11:44:27Z","timestamp":1597232667000},"update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":47,"title":["Example-driven virtual cinematography by learning camera behaviors"],"prefix":"10.1145","volume":"39","author":[{"given":"Hongda","family":"Jiang","sequence":"first","affiliation":[{"name":"Peking University &amp; AICFVE"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bin","family":"Wang","sequence":"additional","affiliation":[{"name":"Beijing Film Academy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xi","family":"Wang","sequence":"additional","affiliation":[{"name":"University Rennes"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc","family":"Christie","sequence":"additional","affiliation":[{"name":"University Rennes"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Baoquan","family":"Chen","sequence":"additional","affiliation":[{"name":"Peking University &amp; AICFVE"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,8,12]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Grammar of the film language","author":"Arijon Daniel","unstructured":"Daniel Arijon. 1991. Grammar of the film language. Silman-James Press."},{"key":"e_1_2_2_2_1","volume-title":"The virtual director: a correlation-based online viewing of human motion. Computer Graphics Forum 29, 2","author":"Assa Jackie","year":"2010","unstructured":"Jackie Assa, Lior Wolf, and Daniel Cohen-Or. 2010. The virtual director: a correlation-based online viewing of human motion. Computer Graphics Forum 29, 2 (2010)."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/354384.354463"},{"key":"e_1_2_2_4_1","volume-title":"Autonomous aerial cinematography in unstructured environments with learned artistic decision-making. Journal of Field Robotics","author":"Bonatti Rogerio","year":"2019","unstructured":"Rogerio Bonatti, Wenshan Wang, Cherie Ho, Aayush Ahuja, Mirko Gschwindt, Efe Camci, Erdal Kayacan, Sanjiban Choudhury, and Sebastian Scherer. 2019. Autonomous aerial cinematography in unstructured environments with learned artistic decision-making. Journal of Field Robotics (2019)."},{"key":"e_1_2_2_5_1","volume-title":"Multitask learning. Machine learning 28, 1","author":"Caruana Rich","year":"1997","unstructured":"Rich Caruana. 1997. Multitask learning. Machine learning 28, 1 (1997)."},{"key":"e_1_2_2_6_1","volume-title":"Camera control in computer graphics. Computer Graphics Forum 27, 8","author":"Christie Marc","year":"2008","unstructured":"Marc Christie, Patrick Olivier, and Jean-Marie Normand. 2008. Camera control in computer graphics. Computer Graphics Forum 27, 8 (2008)."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.343"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/147156.147166"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2822013.2822025"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2522628.2522899"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2668064.2668104"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v29i1.9288"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.169"},{"key":"e_1_2_2_14_1","volume-title":"Proc. IEEE Int'l Conf. Intelligent Robots and Systems.","author":"Gschwindt Mirko","unstructured":"Mirko Gschwindt, Efe Camci, Rogerio Bonatti, Wenshan Wang, Erdal Kayacan, and Sebastian A. Scherer. 2019. Can a Robot Become a Movie Director? Learning Artistic Principles for Aerial Cinematography. Proc. IEEE Int'l Conf. Intelligent Robots and Systems."},{"key":"e_1_2_2_15_1","volume-title":"A camera engine for computer games: Managing the trade-off between constraint satisfaction and frame coherence. Computer Graphics Forum 20, 3","author":"Halper Nicolas","year":"2001","unstructured":"Nicolas Halper, Ralf Helbing, and Thomas Strothotte. 2001. A camera engine for computer games: Managing the trade-off between constraint satisfaction and frame coherence. Computer Graphics Forum 20, 3 (2001)."},{"key":"e_1_2_2_16_1","unstructured":"Chong Huang Yuanjie Dang Peng Chen Xin Yang et al. 2019a. One-Shot Imitation Filming of Human Motion Videos. arXiv preprint arXiv:1912.10609 (2019)."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00437"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793915"},{"key":"e_1_2_2_19_1","volume-title":"Trip Synopsis: 60km in 60sec. Computer Graphics Forum 35, 7","author":"Huang Hui","year":"2016","unstructured":"Hui Huang, Dani Lischinski, Zhuming Hao, Minglun Gong, Marc Christie, and Daniel Cohen-Or. 2016. Trip Synopsis: 60km in 60sec. Computer Graphics Forum 35, 7 (2016)."},{"key":"e_1_2_2_20_1","volume-title":"Adaptive mixtures of local experts. Neural Comput. 3, 1","author":"Jacobs Robert","year":"1991","unstructured":"Robert Jacobs, Michael Jordan, Steven Nowlan, and Geoffrey Hinton. 1991. Adaptive mixtures of local experts. Neural Comput. 3, 1 (1991)."},{"key":"e_1_2_2_21_1","volume-title":"Rousseeuw","author":"Kaufman Leonard","year":"1990","unstructured":"Leonard Kaufman and Peter J. Rousseeuw. 1990. Finding Groups in Data An Introduction to Cluster Analysis. Wiley Interscience."},{"key":"e_1_2_2_22_1","volume-title":"Kingma and Jimmy Ba","author":"Diederick","year":"2015","unstructured":"Diederick P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Int'l Conf. Learning Representations."},{"key":"e_1_2_2_23_1","article-title":"Generating Realistic Camera Shake for Virtual Scenes","volume":"10","author":"Kurz Christian","year":"2014","unstructured":"Christian Kurz, Tobias Ritschel, Elmar Eisemann, Thorsten Thorm\u00e4hlen, and Hans-Peter Seidel. 2014. Generating Realistic Camera Shake for Virtual Scenes. JVRB-Journal of Virtual Reality and Broadcasting 10, 7 (2014).","journal-title":"JVRB-Journal of Virtual Reality and Broadcasting"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766965"},{"key":"e_1_2_2_25_1","volume-title":"Proc. ACM SIGGRAPH\/Eurographics Symp. on Computer Animation.","author":"Lino Christophe","year":"2010","unstructured":"Christophe Lino, Marc Christie, Fabrice Lamarche, Guy Schofield, and Patrick Olivier. 2010. A real-time cinematography system for interactive 3d environments. In Proc. ACM SIGGRAPH\/Eurographics Symp. on Computer Animation."},{"key":"e_1_2_2_26_1","volume-title":"The five C's of cinematography","author":"Mascelli Joseph V.","unstructured":"Joseph V. Mascelli. 1965. The five C's of cinematography. Grafic Publications."},{"key":"e_1_2_2_27_1","volume-title":"Few-shot Learning of Homogeneous Human Locomotion Styles. Computer Graphics Forum 37, 7","author":"Mason Ian","year":"2018","unstructured":"Ian Mason, Sebastian Starke, He Zhang, Hakan Bilen, and Taku Komura. 2018. Few-shot Learning of Homogeneous Human Locomotion Styles. Computer Graphics Forum 37, 7 (2018)."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1599470.1599478"},{"key":"e_1_2_2_29_1","volume-title":"Deep reinforcement learning for controlling frontal person close-up shooting. Neurocomputing 335","author":"Passalis Nikolaos","year":"2019","unstructured":"Nikolaos Passalis and Anastasios Tefas. 2019. Deep reinforcement learning for controlling frontal person close-up shooting. Neurocomputing 335 (2019)."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.2297932"},{"key":"e_1_2_2_31_1","unstructured":"Craig W Reynolds. 1999. Steering behaviors for autonomous characters. In Game developers conference."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2019.2892985"},{"key":"e_1_2_2_33_1","volume-title":"Proc. ACM SIGGRAPH\/Eurographics Symp. on Computer Animation.","author":"Sanokho Cunka Bassirou","year":"2014","unstructured":"Cunka Bassirou Sanokho, Clement Desoche, Billal Merabti, Tsai-Yen Li, and Marc Christie. 2014. Camera motion graphs. In Proc. ACM SIGGRAPH\/Eurographics Symp. on Computer Animation."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3340254"},{"key":"e_1_2_2_35_1","volume-title":"Visualizing Data using t-SNE. Journal of Machine Learning Research 9","author":"van der Maaten Laurens","year":"2008","unstructured":"Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research 9 (2008)."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201366"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3386569.3392427","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3386569.3392427","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T05:35:07Z","timestamp":1750829707000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3386569.3392427"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,12]]},"references-count":36,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,8,31]]}},"alternative-id":["10.1145\/3386569.3392427"],"URL":"https:\/\/doi.org\/10.1145\/3386569.3392427","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,12]]},"assertion":[{"value":"2020-08-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}