{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T18:39:13Z","timestamp":1770230353635,"version":"3.49.0"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2012,4,1]],"date-time":"2012-04-01T00:00:00Z","timestamp":1333238400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2012,4]]},"abstract":"<jats:p>A variety of dynamic objects, such as faces, bodies, and cloth, are represented in computer graphics as a collection of moving spatial landmarks. Spatiotemporal data is inherent in a number of graphics applications including animation, simulation, and object and camera tracking. The principal modes of variation in the spatial geometry of objects are typically modeled using dimensionality reduction techniques, while concurrently, trajectory representations like splines and autoregressive models are widely used to exploit the temporal regularity of deformation. In this article, we present the bilinear spatiotemporal basis as a model that simultaneously exploits spatial and temporal regularity while maintaining the ability to generalize well to new sequences. This factorization allows the use of analytical, predefined functions to represent temporal variation (e.g., B-Splines or the Discrete Cosine Transform) resulting in efficient model representation and estimation. The model can be interpreted as representing the data as a linear combination of spatiotemporal sequences consisting of shape modes oscillating over time at key frequencies. We apply the bilinear model to natural spatiotemporal phenomena, including face, body, and cloth motion data, and compare it in terms of compaction, generalization ability, predictive precision, and efficiency to existing models. We demonstrate the application of the model to a number of graphics tasks including labeling, gap-filling, denoising, and motion touch-up.<\/jats:p>","DOI":"10.1145\/2159516.2159523","type":"journal-article","created":{"date-parts":[[2012,5,1]],"date-time":"2012-05-01T13:43:38Z","timestamp":1335879818000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":93,"title":["Bilinear spatiotemporal basis models"],"prefix":"10.1145","volume":"31","author":[{"given":"Ijaz","family":"Akhter","sequence":"first","affiliation":[{"name":"LUMS School of Science and Engineering and Disney Research Pittsburgh, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tomas","family":"Simon","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University and Disney Research Pittsburgh, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sohaib","family":"Khan","sequence":"additional","affiliation":[{"name":"LUMS School of Science and Engineering, Lahore, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Iain","family":"Matthews","sequence":"additional","affiliation":[{"name":"Disney Research Pittsburgh and Carnegie Mellon University, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yaser","family":"Sheikh","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2012,4,30]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Akhter I. Sheikh Y. Khan S. and Kanade T. 2008. Nonrigid structure from motion in trajectory space. In Advances in Neural Information Processing Systems.  Akhter I. Sheikh Y. Khan S. and Kanade T. 2008. Nonrigid structure from motion in trajectory space. In Advances in Neural Information Processing Systems."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.201"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073207"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141971"},{"key":"e_1_2_2_5_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 690--696","author":"Bregler C."},{"key":"e_1_2_2_6_1","doi-asserted-by":"crossref","unstructured":"Bronstein A. Bronstein M. and Kimmel R. 2008. Numerical Geometry of Non-Rigid Shapes. Springer.   Bronstein A. Bronstein M. and Kimmel R. 2008. Numerical Geometry of Non-Rigid Shapes. Springer.","DOI":"10.1007\/978-0-387-73301-2"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073248"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1061347.1061355"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1006\/cviu.1995.1004"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778843"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360697"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/0166-2236(95)94496-R"},{"key":"e_1_2_2_13_1","volume-title":"A Practical Guide to Splines","author":"Deboor C."},{"key":"e_1_2_2_14_1","unstructured":"Dryden I. L. and Mardia K. V. 1998. Statistical Shape Analysis. Wiley.  Dryden I. L. and Mardia K. V. 1998. Statistical Shape Analysis. Wiley."},{"key":"e_1_2_2_15_1","doi-asserted-by":"crossref","unstructured":"Gabaix X. and Laibson D. 2008. The seven properties of good models. In The Foundations of Positive and Normative Economics: A Handbook A. Caplin and A. Schotter Eds. Oxford University Press.  Gabaix X. and Laibson D. 2008. The seven properties of good models. In The Foundations of Positive and Normative Economics: A Handbook A. Caplin and A. Schotter Eds. Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780195328318.003.0012"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/253284.253321"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280820"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1006\/gmod.2001.0549"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1099-1778(199804\/06)9:2<65::AID-VIS176>3.0.CO;2-Z"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2011.50"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2003.11.009"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-9457(01)00050-1"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-009-0212-6"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2005.67"},{"key":"e_1_2_2_25_1","volume-title":"Fundamentals of Digital Image Processing","author":"Jain A."},{"key":"e_1_2_2_26_1","unstructured":"Lawrence N. D. 2004. Gaussian process latent variable models for visualisation of high dimensional data. In Advances in Neural Information Processing Systems.  Lawrence N. D. 2004. Gaussian process latent variable models for visualisation of high dimensional data. In Advances in Neural Information Processing Systems."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176349259"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2010.41"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778769"},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Animation.","author":"Li L."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1557019.1557078"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00371-006-0080-9"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2010.23"},{"key":"e_1_2_2_34_1","unstructured":"Magnus J. R. and Neudecker H. 1999. Matrix Differential Calculus with Applications in Statistics and Econometrics 2nd Ed. John Wiley & Sons.  Magnus J. R. and Neudecker H. 1999. Matrix Differential Calculus with Applications in Statistics and Econometrics 2nd Ed. John Wiley & Sons."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.2307\/1427764"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1640443.1640452"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1730804.1730811"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2002.804425"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141970"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/11494621_41"},{"key":"e_1_2_2_41_1","doi-asserted-by":"crossref","unstructured":"Rao K. and Yip P. 1990. Discrete Cosine Transform: Algorithms Advantages Applications. Academic New York.   Rao K. and Yip P. 1990. Discrete Cosine Transform: Algorithms Advantages Applications. Academic New York.","DOI":"10.1016\/B978-0-08-092534-9.50007-2"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015754"},{"key":"e_1_2_2_43_1","volume-title":"Proceedings of the International Conference on Artificial Neural Networks. 583--588","author":"Sch\u00f6lkopf B."},{"key":"e_1_2_2_44_1","volume-title":"Proceedings of the European Conference on Computer Vision. 702--718","author":"Sidenbladh H."},{"key":"e_1_2_2_45_1","volume-title":"Proceedings of the European Conference on Computer Vision. 243--257","author":"Sigal L."},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276504"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300015349"},{"key":"e_1_2_2_48_1","unstructured":"Thrun S. Burgard W. and Fox D. 2006. Probabilistic Robotics. Cambridge University Press.  Thrun S. Burgard W. and Fox D. 2006. Probabilistic Robotics. Cambridge University Press."},{"key":"e_1_2_2_49_1","volume-title":"Proceedings of the European Conference on Computer Vision. 801--812","author":"Torresani L."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1167\/2.5.2"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2004.00809.x"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015725"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073209"},{"key":"e_1_2_2_54_1","volume-title":"Proceedings of the 5th Eurographics Symposium on Geometry Processing. 49--58","author":"Wand M."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073224"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.1167"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276420"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/54852.378507"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2159516.2159523","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2159516.2159523","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T10:59:58Z","timestamp":1750244398000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2159516.2159523"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,4]]},"references-count":58,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2012,4]]}},"alternative-id":["10.1145\/2159516.2159523"],"URL":"https:\/\/doi.org\/10.1145\/2159516.2159523","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,4]]},"assertion":[{"value":"2011-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-04-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}