{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,20]],"date-time":"2026-09-20T22:46:28Z","timestamp":1789944388009,"version":"4.0.1"},"reference-count":48,"publisher":"Acoustical Society of America (ASA)","issue":"6","content-domain":{"domain":["pubs.aip.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2012,12,1]]},"abstract":"<jats:p>Speech can be represented as a constellation of constricting vocal tract actions called gestures, whose temporal patterning with respect to one another is expressed in a gestural score. Current speech datasets do not come with gestural annotation and no formal gestural annotation procedure exists at present. This paper describes an iterative analysis-by-synthesis landmark-based time-warping architecture to perform gestural annotation of natural speech. For a given utterance, the Haskins Laboratories Task Dynamics and Application (TADA) model is employed to generate a corresponding prototype gestural score. The gestural score is temporally optimized through an iterative timing-warping process such that the acoustic distance between the original and TADA-synthesized speech is minimized. This paper demonstrates that the proposed iterative approach is superior to conventional acoustically-referenced dynamic timing-warping procedures and provides reliable gestural annotation for speech datasets.<\/jats:p>","DOI":"10.1121\/1.4763545","type":"journal-article","created":{"date-parts":[[2012,12,7]],"date-time":"2012-12-07T12:56:08Z","timestamp":1354884968000},"page":"3980-3989","update-policy":"https:\/\/doi.org\/10.1063\/aip-crossmark-policy-page","source":"Crossref","is-referenced-by-count":23,"title":["A procedure for estimating gestural scores from speech acoustics"],"prefix":"10.1121","volume":"132","author":[{"given":"Hosung","family":"Nam","sequence":"first","affiliation":[{"name":"Haskins Laboratories, 300 George Street, Suite 900, New Haven, Connecticut 06511"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vikramjit","family":"Mitra","sequence":"additional","affiliation":[{"name":"Speech Technology and Research Laboratory, SRI International, 333 Ravenswood Avenue, Menlo Park, California 94025"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark","family":"Tiede","sequence":"additional","affiliation":[{"name":"Haskins Laboratories, 300 George Street, Suite 900, New Haven, Connecticut 06511"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark","family":"Hasegawa-Johnson","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering , Beckman Institute 2011, University of Illinois, Urbana, Illinois 61801"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carol","family":"Espy-Wilson","sequence":"additional","affiliation":[{"name":"Speech Communication Laboratory, Institute for Systems Research and Department of Electrical and Computer Engineering , A.V. Williams 2205, University of Maryland, College Park, Maryland 20742"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elliot","family":"Saltzman","sequence":"additional","affiliation":[{"name":"Department of Physical Therapy and Athletic Training , 635 Commonwealth Avenue, Boston University, Boston, Massachusetts 02215"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Louis","family":"Goldstein","sequence":"additional","affiliation":[{"name":"Department of Linguistics, University of Southern California , Grace Ford Salvatori 301, Los Angeles, California 90089"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"231","published-online":{"date-parts":[[2012,12,6]]},"reference":[{"key":"2023073123221738300_c1","first-page":"81","article-title":"Efficient coding of LPC parameters by temporal decomposition","volume-title":"Proceedings of ICASSP","year":"1983"},{"key":"2023073123221738300_c2","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1159\/000261913","article-title":"Articulatory phonology: An overview","volume":"49","year":"1992","journal-title":"Phonetica"},{"issue":"3","key":"2023073123221738300_c51","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1016\/S0095-4470(19)30376-6","article-title":"Gestural specification using dynamically-defined articulatory structures","volume":"18","year":"1990","journal-title":"J. Phonetics"},{"key":"2023073123221738300_c3","first-page":"19","article-title":"Gestural syllable position effects in American English","volume-title":"Producing Speech: Contemporary Issues (for Katherine Safford Harris)","author":"Bell-Berti","year":"1995"},{"key":"2023073123221738300_c4","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1159\/000262183","article-title":"C-centers revisited","volume":"52","year":"1995","journal-title":"Phonetica"},{"key":"2023073123221738300_c5","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1006\/jpho.1998.0071","article-title":"Intragestural dynamics of multiple phrasal boundaries","volume":"26","year":"1998","journal-title":"J. Phonetics"},{"issue":"2","key":"2023073123221738300_c6","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1016\/S0095-4470(02)00085-2","article-title":"The elastic phrase: Modeling the dynamics of boundary-adjacent lengthening","volume":"31","year":"2003","journal-title":"J. Phonetics"},{"key":"2023073123221738300_c7","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1016\/j.wocn.2008.10.002","article-title":"Timing effects of syllable structure and stress on nasals: A real-time MRI examination","volume":"37","year":"2009","journal-title":"J. Phonetics"},{"issue":"6","key":"2023073123221738300_c8","doi-asserted-by":"publisher","first-page":"3867","DOI":"10.1121\/1.1861893","article-title":"Prosodic strengthening and featural enhancement: Evidence from acoustic and articulatory realizations of \/\u0251,i\/ in English","volume":"117","year":"2005","journal-title":"J. Acoust. Soc. Am."},{"key":"2023073123221738300_c9","first-page":"519","article-title":"Manifestation of prosodic structure in articulatory variation: Evidence from lip movement kinematics in English","volume-title":"Laboratory Phonology 8 (Phonology and Phonetics)","author":"Goldstein","year":"2006"},{"issue":"1","key":"2023073123221738300_c11","doi-asserted-by":"publisher","first-page":"491","DOI":"10.1121\/1.412275","article-title":"The supraglottal articulation of prominence in English: Linguistic stress as localized hyperarticulation","volume":"97","year":"1995","journal-title":"J. Acoust. Soc. Am."},{"key":"2023073123221738300_c12","doi-asserted-by":"publisher","first-page":"3728","DOI":"10.1121\/1.418332","article-title":"Articulatory strengthening at edges of prosodic domains","volume":"101","year":"1997","journal-title":"J. Acoust. Soc. Am."},{"key":"2023073123221738300_c13","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/S0095-4470(19)31446-9","article-title":"Coarticulation and theories of extrinsic timing","volume":"8","year":"1980","journal-title":"J. Phonetics"},{"issue":"4","key":"2023073123221738300_c50","doi-asserted-by":"publisher","first-page":"1016","DOI":"10.1121\/1.393842","article-title":"On the role of speech transition for speech perception","volume":"80","year":"1986","journal-title":"J. Acoust. Soc. Am."},{"key":"2023073123221738300_c14","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1017\/CBO9780511541599","article-title":"The role of vocal tract gestural action units in understanding the evolution of phonology","volume-title":"From Action to Language: The Mirror Neuron System","author":"Arbib","year":"2006"},{"key":"2023073123221738300_c15","first-page":"112","article-title":"Representational bias in unsupervised learning of syllable structure","volume-title":"Proceedings of Computational Natural Language Learning","year":"2005"},{"issue":"3","key":"2023073123221738300_c16","doi-asserted-by":"publisher","first-page":"1158","DOI":"10.1121\/1.1498851","article-title":"A quasiarticulatory approach to controlling acoustic source parameters in a Klatt-type formant synthesizer using HLsyn","volume":"112","year":"2002","journal-title":"J. Acoust. Soc. Am."},{"key":"2023073123221738300_c17","first-page":"628","article-title":"Vowel landmark detection","volume-title":"Proceedings of Interspeech, International Speech Communication Association","year":"2000"},{"key":"2023073123221738300_c18","first-page":"2274","article-title":"FSM-based pronunciation modeling using articulatory phonological code","volume-title":"Proceedings of Interspeech","year":"2010"},{"issue":"2","key":"2023073123221738300_c19","doi-asserted-by":"publisher","first-page":"1154","DOI":"10.1121\/1.2823754","article-title":"Probabilistic landmark detection for automatic speech recognition using acoustic-phonetic information","volume":"123","year":"2008","journal-title":"J. Acoust. Soc. Am."},{"issue":"1","key":"2023073123221738300_c20","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1109\/TSA.1996.481448","article-title":"Deriving gestural scores from articulator-movement records using weighted temporal decomposition","volume":"4","year":"1996","journal-title":"IEEE Trans. Speech Audio Process."},{"key":"2023073123221738300_c21","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/S0095-4470(19)31319-1","article-title":"From general to language-specific capacities: The WRAPSA model of how speech perception develops","volume":"21","year":"1993","journal-title":"J. Phonetics"},{"key":"2023073123221738300_c22","first-page":"565","article-title":"Syllable position effects and gestural organization: Articulatory evidence from Russian","volume-title":"Papers in Laboratory Phonology 8","author":"Goldstein","year":"2006"},{"key":"2023073123221738300_c23","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1006\/jpho.1999.0089","article-title":"Physiological organization of syllables: A review","volume":"27","year":"1999","journal-title":"J. Phonetics"},{"key":"2023073123221738300_c25","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1016\/0010-0277(88)90035-2","article-title":"A precursor of language acquisition in young infants","volume":"29","year":"1988","journal-title":"Cognition"},{"key":"2023073123221738300_c26","doi-asserted-by":"publisher","first-page":"880","DOI":"10.1121\/1.380738","article-title":"Automatic segmentation of speech into syllabic units","volume":"58","year":"1975","journal-title":"J. Acoust. Soc. Am."},{"issue":"6","key":"2023073123221738300_c27","doi-asserted-by":"publisher","first-page":"1027","DOI":"10.1109\/JSTSP.2010.2076013","article-title":"Retrieving tract variables from acoustics: A comparison of different machine learning strategies","volume":"4","year":"2010","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"2023073123221738300_c28","first-page":"2038","article-title":"Robust word recognition using articulatory trajectories and gestures","volume-title":"Proceedings of Interspeech","year":"2010"},{"key":"2023073123221738300_c29","first-page":"5172","article-title":"Gesture-based dynamic Bayesian network for noise robust speech recognition","volume-title":"Proceedings of ICASSP","year":"2011"},{"issue":"3","key":"2023073123221738300_c30","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1016\/j.wocn.2012.02.002","article-title":"Temporal planning in speech: Syllable structure as coupling graph","volume":"40","year":"2012","journal-title":"J. Phonetics"},{"key":"2023073123221738300_c31","first-page":"483","article-title":"Syllable-level intergestural timing model: Split-gesture dynamics focusing on positional asymmetry and moraic structure","volume-title":"Laboratory Phonology 9 (Phonology and Phonetics)","author":"Cole","year":"2007"},{"issue":"5","key":"2023073123221738300_c32","first-page":"2430","article-title":"TADA: An enhanced, portable task dynamics model in Matlab","volume":"115","year":"2004","journal-title":"J. Acoust. Soc. Am."},{"key":"2023073123221738300_c33","first-page":"104","volume-title":"Physiology of Speech Production: Results and Implications of a Quantitative Cineradiographic Study","year":"1969"},{"key":"2023073123221738300_c34","first-page":"4190","article-title":"Reconstructing the full tongue contour from EMA\/X-Ray microbeam","volume-title":"Proceedings of ICASSP","year":"2010"},{"key":"2023073123221738300_c36","first-page":"125","article-title":"CASY and extensions to the task-dynamic model","volume-title":"Proceedings of the First ESCA ETRW on Speech Production Modeling\u20144th Speech Production Seminar: Models and Data","year":"1996"},{"issue":"1","key":"2023073123221738300_c37","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1109\/TASSP.1978.1163055","article-title":"Dynamic programming algorithm optimization for spoken word recognition","volume":"26","year":"1978","journal-title":"IEEE Trans. Acoust., Speech, Signal Process."},{"issue":"4","key":"2023073123221738300_c38","doi-asserted-by":"publisher","first-page":"332","DOI":"10.1207\/s15326969eco0104_2","article-title":"A dynamical approach to gestural patterning in speech production","volume":"1","year":"1989","journal-title":"Ecological Psychol."},{"key":"2023073123221738300_c39","first-page":"175","article-title":"A task-dynamic toolkit for modeling the effects of prosodic structure on articulation","volume-title":"Proceedings of Speech Prosody","year":"2008"},{"key":"2023073123221738300_c40","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1109\/JRPROC.1950.231821","article-title":"A simplified derivation of linear least square smoothing and prediction theory","volume":"38","year":"1950","journal-title":"Proc. IRE"},{"key":"2023073123221738300_c41","first-page":"243","article-title":"Evidence for the role of acoustic boundaries in the perception of speech sounds","volume-title":"Phonetic Linguistics: Essays in Honor of Peter Ladefoged","author":"Fromkin","year":"1985"},{"key":"2023073123221738300_c42","first-page":"499","article-title":"Implementation of a model for lexical access based on features","volume-title":"International Conference on Spoken Language Processing","year":"1992"},{"issue":"2","key":"2023073123221738300_c44","doi-asserted-by":"publisher","first-page":"1086","DOI":"10.1121\/1.1420380","article-title":"An overlapping-feature-based phonological model incorporating linguistic constraints: Applications to speech recognition","volume":"111","year":"2002","journal-title":"J. Acoust. Soc. Am."},{"key":"2023073123221738300_c43","first-page":"31","article-title":"Annotation and use of speech production corpus for building language universal speech recognizers","volume-title":"Proceedings of International Symposium on Chinese Spoken Language Processing","year":"2000"},{"key":"2023073123221738300_c45","first-page":"2771","article-title":"Automatically rating pronunciation through articulatory phonology","volume-title":"Proceedings of Interspeech","year":"2009"},{"key":"2023073123221738300_c46","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1006\/jpho.1996.0032","article-title":"The domain of accentual lengthening in American English","volume":"25","year":"1997","journal-title":"J. Phonetics"},{"key":"2023073123221738300_c47","first-page":"1","volume-title":"X-ray Microbeam Speech Production Database User's Handbook","year":"1994"},{"issue":"5","key":"2023073123221738300_c48","doi-asserted-by":"publisher","first-page":"3878","DOI":"10.1121\/1.2935783","article-title":"Speaker identification on the SCOTUS corpus","volume":"123","year":"2008","journal-title":"J. Acoust. Soc. Am."},{"key":"2023073123221738300_c49","first-page":"2763","article-title":"Articulatory phonological code for word classification","volume-title":"Proceedings of Interspeech","year":"2009"}],"container-title":["The Journal of the Acoustical Society of America"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.aip.org\/asa\/jasa\/article-pdf\/132\/6\/3980\/15304583\/3980_1_online.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/pubs.aip.org\/asa\/jasa\/article-pdf\/132\/6\/3980\/15304583\/3980_1_online.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,31]],"date-time":"2023-07-31T23:22:29Z","timestamp":1690845749000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.aip.org\/jasa\/article\/132\/6\/3980\/915695\/A-procedure-for-estimating-gestural-scores-from"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,12,1]]},"references-count":48,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2012,12,1]]}},"URL":"https:\/\/doi.org\/10.1121\/1.4763545","relation":{},"ISSN":["0001-4966","1520-8524"],"issn-type":[{"value":"0001-4966","type":"print"},{"value":"1520-8524","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2012,12]]},"published":{"date-parts":[[2012,12,1]]}}}