{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:58:58Z","timestamp":1760241538858,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2018,5,11]],"date-time":"2018-05-11T00:00:00Z","timestamp":1525996800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper reports an efficient method for sound extraction from high-speed light spot videos reconstructed from the coded light spot images captured with a low-speed camera based on compressive sensing, but at the expense of consuming time. The proposed method first gets the high-speed video of the light spot that is illuminated on the vibrating target caused by sound. Then the centroid of the light spot is used to recover the sound. Simulations of the proposed method are carried out and experimental results are demonstrated. The results show that high-speed videos with a frame rate of 2000 Hz can be reconstructed with a low-speed (100 Hz) charge-coupled device (CCD) camera, which is randomly modulated by a digital micro-mirror device (DMD) 20 times during each exposure time. This means a speed improvement of 20 times is achieved. The effects of synchronization between CCD image recording and DMD modulation, the optimal sampling patterns of DMD, and sound vibration amplitudes on the performance of the proposed method are evaluated. Using this compressive camera, speech (counting from one to four in Chinese) was recovered well. This has been confirmed by directly listening to the recovered sound, and the intelligibility value (0\u20131) that evaluated the similarity between them was 0.8185. Although we use this compressive camera for sound detection, we expect it to be useful in applications related to vibration and motion.<\/jats:p>","DOI":"10.3390\/s18051524","type":"journal-article","created":{"date-parts":[[2018,5,14]],"date-time":"2018-05-14T02:57:20Z","timestamp":1526266640000},"page":"1524","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["A High-Speed Imaging Method Based on Compressive Sensing for Sound Extraction Using a Low-Speed Camera"],"prefix":"10.3390","volume":"18","author":[{"given":"Ge","family":"Zhu","sequence":"first","affiliation":[{"name":"Center for Quantum Technology Research, School of Physics, Beijing Institute of Technology, Beijing 100081, China"}]},{"given":"Xu-Ri","family":"Yao","sequence":"additional","affiliation":[{"name":"Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China"}]},{"given":"Zhi-Bin","family":"Sun","sequence":"additional","affiliation":[{"name":"Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"given":"Peng","family":"Qiu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"given":"Chao","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China"}]},{"given":"Guang-Jie","family":"Zhai","sequence":"additional","affiliation":[{"name":"Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"given":"Qing","family":"Zhao","sequence":"additional","affiliation":[{"name":"Center for Quantum Technology Research, School of Physics, Beijing Institute of Technology, Beijing 100081, China"}]}],"member":"1968","published-online":{"date-parts":[[2018,5,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zieger, C., Brutti, A., and Svaizer, P. (2009, January 2\u20134). Acoustic Based Surveillance System for Intrusion Detection. Proceedings of the 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, Genova, Italy.","DOI":"10.1109\/AVSS.2009.49"},{"key":"ref_2","unstructured":"Clavel, C., Ehrette, T., and Richard, G. (2005, January 6). Events Detection for an Audio-Based Surveillance System. Proceedings of the 2005 IEEE International Conference on Multimedia and Expo, Amsterdam, The Netherlands."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"091202","DOI":"10.3788\/COL201513.091202","article-title":"Amplitude Measurement of Weak Sinusoidal Water Surface Acoustic Wave Using Laser Interferometer","volume":"13","author":"Zhang","year":"2015","journal-title":"Chin. Opt. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1364\/AO.53.000971","article-title":"Analysis of Backscattering Characteristics of Objects for Remote Laser Voice Acquisition","volume":"53","author":"Li","year":"2014","journal-title":"Appl. Opt."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"732","DOI":"10.3788\/COL20090708.0732","article-title":"Laser Doppler Vibrometer for Real-Time Speech-Signal Acquirement","volume":"7","author":"Shang","year":"2009","journal-title":"Chin. Opt. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1415","DOI":"10.1109\/JSEN.2010.2093125","article-title":"Vibration Characteristics of Various Surfaces Using an LDV for Long-Range Voice Acquisition","volume":"11","author":"Li","year":"2011","journal-title":"IEEE Sens. J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1110","DOI":"10.1109\/TMECH.2010.2077678","article-title":"Vision-Aided Laser Doppler Vibrometry for Remote Automatic Voice Detection","volume":"16","author":"Qu","year":"2011","journal-title":"IEEE\/ASME Trans. Mech."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Qu, Y., Wang, T., and Zhu, Z.G. (2010, January 6\u20139). An Active Multimodal Sensing Platform for Remote Voice Detection. Proceedings of the 2010 IEEE\/ASME International Conference on Advanced Intelligent Mechatronics, Montreal, QC, Canada.","DOI":"10.1109\/AIM.2010.5695763"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1007\/s00340-011-4585-1","article-title":"Laser Vibrometry Based on Analysis of the Speckle Pattern from a Remote Object","volume":"105","author":"Veber","year":"2011","journal-title":"Appl. Phys. B"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3297","DOI":"10.1121\/1.4805440","article-title":"Extract Voice Information Using High-Speed Camera","volume":"133","author":"Akutsu","year":"2013","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1145\/2601097.2601119","article-title":"The Visual Microphone: Passive Recovery of Sound from Video","volume":"33","author":"Davis","year":"2014","journal-title":"ACM Trans. Graph."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"110502","DOI":"10.1117\/1.OE.53.11.110502","article-title":"Audio Extraction from Silent High-Speed Video Using an Optical Technique","volume":"53","author":"Wang","year":"2014","journal-title":"Opt. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"086111","DOI":"10.1063\/1.4961979","article-title":"Note: Sound Recovery from Video Using SVD-Based Information Extraction","volume":"87","author":"Zhang","year":"2016","journal-title":"Rev. Sci. Instrum."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"094105","DOI":"10.1117\/1.OE.56.9.094105","article-title":"Efficient Subtle Motion Detection from High-Speed Video for Sound Recovery and Vibration Analysis Using Singular Value Decomposition-Based Approach","volume":"56","author":"Zhang","year":"2017","journal-title":"Opt. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"21566","DOI":"10.1364\/OE.17.021566","article-title":"Simultaneous Remote Extraction of Multiple Speech Sources and Heart Beats from Secondary Speckles Pattern","volume":"17","author":"Zalevsky","year":"2009","journal-title":"Opt. Express"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1016\/j.optcom.2014.05.038","article-title":"Audio Signal Reconstruction Based on Adaptively Selected Seed Points from Laser Speckle Images","volume":"331","author":"Chen","year":"2014","journal-title":"Opt. Commun."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"026117","DOI":"10.1117\/1.OE.57.2.026117","article-title":"Sound Recovery via Intensity Variations of Speckle Pattern Pixels Selected with Variance-Based Method","volume":"57","author":"Zhu","year":"2018","journal-title":"Opt. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"430","DOI":"10.3390\/s90100430","article-title":"CMOS Image Sensors for High Speed Applications","volume":"9","author":"Deen","year":"2009","journal-title":"Sensors"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"5670","DOI":"10.1109\/TIP.2014.2368359","article-title":"Coded Acquisition of High Frame Rate Video","volume":"23","author":"Pournaghi","year":"2014","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1038\/nmeth.1429","article-title":"Temporal Pixel Multiplexing for Simultaneous High-Speed, High-Resolution Imaging","volume":"7","author":"Bub","year":"2010","journal-title":"Nat. Methods"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Feng, W., Zhang, F., Qu, X., and Zheng, S. (2016). Per-Pixel Coded Exposure for High-Speed and High-Resolution Imaging Using a Digital Micromirror Device Camera. Sensors, 16.","DOI":"10.3390\/s16030331"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1289","DOI":"10.1109\/TIT.2006.871582","article-title":"Compressed sensing","volume":"52","author":"Donoho","year":"2006","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/MSP.2007.914731","article-title":"An introduction to compressive sampling","volume":"25","author":"Candes","year":"2008","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_24","first-page":"14","article-title":"Imaging via compressive sampling [introduction to compressive sampling and recovery via convex programming]","volume":"25","author":"Rombers","year":"2008","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1109\/MSP.2007.4286571","article-title":"Compressive sensing [lecture notes]","volume":"24","author":"Baraniuk","year":"2007","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4961","DOI":"10.3390\/s130404961","article-title":"Compressive Sensing Image Sensors-Hardware Implementation","volume":"13","author":"Dadkhah","year":"2013","journal-title":"Sensors"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sankaranarayanan, A.C., Studer, C., and Baraniuk, R.G. (2012, January 28\u201329). CS-MUVI: Video Compressive Sensing for Spatial-Multiplexing Cameras. Proceedings of the IEEE International Conference on Computational Photography, Seattle, WA, USA.","DOI":"10.1109\/ICCPhot.2012.6215212"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Reddy, D., Veeraraghavan, A., and Chellappa, R. (2011, January 20\u201325). P2C2: Programmable Pixel Compressive Camera for High Speed Imaging. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995542"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T., and Nayar, S.K. (2011, January 6\u201313). Video from a Single Coded Exposure Photograph Using a Learned over-Complete Dictionary. Proceedings of the International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126254"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1109\/TPAMI.2013.129","article-title":"Efficient Space-Time Sampling with Pixel-Wise Coded Exposure for High-Speed Imaging","volume":"36","author":"Liu","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"10526","DOI":"10.1364\/OE.21.010526","article-title":"Coded Aperture Compressive Temporal Imaging","volume":"21","author":"Llull","year":"2013","journal-title":"Opt. Express"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"15992","DOI":"10.1364\/OE.23.015992","article-title":"High Spatio-Temporal Resolution Video with Compressed Sensing","volume":"23","author":"Koller","year":"2015","journal-title":"Opt. Express"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1109\/TPAMI.2010.87","article-title":"Coded Strobing Photography: Compressive Sensing of High Speed Periodic Videos","volume":"33","author":"Veeraraghavan","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Holloway, J., Sankaranarayanan, A.C., Veeraraghavan, A., and Tambe, S. (2012, January 28\u201329). Flutter Shutter Video Camera for Compressive Sensing of Videos. Proceedings of the IEEE International Conference on Computational Photography, Seattle, WA, USA.","DOI":"10.1109\/ICCPhot.2012.6215211"},{"key":"ref_35","unstructured":"Serrano, A., Gutierrez, D., and Masia, B. (2015, January 1\u20133). Compressive High Speed Video Acquisition. Proceedings of the Spanish Computer Graphics Conference (CEIG), Benic\u00e0ssim, Spain."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Yuan, X. (2016, January 25\u201328). Generalized Alternating Projection Based Total Variation Minimization for Compressive Sensing. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7532817"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2992","DOI":"10.1109\/TIP.2007.909319","article-title":"A New TwIST: Two-Step Iterative Shrinkage\/Thresholding Algorithms for Image Restoration","volume":"16","author":"Figueiredo","year":"2007","journal-title":"IEEE Trans. Image Process."},{"key":"ref_38","unstructured":"Boldt, J.B., and Ellis, D.P.W. (2009, January 24\u201328). A Simple Correlation-Based Model of Intelligibility for Nonlinear Speech Enhancement and Separation. Proceedings of the 2009 17th European Signal Processing Conference, Glasgow, UK."},{"key":"ref_39","unstructured":"(2005, August 30). 0.7 XGA 12\u00b0 DDR DMD Discovery. Available online: http:\/\/www.loreti.it\/Download\/PDF\/DMD\/7XGAdlp.pdf."},{"key":"ref_40","unstructured":"(2016, July 08). MantaG-145-30fps. Available online: https:\/\/www.alliedvision.com\/en\/products\/cameras\/detail\/Manta\/G-145-30fps\/action\/pdf.html."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Bloshkina, A.I., Li, L., Gubarev, F.A., and Klenovskii, M.S. (July, January 30). Investigation of Extracting Information from Vibrating Objects by Digital Speckle Correlation. Proceedings of the 2016 17th International Conference of Young Specialists on Micro\/Nanotechnologies and Electron Devices (EDM), Erlagol, Russia.","DOI":"10.1109\/EDM.2016.7538813"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Ri, S.E., Matsunaga, Y., Fujigaki, M., Matui, T., and Morimoto, Y. (2005). Development of DMD Reflection-Type CCD Camera for Phase Analysis and Shape Measurement. Optomech. Sens. Instrum., 6049.","DOI":"10.1117\/12.647938"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/5\/1524\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:03:56Z","timestamp":1760195036000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/5\/1524"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,5,11]]},"references-count":42,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2018,5]]}},"alternative-id":["s18051524"],"URL":"https:\/\/doi.org\/10.3390\/s18051524","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2018,5,11]]}}}