{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:37:15Z","timestamp":1760243835259,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2011,7,1]],"date-time":"2011-07-01T00:00:00Z","timestamp":1309478400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This study presents efficient vision-based finger detection, tracking, and event identification techniques and a low-cost hardware framework for multi-touch sensing and display applications. The proposed approach uses a fast bright-blob segmentation process based on automatic multilevel histogram thresholding to extract the pixels of touch blobs obtained from scattered infrared lights captured by a video camera. The advantage of this automatic multilevel thresholding approach is its robustness and adaptability when dealing with various ambient lighting conditions and spurious infrared noises. To extract the connected components of these touch blobs, a connected-component analysis procedure is applied to the bright pixels acquired by the previous stage. After extracting the touch blobs from each of the captured image frames, a blob tracking and event recognition process analyzes the spatial and temporal information of these touch blobs from consecutive frames to determine the possible touch events and actions performed by users. This process also refines the detection results and corrects for errors and occlusions caused by noise and errors during the blob extraction process. The proposed blob tracking and touch event recognition process includes two phases. First, the phase of blob tracking associates the motion correspondence of blobs in succeeding frames by analyzing their spatial and temporal features. The touch event recognition process can identify meaningful touch events based on the motion information of touch blobs, such as finger moving, rotating, pressing, hovering, and clicking actions. Experimental results demonstrate that the proposed vision-based finger detection, tracking, and event identification system is feasible and effective for multi-touch sensing applications in various operational environments and conditions.<\/jats:p>","DOI":"10.3390\/s110706868","type":"journal-article","created":{"date-parts":[[2011,7,1]],"date-time":"2011-07-01T11:36:59Z","timestamp":1309520219000},"page":"6868-6892","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Vision-Based Finger Detection, Tracking, and Event Identification Techniques for Multi-Touch Sensing and Display Systems"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7717-9393","authenticated-orcid":false,"given":"Yen-Lin","family":"Chen","sequence":"first","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Taipei University of Technology, 1, Sec. 3, Chung-hsiao E. Rd., Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wen-Yew","family":"Liang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Taipei University of Technology, 1, Sec. 3, Chung-hsiao E. Rd., Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chuan-Yen","family":"Chiang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, National Chiao Tung University, 1001 University Road, Hsinchu 30050, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tung-Ju","family":"Hsieh","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Taipei University of Technology, 1, Sec. 3, Chung-hsiao E. Rd., Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Da-Cheng","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Taipei University of Technology, 1, Sec. 3, Chung-hsiao E. Rd., Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shyan-Ming","family":"Yuan","sequence":"additional","affiliation":[{"name":"Department of Computer Science, National Chiao Tung University, 1001 University Road, Hsinchu 30050, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yang-Lang","family":"Chang","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Taipei University of Technology, 1, Sec. 3, Chung-hsiao E. Rd., Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2011,7,1]]},"reference":[{"key":"ref_1","unstructured":"Buxton, B Available online: http:\/\/www.cs.berkeley.edu\/~tlavian\/spring2009\/Projects\/Multy%20touch%20systems.pdf (accessed on 30 March 2011) Microsoft: Toronto, ON, Canada, 2007."},{"key":"ref_2","unstructured":"Mehta, N (1982). A Flexible Machine Interface, M.A.Sc. Thesis, Department of Electrical Engineering, University of Toronto, Toronto, ON, Canada."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Nakatani, LH, and Rohrlich, JA (1983, January 12\u201315). Soft Machines: A Philosophy of User-Computer Interface Design. Boston, MA, USA.","DOI":"10.1145\/800045.801573"},{"key":"ref_4","unstructured":"Available online: http:\/\/www.apple.com\/iphone\/ (accessed on 15 April 2011)."},{"key":"ref_5","unstructured":"Available online: http:\/\/www.apple.com\/macbookair\/ (accessed on 15 April 2011)."},{"key":"ref_6","unstructured":"Microsoft surface. Available online: http:\/\/www.microsoft.com\/surface\/ (accessed on 15 April 2011)."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"846","DOI":"10.1109\/21.35348","article-title":"Model-based object recognition using a large-field passive tactile sensor","volume":"19","author":"Roach","year":"1989","journal-title":"IEEE Trans. Syst. Man Cybernet"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1109\/28.55954","article-title":"The electroquasistatics of the capacitive touch panel","volume":"26","author":"Krein","year":"1990","journal-title":"IEEE Trans. Ind. Appl"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1109\/JSEN.2007.891996","article-title":"An analytical solution to circular touch mode capacitor","volume":"7","author":"Maxime","year":"2007","journal-title":"IEEE Sens. J"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/JDT.2007.904368","article-title":"Remote-touch: A laser input user- display interaction technology","volume":"4","author":"Pasquariello","year":"2008","journal-title":"J. Disp. Technol"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hodges, S, Izadi, S, Butler, A, Rrustemi, A, and Buxton, B (2007, January 7\u201310). ThinSight: Versatile Multi-Touch Sensing for Thin Form-Factor Displays. Newport, RI, USA.","DOI":"10.1145\/1294211.1294258"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2416","DOI":"10.1109\/TCE.2009.5373818","article-title":"The multi-touch system with high applicability using tri-axial coordinate infrared LEDs","volume":"55","author":"Lee","year":"2009","journal-title":"IEEE Trans. Consumer Electron"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1448","DOI":"10.1109\/TCSVT.2005.857311","article-title":"Voting-based simultaneous tracking of multiple video objects","volume":"15","author":"Amer","year":"2005","journal-title":"IEEE Trans. Circuit Syst. Video Technol"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.cviu.2006.08.002","article-title":"A survey of advances in vision-based human motion capture and analysis","volume":"104","author":"Moeslund","year":"2006","journal-title":"Comput. Vis. Image Underst"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2252","DOI":"10.3390\/s90402252","article-title":"Visual sensor technology for advanced surveillance systems: historical view, technological aspects and research activities in Italy","volume":"9","author":"Foresti","year":"2009","journal-title":"Sensors"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"8865","DOI":"10.3390\/s101008865","article-title":"Stereo vision tracking of multiple objects in complex indoor environments","volume":"10","author":"Sotelo","year":"2010","journal-title":"Sensors"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1109\/TPAMI.2006.104","article-title":"On-road vehicle detection: A review","volume":"28","author":"Sun","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mac. Intell"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1722","DOI":"10.3390\/s90301722","article-title":"CMOS image sensor with a built-in lane detector","volume":"9","author":"Hsiao","year":"2009","journal-title":"Sensors"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3480","DOI":"10.3390\/s100403480","article-title":"Webcams for bird detection and monitoring: A demonstration study","volume":"10","author":"Verstraeten","year":"2010","journal-title":"Sensors"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2030","DOI":"10.1109\/TIE.2010.2055771","article-title":"A real-time vision system for nighttime vehicle detection and traffic surveillance","volume":"58","author":"Chen","year":"2011","journal-title":"IEEE Trans. Ind. Electron"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Letessier, J, and Berard, F (2004, January 24\u201327). Visual Tracking of Bare Fingers for Interactive Surfaces. Santa Fe, NM, USA.","DOI":"10.1145\/1029632.1029652"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Malik, S, and Laszlo, J (2004, January 13\u201315). Visual Touchpad: A Two-Handed Gestural Input Device. State College, PA, USA.","DOI":"10.1145\/1027933.1027980"},{"key":"ref_23","unstructured":"Xing, J, Wang, W, Zhao, W, and Huang, J (2009, January 15\u201316). A Novel Multi-Touch Human-Computer-Interface Based on Binocular Stereovision. Chengdu, China."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wilson, A (2005, January 23\u201326). PlayAnywhere: A Compact Interactive Tabletop Projection-Vision System. Seattle, WA, USA.","DOI":"10.1145\/1095034.1095047"},{"key":"ref_25","unstructured":"Chung, PK, Fang, B, and Quek, F (August, January 29). MirrorTrack\u2014A Vision Based Multi-Touch System for Glossy Display Surfaces. Xi\u2019an, China."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chung, PK, Fang, B, Ehrich, RW, and Quek, F (2008, January 15\u201317). MirrorTrack\u2014A Real-Time Multiple Camera Approach for Multi-Touch Interactions on Glossy Display Surfaces. Washington DC, USA.","DOI":"10.1109\/AIPR.2008.4906442"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Matsushita, N, and Rekimoto, J (1997, January 14\u201317). HoloWall: Designing a Finger, Hand, Body, and Object Sensitive Wall. Banff, AB, Canada.","DOI":"10.1145\/263407.263549"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Han, JY (2005, January 23\u201326). Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection. Seattle, WA, USA.","DOI":"10.1145\/1095034.1095054"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, F, Ren, X, and Liu, Z (2008, January 10\u201312). A Robust Blob Recognition and Tracking Method in Vision-Based Multi-Touch Technique. Sydney, Australia.","DOI":"10.1109\/ISPA.2008.129"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"de FO Ara\u00fajo, T, Lima, AMN, and dos Santos, AJV (2009, January 21\u201324). Detecting Hands, Fingers and Blobs for Multi-Touch Display Applications. Leipzig, Germany.","DOI":"10.1109\/HPCSIM.2009.5192305"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Otsu, N (1979). A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern, 62\u201366. SMC-9.","DOI":"10.1109\/TSMC.1979.4310076"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1109\/34.476511","article-title":"Goal-directed evaluation of binarization methods","volume":"17","author":"Trier","year":"1995","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.1109\/83.935031","article-title":"Stroke-model-based character extraction from gray-level document images","volume":"10","author":"Ye","year":"2001","journal-title":"IEEE Trans. Image Process"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1419","DOI":"10.1016\/j.patcog.2008.10.032","article-title":"A multi-plane segmentation approach for text extraction from complex document images","volume":"42","author":"Chen","year":"2009","journal-title":"Pattern Recog"},{"key":"ref_35","unstructured":"Rueda, L (2008). Structural, Syntactic, Statistical Pattern Recognition, Springer-Verlag. LNCS 5342, Volume 5342\/2008."},{"key":"ref_36","unstructured":"Rojas, D, Rueda, L, Urrutia, H, and Ngom, A (2009). Pattern Recognition in Bioinformatics, Springer-Verlag. LNBI 5780, Volume 5780\/2009."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Wu, BF, Chen, YL, and Chiu, CC (2005). A discriminant analysis based recursive automatic thresholding approach for image segmentation. IEICE Trans Info Syst, 1716\u20131723. E88-D.","DOI":"10.1093\/ietisy\/e88-d.7.1716"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S1077-3142(02)00030-9","article-title":"Linear-time connected-component labeling based on sequential local operations","volume":"89","author":"Suzuki","year":"2003","journal-title":"Comput. Vis. Image Underst"},{"key":"ref_39","unstructured":"Sneath, P, and Sokal, R (1973). Numerical Taxonomy The Principle and Practice of Numerical Classification, W.H. Freeman."},{"key":"ref_40","unstructured":"Available online: http:\/\/gpgpu.org\/ (accessed on 17 June 2011)."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Plaza, A, Du, Q, Chang, YL, and King, RL High performance computing for hyperspectral remote sensing. to be published, http:\/\/dx.doi.org\/10.1109\/JSTARS.2010.2095495.","DOI":"10.1109\/JSTARS.2010.2095495"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/11\/7\/6868\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:56:38Z","timestamp":1760219798000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/11\/7\/6868"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,7,1]]},"references-count":41,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2011,7]]}},"alternative-id":["s110706868"],"URL":"https:\/\/doi.org\/10.3390\/s110706868","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2011,7,1]]}}}