{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T10:32:12Z","timestamp":1778754732355,"version":"3.51.4"},"reference-count":54,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2018,8,28]],"date-time":"2018-08-28T00:00:00Z","timestamp":1535414400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001602","name":"Science Foundation Ireland","doi-asserted-by":"publisher","award":["13\/RC\/2077"],"award-info":[{"award-number":["13\/RC\/2077"]}],"id":[{"id":"10.13039\/501100001602","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["13\/RC\/2077"],"award-info":[{"award-number":["13\/RC\/2077"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The Lensless Smart Sensor (LSS) developed by Rambus, Inc. is a low-power, low-cost visual sensing technology that captures information-rich optical data in a tiny form factor using a novel approach to optical sensing. The spiral gratings of LSS diffractive grating, coupled with sophisticated computational algorithms, allow point tracking down to millimeter-level accuracy. This work is focused on developing novel algorithms for the detection of multiple points and thereby enabling hand tracking and gesture recognition using the LSS. The algorithms are formulated based on geometrical and mathematical constraints around the placement of infrared light-emitting diodes (LEDs) on the hand. The developed techniques dynamically adapt the recognition and orientation of the hand and associated gestures. A detailed accuracy analysis for both hand tracking and gesture classification as a function of LED positions is conducted to validate the performance of the system. Our results indicate that the technology is a promising approach, as the current state-of-the-art focuses on human motion tracking that requires highly complex and expensive systems. A wearable, low-power, low-cost system could make a significant impact in this field, as it does not require complex hardware or additional sensors on the tracked segments.<\/jats:p>","DOI":"10.3390\/s18092834","type":"journal-article","created":{"date-parts":[[2018,8,28]],"date-time":"2018-08-28T12:19:06Z","timestamp":1535458746000},"page":"2834","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Hand Tracking and Gesture Recognition Using Lensless Smart Sensors"],"prefix":"10.3390","volume":"18","author":[{"given":"Lizy","family":"Abraham","sequence":"first","affiliation":[{"name":"Micro and Nano Systems Centre of Tyndall National Institute, University College Cork, Cork T12 R5CP, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrea","family":"Urru","sequence":"additional","affiliation":[{"name":"Micro and Nano Systems Centre of Tyndall National Institute, University College Cork, Cork T12 R5CP, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Niccol\u00f2","family":"Normani","sequence":"additional","affiliation":[{"name":"Micro and Nano Systems Centre of Tyndall National Institute, University College Cork, Cork T12 R5CP, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2780-1110","authenticated-orcid":false,"given":"Mariusz P.","family":"Wilk","sequence":"additional","affiliation":[{"name":"Micro and Nano Systems Centre of Tyndall National Institute, University College Cork, Cork T12 R5CP, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Walsh","sequence":"additional","affiliation":[{"name":"Micro and Nano Systems Centre of Tyndall National Institute, University College Cork, Cork T12 R5CP, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Brendan","family":"O\u2019Flynn","sequence":"additional","affiliation":[{"name":"Micro and Nano Systems Centre of Tyndall National Institute, University College Cork, Cork T12 R5CP, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,8,28]]},"reference":[{"key":"ref_1","first-page":"22","article-title":"The Anatomy and Mechanics of the Human Hand","volume":"2","author":"Taylor","year":"1995","journal-title":"Artif. Limbs"},{"key":"ref_2","first-page":"1","article-title":"Vision Based Hand Gesture Recognition for Human Computer Interaction: A Survey","volume":"43","author":"Rautaray","year":"2012","journal-title":"Springer Trans. Artif. Intell. Rev."},{"key":"ref_3","first-page":"972","article-title":"Vision Based Hand Gesture Recognition","volume":"3","author":"Garg","year":"2009","journal-title":"World Acad. Sci. Eng. Technol."},{"key":"ref_4","unstructured":"Yang, D.D., Jin, L.W., and Yin, J.X. (2005, January 18\u201321). An effective robust fingertip detection method for finger writing character recognition system. Proceedings of the 4th International Conference on Machine Learning and Cybernetics, Guangzhou, China."},{"key":"ref_5","unstructured":"Oka, K., Sato, Y., and Koike, H. (2002, January 21). Real time Tracking of Multiple Fingertips and Gesture Recognition for Augmented Desk Interface Systems. Proceedings of the 5th IEEE International Conference on Automatic Face and Gesture Recognition (FGR.02), Washington, DC, USA."},{"key":"ref_6","unstructured":"Quek, F.K.H. (1995, January 26\u201328). Finger Mouse: A Free hand Pointing Computer Interface. Proceedings of the International Workshop on Automatic Face and Gesture Recognition, Zurich, Switzerland."},{"key":"ref_7","unstructured":"Crowley, J., Berard, F., and Coutaz, J. (1995, January 26\u201328). Finger Tacking as an Input Device for Augmented Reality. Proceedings of the International Workshop on Automatic Face and Gesture Recognition, Zurich, Switzerland."},{"key":"ref_8","unstructured":"Hung, Y.P., Yang, Y.S., Chen, Y.S., Hsieh, B., and Fuh, C.S. (1998, January 20). Free-Hand Pointer by Use of an Active Stereo Vision System. Proceedings of the 14th International Conference on Pattern Recognition (ICPR), Brisbane, Australia."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1372","DOI":"10.1109\/TPAMI.2006.189","article-title":"Model-based hand tracking using a hierarchical bayesian filter","volume":"28","author":"Stenger","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","first-page":"24","article-title":"A robust method for hand tracking using mean-shift algorithm and kalman filter in stereo color image sequences","volume":"6","author":"Elmezain","year":"2010","journal-title":"Int. J. Inf. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.cviu.2006.10.012","article-title":"Vision based hand pose estimation: A review","volume":"108","author":"Erol","year":"2007","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1109\/TSMCC.2008.923862","article-title":"A Survey of Glove-Based Systems and their Applications","volume":"38","author":"Dipietro","year":"2008","journal-title":"IEEE Trans. Syst. Man Cybern. Part C Appl. Rev."},{"key":"ref_13","unstructured":"Dorfm\u00fcller-Ulhaas, K., and Schmalstieg, D. (2001). Finger Tracking for Interaction in Augmented Environments, Institute of Computer Graphics and Algorithms, Vienna University of Technology. Technical Report TR186-2-01-03."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kim, J.H., Thang, N.D., and Kim, T.S. (2009, January 5\u20138). 3-D hand Motion Tracking and Gesture Recognition Using a Data Glove. Proceedings of the 2009 IEEE International Symposium on Industrial Electronics, Seoul, Korea.","DOI":"10.1109\/ISIE.2009.5221998"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1016\/j.amc.2008.05.099","article-title":"Online Classifier Construction Algorithm for Human Activity Detection using a Triaxial Accelerometer","volume":"205","author":"Chen","year":"2008","journal-title":"Appl. Math. Comput."},{"key":"ref_16","first-page":"23","article-title":"Novel Smart Glove Technology as a Biomechanical Monitoring Tool","volume":"193","author":"Sachez","year":"2015","journal-title":"Sens. Transducers J."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hsiao, P.C., Yang, S.Y., Lin, B.S., Lee, I.J., and Chou, W. (2015, January 25\u201329). Data glove embedded with 9-axis IMU and force sensing sensors for evaluation of hand function. Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.","DOI":"10.1109\/EMBC.2015.7319426"},{"key":"ref_18","unstructured":"(2017, November 06). Gloveone-Neurodigital. Available online: https:\/\/www.neurodigital.es\/gloveone\/."},{"key":"ref_19","unstructured":"Manus, V.R. (2017, November 06). The Pinnacle of Virtual Reality Controllers. Available online: https:\/\/manus-vr.com\/."},{"key":"ref_20","unstructured":"(2017, November 06). Hi5 VR Glove. Available online: https:\/\/hi5vrglove.com\/."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Tzemanaki, A., Burton, T.M., Gillatt, D., Melhuish, C., Persad, R., Pipe, A.G., and Dogramadzi, S. (2014, January 12\u201315). \u03bcAngelo: A Novel Minimally Invasive Surgical System based on an Anthropomorphic Design. Proceedings of the 5th IEEE RAS EMBS International Conference on Biomedical Robotics and Biomechatronics, Sao Paulo, Brazil.","DOI":"10.1109\/BIOROB.2014.6913804"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/38.250916","article-title":"A survey of glove-based input","volume":"14","author":"Sturman","year":"1994","journal-title":"IEEE Comput. Graph. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1145\/1531326.1531369","article-title":"Real-time hand-tracking with a color glove","volume":"28","author":"Wang","year":"2009","journal-title":"ACM Trans. Graph."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"6380","DOI":"10.3390\/s130506380","article-title":"Analysis of the accuracy and robustness of the leap motion controller","volume":"13","author":"Weichert","year":"2013","journal-title":"Sensors"},{"key":"ref_25","first-page":"105","article-title":"A survey on time-of-flight stereo fusion","volume":"Volume 8200","author":"Nair","year":"2013","journal-title":"Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, LNCS"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MMUL.2012.24","article-title":"Microsoft Kinect sensor and its effect","volume":"19","author":"Zhang","year":"2012","journal-title":"IEEE Multimed."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Suarez, J., and Murphy, R.R. (2012, January 9\u201313). Hand Gesture Recognition with Depth Images: A Review. Proceedings of the IEEE RO-MAN, Paris, France.","DOI":"10.1109\/ROMAN.2012.6343787"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1659","DOI":"10.1109\/TCSVT.2015.2469551","article-title":"Survey on 3D hand gesture recognition","volume":"26","author":"Cheng","year":"2016","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Tzionas, D., Srikantha, A., Aponte, P., and Gall, J. (2014). Capturing hand motion with an RGB-D sensor, fusing a generative model with salient points. GCPR 2014 36th German Conference on Pattern Recognition, Springer.","DOI":"10.1007\/978-3-319-11752-2_22"},{"key":"ref_30","unstructured":"Li, Y. (2012, January 22\u201324). Hand gesture recognition using Kinect. Proceedings of the 2012 IEEE International Conference on Computer Science and Automation Engineering, Beijing, China."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wang, R., Paris, S., and Popovi\u0107, J. (2011). 6D hands: Markerless hand-tracking for computer aided design. 24th Annual ACM Symposium on User Interface Software and Technology, ACM.","DOI":"10.1145\/2047196.2047269"},{"key":"ref_32","unstructured":"Sridhar, S., Bailly, G., Heydrich, E., Oulasvirta, A., and Theobalt, C. (2016). Full Hand: Markerless Skeleton-based Tracking for Free-Hand Interaction. Full Hand: Markerless Skeleton-based Tracking for Free-Hand Interaction, Max Planck Institute for Informatics. MPI-I-2016-4-002."},{"key":"ref_33","first-page":"640","article-title":"Motion capture of hands in action using discriminative salient points","volume":"Volume 7577","author":"Ballan","year":"2012","journal-title":"European Conference on Computer Vision"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"36","DOI":"10.5772\/50093","article-title":"Markerless Kinect-based hand tracking for robot teleoperation","volume":"9","author":"Du","year":"2012","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sridhar, S., Mueller, F., Oulasvirta, A., and Theobalt, C. (2015, January 7\u201312). Fast and robust hand tracking using detection-guided optimization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298941"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1145\/2980179.2980226","article-title":"Sphere-Meshes for Real-Time Hand Modeling and Tracking","volume":"35","author":"Tkach","year":"2016","journal-title":"ACM Trans. Graph."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"4405","DOI":"10.1007\/s11042-015-3177-1","article-title":"A survey of depth and inertial sensor fusion for human action recognition","volume":"76","author":"Chen","year":"2015","journal-title":"Multimed. Tools Appl."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1898","DOI":"10.1109\/JSEN.2014.2306094","article-title":"Fusion of inertial and depth sensor data for robust hand gesture recognition","volume":"14","author":"Liu","year":"2014","journal-title":"IEEE Sens. J."},{"key":"ref_39","unstructured":"(2017, November 06). Lensless Smart Sensors-Rambus. Available online: https:\/\/www.rambus.com\/emerging-solutions\/lensless-smart-sensors\/."},{"key":"ref_40","unstructured":"Stork, D., and Gill, P. (2013, January 3\u20135). Lensless Ultra-Miniature CMOS Computational Imagers and Sensors. Proceedings of the International Conference Sensor Technology, Wellington, New Zealand."},{"key":"ref_41","unstructured":"Abraham, L., Urru, A., Wilk, M.P., Tedesco, S., and O\u2019Flynn, B. (2017, November 13). 3D Ranging and Tracking Using Lensless Smart Sensors. Available online: https:\/\/www.researchgate.net\/publication\/321027909_Target_Tracking?showFulltext=1&linkId=5a097fa00f7e9b68229d000e."},{"key":"ref_42","unstructured":"Abraham, L., Urru, A., Wilk, M.P., Tedesco, S., Walsh, M., and O\u2019Flynn, B. (November, January 29). Point Tracking with lensless smart sensors. Proceedings of the IEEE Sensors, Glasgow, UK."},{"key":"ref_43","first-page":"201","article-title":"Optical, mathematical, and computational foundations of lensless ultra-miniature diffractive imagers and sensors","volume":"7","author":"Stork","year":"2014","journal-title":"Int. J. Adv. Syst. Meas."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Gill, P., and Vogelsang, T. (2016, January 15\u201317). Lensless smart sensors: Optical and thermal sensing for the Internet of Things. Proceedings of the 2016 IEEE Symposium on VLSI Circuits (VLSI-Circuits), Honolulu, HI, USA.","DOI":"10.1109\/VLSIC.2016.7573486"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Liebgott, H., and Cachard, C. (2012, January 2\u20135). Tracking micro tool in a dynamic 3D ultrasound situation using kalman filter and ransac algorithm. Proceedings of the 9th IEEE International Symposium on Biomedical Imaging (ISBI), Barcelona, Spain.","DOI":"10.1109\/ISBI.2012.6235745"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Hoehmann, L., and Kummert, A. (2007, January 27\u201329). Laser range finder based obstacle tracking by means of a two-dimensional Kalman filter. Proceedings of the 2007 International Workshop on Multidimensional (nD) Systems, Aveiro, Portugal.","DOI":"10.1109\/NDS.2007.4509534"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Domuta, I., and Palade, T.P. (2016, January 19\u201320). Adaptive Kalman Filter for Target Tracking in the UWB Networks. Proceedings of the 13th Workshop on Positioning, Navigation and Communications, Bremen, Germany.","DOI":"10.1109\/WPNC.2016.7822855"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Wilk, M.P., Urru, A., Tedesco, S., and O\u2019Flynn, B. (2017, January 20\u201321). Sub-pixel point detection algorithm for point tracking with low-power wearable camera systems. Proceedings of the Irish Signals and Systems Conference, ISSC, Killarney, Ireland.","DOI":"10.1109\/ISSC.2017.7983629"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1007\/s11784-007-0042-5","article-title":"Euler\u2019s fixed point theorem: The axis of a rotation","volume":"2","author":"Palais","year":"2007","journal-title":"J. Fixed Point Theory Appl."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Belgioioso, G., Cenedese, A., Cirillo, G.I., Fraccaroli, F., and Susto, G.A. (2014, January 15\u201317). A machine learning based approach for gesture recognition from inertial measurements. Proceedings of the IEEE Conference on Decision Control, Los Angeles, CA, USA.","DOI":"10.1109\/CDC.2014.7040154"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Normani, N., Urru, A., Abraham, L., Walsh, M., Tedesco, S., Cenedese, A., Susto, G.A., and O\u2019Flynn, B. (2018, January 4\u20137). A Machine Learning Approach for Gesture Recognition with a Lensless Smart Sensor System. Proceedings of the IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Las Vegas, NV, USA.","DOI":"10.1109\/BSN.2018.8329677"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Tian, S., Zhang, X., Tian, J., and Sun, Q. (2016). Random Forest Classification of Wetland Landcovers from Multi-Sensor Data in the Arid Region of Xinjiang, China. Remote Sens., 8.","DOI":"10.3390\/rs8110954"},{"key":"ref_53","unstructured":"(2017, November 06). Hand Tracking Using Rambus LSS. Available online: https:\/\/vimeo.com\/240882492\/."},{"key":"ref_54","unstructured":"(2017, November 02). Gesture Recognition Using Rambus LSS. Available online: https:\/\/vimeo.com\/240882649\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/2834\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:21:32Z","timestamp":1760196092000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/2834"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,28]]},"references-count":54,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["s18092834"],"URL":"https:\/\/doi.org\/10.3390\/s18092834","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,8,28]]}}}