{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T22:31:48Z","timestamp":1776119508631,"version":"3.50.1"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2018,9,18]],"date-time":"2018-09-18T00:00:00Z","timestamp":1537228800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2018,9,18]]},"abstract":"<jats:p>People with Visual Impairments (PVI) experience greater difficulties with daily tasks, such as supermarket shopping. Identifying and purchasing an item proves challenging for PVI. Using a user-centered design process, we understand the difficulties PVI encounter in their daily routines. Consequently, the previous FingerReader model was elevated to a new level. In contrast, FingerReader2.0 incorporates a highly integrated hardware design, as it is standalone, wearable, and not tethered to a computer. Software-wise, the prototype utilizes a deep learning system, relying on a hybrid, an on-board and a cloud-based model. The advanced design significantly extends the range of mobile assistive technology, particularly for shopping purposes. This paper presents the findings from interviews, several iterative studies, and a field study in supermarkets to demonstrate the FingerReader2.0's enhanced capabilities for those with varied levels of visual impairment.<\/jats:p>","DOI":"10.1145\/3264904","type":"journal-article","created":{"date-parts":[[2018,9,19]],"date-time":"2018-09-19T11:58:41Z","timestamp":1537358321000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":39,"title":["FingerReader2.0"],"prefix":"10.1145","volume":"2","author":[{"given":"Roger","family":"Boldu","sequence":"first","affiliation":[{"name":"Augmented Human Lab, Auckland Bioengineering Institute, The University of Auckland, Auckland, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandru","family":"Dancu","sequence":"additional","affiliation":[{"name":"Augmented Human Lab, Singapore University of Technology and Design, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Denys J.C.","family":"Matthies","sequence":"additional","affiliation":[{"name":"Augmented Human Lab, Auckland Bioengineering Institute, The University of Auckland, Auckland, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thisum","family":"Buddhika","sequence":"additional","affiliation":[{"name":"Augmented Human Lab, Auckland Bioengineering Institute, The University of Auckland, Auckland, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shamane","family":"Siriwardhana","sequence":"additional","affiliation":[{"name":"Augmented Human Lab, Auckland Bioengineering Institute, The University of Auckland, Auckland, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Suranga","family":"Nanayakkara","sequence":"additional","affiliation":[{"name":"Augmented Human Lab, Auckland Bioengineering Institute, The University of Auckland, Auckland, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,9,18]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"BlindWays. http:\/\/www.perkins.org\/solutions\/featured-products\/blindways  BlindWays. http:\/\/www.perkins.org\/solutions\/featured-products\/blindways"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2481291"},{"key":"e_1_2_2_3_1","volume-title":"Challenges faced by older adults with vision loss: a qualitative study with implications for rehabilitation. Clinical rehabilitation 26, 8","author":"Cimarolli Verena R","year":"2012","unstructured":"Verena R Cimarolli , Kathrin Boerner , Mark Brennan-Ing , Joann P Reinhardt , and Amy Horowitz . 2012. Challenges faced by older adults with vision loss: a qualitative study with implications for rehabilitation. Clinical rehabilitation 26, 8 ( 2012 ), 748--757. Verena R Cimarolli, Kathrin Boerner, Mark Brennan-Ing, Joann P Reinhardt, and Amy Horowitz. 2012. Challenges faced by older adults with vision loss: a qualitative study with implications for rehabilitation. Clinical rehabilitation 26, 8 (2012), 748--757."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2682571.2797066"},{"key":"e_1_2_2_5_1","unstructured":"HIMS International | Blaze ET. http:\/\/himsintl.com\/product\/blaze-et\/  HIMS International | Blaze ET. http:\/\/himsintl.com\/product\/blaze-et\/"},{"key":"e_1_2_2_6_1","unstructured":"Eyra. 2014. Horus. https:\/\/horus.tech.  Eyra. 2014. Horus. https:\/\/horus.tech."},{"key":"e_1_2_2_7_1","unstructured":"KNFB Reader App features the best OCR. Turn print into speech or Braille instantly. iOS 3.0 now available. | KNFB Reader. https:\/\/knfbreader.com\/  KNFB Reader App features the best OCR. Turn print into speech or Braille instantly. iOS 3.0 now available. | KNFB Reader. https:\/\/knfbreader.com\/"},{"key":"e_1_2_2_8_1","unstructured":"Grace Sze-en Foo. 2009. Grocery Shopping Assistant for the Blind \/ Visually Impaired.. http:\/\/grozi.calit2.net\/files\/TIESGroZiSu09.pdf.  Grace Sze-en Foo. 2009. Grocery Shopping Assistant for the Blind \/ Visually Impaired.. http:\/\/grozi.calit2.net\/files\/TIESGroZiSu09.pdf."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/259963.260056"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2851581.2892371"},{"key":"e_1_2_2_11_1","volume-title":"Observations on active touch. Psychological review 69, 6","author":"Gibson James J","year":"1962","unstructured":"James J Gibson . 1962. Observations on active touch. Psychological review 69, 6 ( 1962 ), 477. James J Gibson. 1962. Observations on active touch. Psychological review 69, 6 (1962), 477."},{"key":"e_1_2_2_12_1","unstructured":"Google Brain. TensorFlow Release 1.2.1. https:\/\/goo.gl\/WZqjLs.  Google Brain. TensorFlow Release 1.2.1. https:\/\/goo.gl\/WZqjLs."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1622176.1622199"},{"key":"e_1_2_2_14_1","unstructured":"Step Hear. http:\/\/www.step-hear.com\/  Step Hear. http:\/\/www.step-hear.com\/"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2160125.2160137"},{"key":"e_1_2_2_16_1","volume-title":"CoRR abs\/1611.10012","author":"Huang Jonathan","year":"2016","unstructured":"Jonathan Huang , Vivek Rathod , Chen Sun , Menglong Zhu , Anoop Korattikara , Alireza Fathi , Ian Fischer , Zbigniew Wojna , Yang Song , Sergio Guadarrama , and Kevin Murphy . 2016. Speed\/accuracy trade-offs for modern convolutional object detectors. CoRR abs\/1611.10012 ( 2016 ). http:\/\/arxiv.org\/abs\/1611.10012 Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, and Kevin Murphy. 2016. Speed\/accuracy trade-offs for modern convolutional object detectors. CoRR abs\/1611.10012 (2016). http:\/\/arxiv.org\/abs\/1611.10012"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2541831.2541875"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02556-3_58"},{"key":"e_1_2_2_19_1","unstructured":"Ryo Kawamura. RectLabel -- Labeling images for object detection for MacOS. https:\/\/goo.gl\/GVqq9H.  Ryo Kawamura. RectLabel -- Labeling images for object detection for MacOS. https:\/\/goo.gl\/GVqq9H."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CISIS.2012.123"},{"key":"e_1_2_2_21_1","volume-title":"Proceedings of the 2010 Rehabilitation Engineering and Assistive Technology Society of North America Conference (RESNA","volume":"703","author":"Kulyukin Vladimir","year":"2010","unstructured":"Vladimir Kulyukin and Aliasgar Kutiyanawala . 2010 . From ShopTalk to ShopMobile: vision-based barcode scanning with mobile phones for independent blind grocery shopping . In Proceedings of the 2010 Rehabilitation Engineering and Assistive Technology Society of North America Conference (RESNA 2010), Las Vegas, NV , Vol. 703 . 1--5. Vladimir Kulyukin and Aliasgar Kutiyanawala. 2010. From ShopTalk to ShopMobile: vision-based barcode scanning with mobile phones for independent blind grocery shopping. In Proceedings of the 2010 Rehabilitation Engineering and Assistive Technology Society of North America Conference (RESNA 2010), Las Vegas, NV, Vol. 703. 1--5."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCA.2002.1229277"},{"key":"e_1_2_2_23_1","volume-title":"Trinetra: Assistive Technologies for Grocery Shopping for the Blind.. In ISWC. 147--148.","author":"Lanigan Patrick E","year":"2006","unstructured":"Patrick E Lanigan , Aaron M Paulos , Andrew W Williams , Dan Rossi , and Priya Narasimhan . 2006 . Trinetra: Assistive Technologies for Grocery Shopping for the Blind.. In ISWC. 147--148. Patrick E Lanigan, Aaron M Paulos, Andrew W Williams, Dan Rossi, and Priya Narasimhan. 2006. Trinetra: Assistive Technologies for Grocery Shopping for the Blind.. In ISWC. 147--148."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132525.3134817"},{"key":"e_1_2_2_25_1","volume-title":"Berg","author":"Liu Wei","year":"2015","unstructured":"Wei Liu , Dragomir Anguelov , Dumitru Erhan , Christian Szegedy , Scott E. Reed , Cheng-Yang Fu , and Alexander C . Berg . 2015 . SSD : Single Shot MultiBox Detector. CoRR abs\/1512.02325 (2015). http:\/\/arxiv.org\/abs\/1512.02325 Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott E. Reed, Cheng-Yang Fu, and Alexander C. Berg. 2015. SSD: Single Shot MultiBox Detector. CoRR abs\/1512.02325 (2015). http:\/\/arxiv.org\/abs\/1512.02325"},{"key":"e_1_2_2_26_1","volume-title":"Tactual perception. Handbook of perception and human performances 2","author":"Loomis Jack M","year":"1986","unstructured":"Jack M Loomis and Susan J Lederman . 1986. Tactual perception. Handbook of perception and human performances 2 ( 1986 ), 2. Jack M Loomis and Susan J Lederman. 1986. Tactual perception. Handbook of perception and human performances 2 (1986), 2."},{"key":"e_1_2_2_27_1","unstructured":"Aipoly Fully Autonomous Markets. https:\/\/www.aipoly.com\/  Aipoly Fully Autonomous Markets. https:\/\/www.aipoly.com\/"},{"key":"e_1_2_2_28_1","unstructured":"Microsoft. 2018. Seeing-AI. https:\/\/www.microsoft.com\/en-us\/seeing-ai\/.  Microsoft. 2018. Seeing-AI. https:\/\/www.microsoft.com\/en-us\/seeing-ai\/."},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.2174\/1874943700902010011"},{"key":"e_1_2_2_30_1","doi-asserted-by":"crossref","DOI":"10.1201\/b15703","volume-title":"User centred systems design","author":"Norman DA","year":"1986","unstructured":"DA Norman and SW Draper . 1986. User centred systems design . Hillsdale, NJ : LEA ( 1986 ). DA Norman and SW Draper. 1986. User centred systems design. Hillsdale, NJ: LEA (1986)."},{"key":"e_1_2_2_31_1","unstructured":"World Health Organization. Visual impairment and blindness howpublished = \"http:\/\/www.who.int\/mediacentre\/factsheets\/fs282\/en\/\" year=2017.  World Health Organization. Visual impairment and blindness howpublished = \"http:\/\/www.who.int\/mediacentre\/factsheets\/fs282\/en\/\" year=2017."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/SoCPaR.2009.117"},{"key":"e_1_2_2_33_1","volume-title":"Cats and Dogs. In IEEE Conference on Computer Vision and Pattern Recognition.","author":"Parkhi O. M.","unstructured":"O. M. Parkhi , A. Vedaldi , A. Zisserman , and C. V. Jawahar . 2012 . Cats and Dogs. In IEEE Conference on Computer Vision and Pattern Recognition. O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. V. Jawahar. 2012. Cats and Dogs. In IEEE Conference on Computer Vision and Pattern Recognition."},{"key":"e_1_2_2_34_1","unstructured":"K.R. Prince. 1996. Finger mounted computer input device. https:\/\/www.google.com\/patents\/US5581484 US Patent 5 581 484.  K.R. Prince. 1996. Finger mounted computer input device. https:\/\/www.google.com\/patents\/US5581484 US Patent 5 581 484."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-39351-8_6"},{"key":"e_1_2_2_36_1","first-page":"5","article-title":"Co-creation and the new landscapes of design","volume":"4","author":"Sanders Elizabeth B-N","year":"2008","unstructured":"Elizabeth B-N Sanders and Pieter Jan Stappers . 2008 . Co-creation and the new landscapes of design . Co-design 4 , 1 (2008), 5 -- 18 . Elizabeth B-N Sanders and Pieter Jan Stappers. 2008. Co-creation and the new landscapes of design. Co-design 4, 1 (2008), 5--18.","journal-title":"Co-design"},{"key":"e_1_2_2_37_1","volume-title":"Timothy Lillicrap. Meta-Learning with Memory-Augmented Neural Networks. In International Conference on Machine Learning (2016-06-11). 1842","author":"Santoro Adam","year":"1850","unstructured":"Adam Santoro , Sergey Bartunov , Matthew Botvinick , Daan Wierstra , and Timothy Lillicrap. Meta-Learning with Memory-Augmented Neural Networks. In International Conference on Machine Learning (2016-06-11). 1842 -- 1850 . http:\/\/proceedings.mlr.press\/v48\/santoro16.html Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-Learning with Memory-Augmented Neural Networks. In International Conference on Machine Learning (2016-06-11). 1842--1850. http:\/\/proceedings.mlr.press\/v48\/santoro16.html"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702421"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2828993"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132525.3134812"},{"key":"e_1_2_2_41_1","volume-title":"ECCV Workshops (3). 615--631","author":"Stearns Lee","year":"2014","unstructured":"Lee Stearns , Ruofei Du , Uran Oh , Yumeng Wang , Leah Findlater , Rama Chellappa , and Jon E Froehlich . 2014 . The Design and Preliminary Evaluation of a Finger-Mounted Camera and Feedback System to Enable Reading of Printed Text for the Blind .. In ECCV Workshops (3). 615--631 . Lee Stearns, Ruofei Du, Uran Oh, Yumeng Wang, Leah Findlater, Rama Chellappa, and Jon E Froehlich. 2014. The Design and Preliminary Evaluation of a Finger-Mounted Camera and Feedback System to Enable Reading of Printed Text for the Blind.. In ECCV Workshops (3). 615--631."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3161416"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2971648.2971723"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2957265.2961860"},{"key":"e_1_2_2_45_1","unstructured":"Tzutalin. 2015. LabelImg graphical image annotation tool on Windows and Linux. https:\/\/github.com\/tzutalin\/labelImg.  Tzutalin. 2015. LabelImg graphical image annotation tool on Windows and Linux. https:\/\/github.com\/tzutalin\/labelImg."},{"key":"e_1_2_2_46_1","unstructured":"Wayne Walls. 2017. Comparing image tagging services: Google Vision Microsoft Cognitive Services Amazon Rekognition and Clarifai. https:\/\/goo.gl\/TVdzUR.  Wayne Walls. 2017. Comparing image tagging services: Google Vision Microsoft Cognitive Services Amazon Rekognition and Clarifai. https:\/\/goo.gl\/TVdzUR."},{"key":"e_1_2_2_47_1","unstructured":"Help People who are Blind or Partially Sighted. https:\/\/www.orcam.com\/en\/  Help People who are Blind or Partially Sighted. https:\/\/www.orcam.com\/en\/"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380137"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240836"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2971648.2971730"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCE.2016.2614422"},{"key":"e_1_2_2_52_1","first-page":"12","article-title":"The functions of hand and eye movements in the process of perception","volume":"1","author":"Zinchenko VP","year":"1960","unstructured":"VP Zinchenko and BF Lomov . 1960 . The functions of hand and eye movements in the process of perception . Problems of Psychology 1 , 2 (1960), 12 -- 25 . VP Zinchenko and BF Lomov. 1960. The functions of hand and eye movements in the process of perception. Problems of Psychology 1, 2 (1960), 12--25.","journal-title":"Problems of Psychology"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3264904","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3264904","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:07:59Z","timestamp":1750212479000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3264904"}},"subtitle":["Designing and Evaluating a Wearable Finger-Worn Camera to Assist People with Visual Impairments while Shopping"],"short-title":[],"issued":{"date-parts":[[2018,9,18]]},"references-count":52,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2018,9,18]]}},"alternative-id":["10.1145\/3264904"],"URL":"https:\/\/doi.org\/10.1145\/3264904","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,9,18]]},"assertion":[{"value":"2018-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-09-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}