{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:34:34Z","timestamp":1760146474953,"version":"build-2065373602"},"reference-count":49,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,11,8]],"date-time":"2024-11-08T00:00:00Z","timestamp":1731024000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council (ERC)","doi-asserted-by":"publisher","award":["816006","861166"],"award-info":[{"award-number":["816006","861166"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]},{"name":"European Union","award":["816006","861166"],"award-info":[{"award-number":["816006","861166"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Understanding document layouts is vital for enhancing document exploration and information retrieval for sighted individuals. However, for blind and visually impaired people, it becomes challenging to have access to layout information using typical assistive technologies such as screen readers. In this paper, we examine the potential benefits of presenting documents on two-dimensional (2D) refreshable tactile displays. These displays enable the tactile perception of 2D data, offering the advantage of dynamic and interactive functionality. Despite their potential, the development of user interfaces (UIs) for such displays has not advanced significantly. Thus, we propose a design of an intelligent tactile user interface (TUI), incorporating touch and audio feedback to represent documents in a tactile format. Our exploratory study for evaluating this approach revealed satisfaction from participants with the experience of directly viewing documents in their true form, rather than relying on screen-reading interpretations. Additionally, participants offered recommendations for incorporating additional features and refining the approach in future iterations. To facilitate further research and development, we have made our dataset and models publicly available.<\/jats:p>","DOI":"10.3390\/mti8110102","type":"journal-article","created":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T03:53:14Z","timestamp":1731383594000},"page":"102","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Designing a Tactile Document UI for 2D Refreshable Tactile Displays: Towards Accessible Document Layouts for Blind People"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3533-2473","authenticated-orcid":false,"given":"Sara","family":"Alzalabny","sequence":"first","affiliation":[{"name":"NeptunLab, University of Freiburg, 79110 Freiburg, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4227-8417","authenticated-orcid":false,"given":"Omar","family":"Moured","sequence":"additional","affiliation":[{"name":"ACCESS@KIT, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany"},{"name":"CVHCI@KIT, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany"}]},{"given":"Karin","family":"M\u00fcller","sequence":"additional","affiliation":[{"name":"ACCESS@KIT, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7346-5744","authenticated-orcid":false,"given":"Thorsten","family":"Schwarz","sequence":"additional","affiliation":[{"name":"ACCESS@KIT, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3955-0291","authenticated-orcid":false,"given":"Bastian","family":"Rapp","sequence":"additional","affiliation":[{"name":"Freiburg Center of Interactive Materials and Bioinspired Technologies, University of Freiburg, 79110 Freiburg, Germany"}]},{"given":"Rainer","family":"Stiefelhagen","sequence":"additional","affiliation":[{"name":"ACCESS@KIT, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany"},{"name":"CVHCI@KIT, Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3458024","article-title":"Accessible web development: Opportunities to improve the education and practice of web development with a screen reader","volume":"14","author":"Hurst","year":"2021","journal-title":"TACCESS"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Li, J., Kim, S., Miele, J.A., Agrawala, M., and Follmer, S. (2019, January 4\u20139). Editing spatial layouts through tactile templates for people with visual impairments. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK.","DOI":"10.1145\/3290605.3300436"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Potluri, V., Grindeland, T.E., Froehlich, J.E., and Mankoff, J. (2021, January 8\u201313). Examining visual semantic understanding in blind and low-vision technology users. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan.","DOI":"10.1145\/3411764.3445040"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1016\/S0306-4573(98)00061-2","article-title":"Document structure and digital libraries: How researchers mobilize information in journal articles","volume":"35","author":"Bishop","year":"1999","journal-title":"Inf. Process. Manag."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Dorigo, M., Harriehausen-M\u00fchlbauer, B., Stengel, I., and Dowland, P.S. (2011, January 9\u201314). Survey: Improving document accessibility from the blind and visually impaired user\u2019s point of view. Proceedings of the Universal Access in Human-Computer Interaction. Applications and Services: 6th International Conference, UAHCI 2011, Held as Part of HCI International 2011, Orlando, FL, USA. Proceedings, Part IV 6.","DOI":"10.1007\/978-3-642-21657-2_14"},{"key":"ref_6","unstructured":"Wang, L.L., Cachola, I., Bragg, J., Cheng, E.Y.Y., Haupt, C.H., Latzke, M., Kuehl, B., van Zuylen, M., Wagner, L.M., and Weld, D.S. (2021). Improving the Accessibility of Scientific Documents: Current State, User Needs, and a System Solution to Enhance Scientific PDF Accessibility for Blind and Low Vision Users. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Borges Oliveira, D.A., and Viana, M.P. (2017, January 22\u201329). Fast CNN-Based Document Layout Analysis. Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), Venice, Italy.","DOI":"10.1109\/ICCVW.2017.142"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Breuel, T.M. (2002). Two Geometric Algorithms for Layout Analysis. Document Analysis Systems V, Springer.","DOI":"10.1007\/3-540-45869-7_23"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Li, M., Xu, Y., Cui, L., Huang, S., Wei, F., Li, Z., and Zhou, M. (2020). DocBank: A Benchmark Dataset for Document Layout Analysis. arXiv.","DOI":"10.18653\/v1\/2020.coling-main.82"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wang, J., Krumdick, M., Tong, B., Halim, H., Sokolov, M., Barda, V., Vendryes, D., and Tanner, C. (2023, January 21\u201326). A graphical approach to document layout analysis. Proceedings of the International Conference on Document Analysis and Recognition, San Jos\u00e9, CA, USA.","DOI":"10.1007\/978-3-031-41734-4_4"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3355610","article-title":"Document layout analysis: A comprehensive survey","volume":"52","author":"Binmakhashen","year":"2019","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1007\/s10032-024-00461-2","article-title":"Datasets and annotations for layout analysis of scientific articles","volume":"27","author":"Gemelli","year":"2024","journal-title":"Int. J. Doc. Anal. Recognit. (IJDAR)"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Pontelli, E., Gillan, D., Xiong, W., Saad, E., Gupta, G., and Karshmer, A.I. (2002, January 8\u201310). Navigation of HTML Tables, Frames, and XML Fragments. Proceedings of the Fifth International ACM Conference on Assistive Technologies, Edinburgh, UK. Assets \u201902.","DOI":"10.1145\/638252.638256"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Leporini, B., and Buzzi, M. (2022, January 21\u201323). Visually-Impaired People Studying via eBook: Investigating Current Use and Potential for Improvement. Proceedings of the 2022 6th International Conference on Education and E-Learning (ICEEL), Yamanashi, Japan.","DOI":"10.1145\/3578837.3578879"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1007\/s10209-013-0325-0","article-title":"Accessible haptic user interface design approach for users with visual impairments","volume":"13","author":"Kim","year":"2014","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_16","unstructured":"Wang, A., Chen, H., Liu, L., Chen, K., Lin, Z., Han, J., and Ding, G. (2024). Yolov10: Real-time end-to-end object detection. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, J., Xu, Y., Lv, T., Cui, L., Zhang, C., and Wei, F. (2022, January 10\u201314). Dit: Self-supervised pre-training for document image transformer. Proceedings of the 30th ACM International Conference on Multimedia, Lisboa, Portugal.","DOI":"10.1145\/3503161.3547911"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chen, Y., Zhang, J., Peng, K., Zheng, J., Liu, R., Torr, P., and Stiefelhagen, R. (2024, January 16\u201322). RoDLA: Benchmarking the Robustness of Document Layout Analysis Models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.01473"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Huang, Y., Lv, T., Cui, L., Lu, Y., and Wei, F. (2022, January 10\u201314). Layoutlmv3: Pre-training for document ai with unified text and image masking. Proceedings of the 30th ACM International Conference on Multimedia, Lisboa, Portugal.","DOI":"10.1145\/3503161.3548112"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhong, X., Tang, J., and Jimeno Yepes, A. (2019, January 20\u201325). PubLayNet: Largest Dataset Ever for Document Layout Analysis. Proceedings of the 2019 International Conference on Document Analysis and Recognition (ICDAR), Sydney, NSW, Australia.","DOI":"10.1109\/ICDAR.2019.00166"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Pfitzmann, B., Auer, C., Dolfi, M., Nassar, A.S., and Staar, P. (2022, January 14\u201318). Doclaynet: A large human-annotated dataset for document-layout segmentation. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA.","DOI":"10.1145\/3534678.3539043"},{"key":"ref_22","first-page":"1","article-title":"Funsd: A dataset for form understanding in noisy scanned documents","volume":"Volume 2","author":"Jaume","year":"2019","journal-title":"Proceedings of the 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW)"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, Z., Xu, Y., Cui, L., Shang, J., and Wei, F. (2021). Layoutreader: Pre-training of text and layout for reading order detection. arXiv.","DOI":"10.18653\/v1\/2021.emnlp-main.389"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, L., Hu, A., Xu, H., Yan, M., Xu, Y., Jin, Q., Zhang, J., and Huang, F. (2024). Tinychart: Efficient chart understanding with visual token merging and program-of-thoughts learning. arXiv.","DOI":"10.18653\/v1\/2024.emnlp-main.112"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Moured, O., Baumgarten-Egemole, M., M\u00fcller, K., Roitberg, A., Schwarz, T., and Stiefelhagen, R. (2024, January 18\u201321). Chart4blind: An intelligent interface for chart accessibility conversion. Proceedings of the 29th International Conference on Intelligent User Interfaces, Greenville, SC, USA.","DOI":"10.1145\/3640543.3645175"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Sechayk, Y., Shamir, A., and Igarashi, T. (2024, January 11\u201316). SmartLearn: Visual-Temporal Accessibility for Slide-based e-learning Videos. Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3613905.3650883"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wang, L.L., Cachola, I., Bragg, J., Cheng, E.Y.Y., Haupt, C., Latzke, M., Kuehl, B., van Zuylen, M.N., Wagner, L., and Weld, D. (2021, January 18\u201322). Scia11y: Converting scientific papers to accessible html. Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility, Virtual Event.","DOI":"10.1145\/3441852.3476545"},{"key":"ref_28","unstructured":"Stockman, T., and Metatla, O. (, January January). The influence of screen-readers on web cognition. Proceedings of the Accessible Design in the Digital World Conference (ADDW 2008), York, UK."},{"key":"ref_29","first-page":"1","article-title":"A Comparative Evaluation of PDF-to-HTML Conversion Tools","volume":"Volume 6","author":"Pathirana","year":"2023","journal-title":"Proceedings of the 2023 International Research Conference on Smart Computing and Systems Engineering (SCSE)"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Morris, M.R., Johnson, J., Bennett, C.L., and Cutrell, E. (2018, January 21\u201326). Rich representations of visual content for screen reader users. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3173574.3173633"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wang, L.L., Bragg, J., and Weld, D.S. (2023). Paper to HTML: A Publicly Available Web Tool for Converting Scientific Pdfs into Accessible HTML. ACM SIGACCESS Access. Comput., 1\u201311.","DOI":"10.1145\/3582298.3582299"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Peng, Y.H., Chi, P., Kannan, A., Morris, M.R., and Essa, I. (2023, January 23\u201328). Slide Gestalt: Automatic Structure Extraction in Slide Decks for Non-Visual Access. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany.","DOI":"10.1145\/3544548.3580921"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Khurana, R., McIsaac, D., Lockerman, E., and Mankoff, J. (2018, January 21\u201326). Nonvisual interaction techniques at the keyboard surface. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3173574.3173585"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Gadde, P., and Bolchini, D. (2014, January 20\u201322). From screen reading to aural glancing: Towards instant access to key page sections. Proceedings of the 16th international ACM SIGACCESS Conference on Computers & Accessibility, Rochester, NY, USA.","DOI":"10.1145\/2661334.2661363"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Vtyurina, A., Fourney, A., Morris, M.R., Findlater, L., and White, R.W. (2019, January 28\u201330). Verse: Bridging screen readers and voice assistants for enhanced eyes-free web search. Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility, Pittsburgh, PA, USA.","DOI":"10.1145\/3308558.3314136"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ahmed, F., Borodin, Y., Soviak, A., Islam, M., Ramakrishnan, I., and Hedgpeth, T. (2012, January 7\u201310). Accessible skimming: Faster screen reading of web pages. Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, Cambridge, MA, USA.","DOI":"10.1145\/2380116.2380164"},{"key":"ref_37","unstructured":"Safi, W., Maurel, F., Routoure, J.M., Beust, P., and Dias, G. (2014, January 10\u201313). Blind browsing on hand-held devices: Touching the web... to understand it better. Proceedings of the Data Visualization Workshop (DataWiz 2014) associated to 25th ACM Conference on Hypertext and Social Media (HYPERTEXT 2014), Poznan, Poland."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1016\/j.procs.2012.10.036","article-title":"Haptic Perception of Document Structure for Visually Impaired People on Handled Devices","volume":"14","author":"Maurel","year":"2012","journal-title":"Procedia Comput. Sci."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Chase, E.D., Siu, A.F., Boadi-Agyemang, A., Kim, G.S., Gonzalez, E.J., and Follmer, S. (2020, January 26\u201328). PantoGuide: A Haptic and Audio Guidance System To Support Tactile Graphics Exploration. Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility, Virtual Event.","DOI":"10.1145\/3373625.3418023"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Ma\u0107kowski, M., and Brzoza, P. (2022). Accessible tutoring platform using audio-tactile graphics adapted for visually impaired people. Sensors, 22.","DOI":"10.3390\/s22228753"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Chang, R.C., Yong, S., Liao, F.Y., Tsao, C.A., and Chen, B.Y. (2023, January 23\u201328). Understanding (Non-) Visual Needs for the Design of Laser-Cut Models. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany.","DOI":"10.1145\/3544548.3580684"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Prescher, D., Weber, G., and Spindler, M. (2010, January 25\u201327). A Tactile Windowing System for Blind Users. Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility, New York, NY, USA. ASSETS \u201910.","DOI":"10.1145\/1878803.1878821"},{"key":"ref_43","unstructured":"metec, A. (2024, October 01). The \u201cLaptop for the Blind\u201d. Available online: https:\/\/metec-ag.de\/."},{"key":"ref_44","unstructured":"(2024, October 01). Artistic Document Layout Dataset (ArtDocLay). Available online: https:\/\/github.com\/moured\/layout-for-all."},{"key":"ref_45","unstructured":"Shneiderman, B. (1996, January 3\u20136). The eyes have it: A task by data type taxonomy for information visualizations. Proceedings of the 1996 IEEE Symposium on Visual Languages, Boulder, CO, USA."},{"key":"ref_46","unstructured":"Zhao, H., Plaisant, C., Shneiderman, B., and Duraiswami, R. (2004, January 6\u20139). Sonification of Geo-Referenced Data for Auditory Information Seeking: Design Principle and Pilot Study. Proceedings of the International Conference on Auditory Display, Sydney, NSW, Australia."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1145\/3610592","article-title":"What Are You Reading?","volume":"30","author":"Russell","year":"2023","journal-title":"Interactions"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1145\/3607867","article-title":"Shining a Light on the Dark Web","volume":"66","author":"Shein","year":"2023","journal-title":"Commun. ACM"},{"key":"ref_49","unstructured":"NV Access Limited (2024, October 01). NVDA Screen Reader 2024. Available online: https:\/\/www.nvaccess.org\/download\/."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/8\/11\/102\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:28:59Z","timestamp":1760113739000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/8\/11\/102"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,8]]},"references-count":49,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["mti8110102"],"URL":"https:\/\/doi.org\/10.3390\/mti8110102","relation":{},"ISSN":["2414-4088"],"issn-type":[{"type":"electronic","value":"2414-4088"}],"subject":[],"published":{"date-parts":[[2024,11,8]]}}}