{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T05:11:08Z","timestamp":1755839468386,"version":"3.41.0"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,8,21]],"date-time":"2024-08-21T00:00:00Z","timestamp":1724198400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Web"],"published-print":{"date-parts":[[2024,8,31]]},"abstract":"<jats:p>In this article, we present a novel approach to quantitative evaluation of a model for parsing web pages as visual images, intended to provide improvements for users with assistive needs (cognitive or visual deficits, enabling decluttering or zooming and supporting more effective screen reader output). This segmentation-classification pipeline is tested in stages: We first discuss the validation of the segmentation algorithm, showing that our approach produces automated segmentations that are very similar to those produced by real users when making use of a drawing interface to designate edges and regions. We also examine the properties of these ground truth segmentations produced under different conditions. We then describe our Hidden Markov tree approach for classification and present results which serve provide important validation for this model. The analysis is set against effective choices for dataset and pruning options, measured with respect to manual ground truth labelling of regions. In all, we offer a detailed quantitative validation (focused on complex news pages) of a fully pipelined approach for interpreting web pages as visual images, an approach which enables important advances for users with assistive needs.<\/jats:p>","DOI":"10.1145\/3580519","type":"journal-article","created":{"date-parts":[[2023,1,21]],"date-time":"2023-01-21T11:46:32Z","timestamp":1674301592000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Validation of an Improved Vision-Based Web Page Parsing Pipeline"],"prefix":"10.1145","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3851-1346","authenticated-orcid":false,"given":"Michael","family":"Cormier","sequence":"first","affiliation":[{"name":"Mount Allison University, Sackville, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5958-3955","authenticated-orcid":false,"given":"Robin","family":"Cohen","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, Canada"},{"name":"University of Waterloo, Waterloo, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7172-2180","authenticated-orcid":false,"given":"Richard","family":"Mann","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7081-0709","authenticated-orcid":false,"given":"Karyn","family":"Moffatt","sequence":"additional","affiliation":[{"name":"McGill School of Information Studies, Montreal, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7620-0541","authenticated-orcid":false,"given":"Daniel","family":"Vogel","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, Canada"},{"name":"University of Waterloo, Waterloo, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5879-0867","authenticated-orcid":false,"given":"Mengfei","family":"Liu","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, Canada"},{"name":"University of Waterloo, Waterloo, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6291-5578","authenticated-orcid":false,"given":"Shangshang","family":"Zheng","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,8,21]]},"reference":[{"volume-title":"Research-Based Web Design & Usability Guidelines","year":"2006","unstructured":"2006. Research-Based Web Design & Usability Guidelines. U.S. Dept. of Health and Human Services.","key":"e_1_3_3_2_2"},{"doi-asserted-by":"publisher","key":"e_1_3_3_3_2","DOI":"10.1145\/1385569.1385576"},{"key":"e_1_3_3_4_2","doi-asserted-by":"crossref","first-page":"238","DOI":"10.1007\/978-3-319-04244-2_22","volume-title":"Lecture Notes in Computer Science: Current Trends in Web Engineering","author":"Akpinar M. Elgin","year":"2013","unstructured":"M. Elgin Akpinar and Yeliz Ye\u015filada. 2013. Vision based page segmentation algorithm: Extended and perceived success. In Lecture Notes in Computer Science: Current Trends in Web Engineering. Springer International Publishing, 238\u2013252."},{"issue":"6","key":"e_1_3_3_5_2","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1093\/iwc\/iwx015","article-title":"Discovering visual elements of web pages and their roles: Users\u2019 perception","volume":"29","author":"Akpinar M. Elgin","year":"2017","unstructured":"M. Elgin Akpinar and Yeliz Ye\u015filada. 2017. Discovering visual elements of web pages and their roles: Users\u2019 perception. Interacting with Computers 29, 6 (2017), 845\u2013867.","journal-title":"Interacting with Computers"},{"key":"e_1_3_3_6_2","first-page":"172","volume-title":"ASSETS 2000","author":"Asakawa C.","year":"2000","unstructured":"C. Asakawa and H. Takagi. 2000. Annotation-based transcoding for nonvisual web access. In ASSETS 2000. ACM Press, 172\u2013179."},{"key":"e_1_3_3_7_2","volume-title":"Semantic Automation from Screen Capture","author":"Barkol Omer","year":"2009","unstructured":"Omer Barkol, Ruth Bergman, Ayelet Pnueli, and Sagi Schein. 2009. Semantic Automation from Screen Capture. Technical Report HPL-2009-161. HP Labs."},{"key":"e_1_3_3_8_2","volume-title":"XHTML Role Attribute Module","author":"Birbeck Mark","year":"2010","unstructured":"Mark Birbeck, Richard Schwerdtfeger, T. V. Raman, Steven Pemberton, and Shane McCarron. 2010. XHTML Role Attribute Module. W3C Note. W3C. http:\/\/www.w3.org\/TR\/2010\/NOTE-xhtml-role-20101216\/."},{"doi-asserted-by":"publisher","key":"e_1_3_3_9_2","DOI":"10.1145\/1243441.1243444"},{"key":"e_1_3_3_10_2","volume-title":"VIPS: A Vision-based Page Segmentation Algorithm","author":"Cai Deng","year":"2003","unstructured":"Deng Cai, Shipeng Yu, Ji-Rong Wen, and Wei-Ying Ma. 2003. VIPS: A Vision-based Page Segmentation Algorithm. Technical Report MSR-TR-2003-79. Microsoft."},{"doi-asserted-by":"publisher","key":"e_1_3_3_11_2","DOI":"10.1080\/17445760802429585"},{"doi-asserted-by":"publisher","key":"e_1_3_3_12_2","DOI":"10.1109\/ICMLA.2006.21"},{"key":"e_1_3_3_13_2","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1109\/ICDAR.2017.161","volume-title":"2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR)","volume":"1","author":"Chen Kai","year":"2017","unstructured":"Kai Chen, Mathias Seuret, Jean Hennebert, and Rolf Ingold. 2017. Convolutional neural networks for page segmentation of historical document images. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Vol. 1. IEEE, 965\u2013970."},{"doi-asserted-by":"publisher","key":"e_1_3_3_14_2","DOI":"10.1177\/001316446002000104"},{"key":"e_1_3_3_15_2","volume-title":"2016 IEEE\/WIC\/ACM International Conference on Web Intelligence","author":"Cormier Michael","year":"2016","unstructured":"Michael Cormier, Richard Mann, Robin Cohen, and Karyn Moffatt. 2016. Classification via hidden Markov trees for a vision-based approach to conveying webpages to users with assistive needs. In 2016 IEEE\/WIC\/ACM International Conference on Web Intelligence."},{"key":"e_1_3_3_16_2","volume-title":"2017 Conference on Computer and Robot Vision (CRV)","author":"Cormier Michael","year":"2017","unstructured":"Michael Cormier, Richard Mann, Karyn Moffatt, and Robin Cohen. 2017. Towards an improved vision-based web page segmentation algorithm. In 2017 Conference on Computer and Robot Vision (CRV)."},{"key":"e_1_3_3_17_2","article-title":"Purely vision-based segmentation of web pages for assistive technology","author":"Cormier Michael","year":"2016","unstructured":"Michael Cormier, Karyn Moffatt, Robin Cohen, and Richard Mann. 2016. Purely vision-based segmentation of web pages for assistive technology. CVIU Special Issue on Assistive Computer Vision and Robotics (2016).","journal-title":"CVIU Special Issue on Assistive Computer Vision and Robotics"},{"key":"e_1_3_3_18_2","first-page":"1","volume-title":"ACM Symposium on Document Engineering 2020","author":"Matteo Nicola R. Di","year":"2020","unstructured":"Nicola R. Di Matteo and James Blustein. 2020. A framework to evaluate webpage segment recognizers. In ACM Symposium on Document Engineering 2020. 1\u20134."},{"doi-asserted-by":"publisher","key":"e_1_3_3_19_2","DOI":"10.1145\/3192714.3196827"},{"doi-asserted-by":"publisher","key":"e_1_3_3_20_2","DOI":"10.1145\/3372497"},{"key":"e_1_3_3_21_2","first-page":"254","volume-title":"2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR)","volume":"1","author":"He Dafang","year":"2017","unstructured":"Dafang He, Scott Cohen, Brian Price, Daniel Kifer, and C. Lee Giles. 2017. Multi-scale multi-task fcn for semantic page segmentation and table detection. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Vol. 1. IEEE, 254\u2013261."},{"key":"e_1_3_3_22_2","first-page":"3047","volume-title":"29th ACM International Conference on Information & Knowledge Management","author":"Kiesel Johannes","year":"2020","unstructured":"Johannes Kiesel, Florian Kneist, Lars Meyer, Kristof Komlossy, Benno Stein, and Martin Potthast. 2020. Web page segmentation revisited: Evaluation framework and dataset. In 29th ACM International Conference on Information & Knowledge Management. 3047\u20133054."},{"doi-asserted-by":"publisher","key":"e_1_3_3_23_2","DOI":"10.1007\/978-3-030-72240-1_5"},{"key":"e_1_3_3_24_2","volume-title":"Web Content Accessibility Guidelines (WCAG) 2.1","author":"Kirkpatrick Andrew","year":"2018","unstructured":"Andrew Kirkpatrick, Joshue O. Connor, Michael Cooper, and Alastair Campbell. 2018. Web Content Accessibility Guidelines (WCAG) 2.1. W3C Recommendation. W3C. https:\/\/www.w3.org\/TR\/2018\/REC-WCAG21-20180605\/."},{"key":"e_1_3_3_25_2","volume-title":"Handbook of Document Image Processing and Recognition","author":"Kise Koichi","year":"2014","unstructured":"Koichi Kise. 2014. Handbook of Document Image Processing and Recognition. Springer-Verlag, London, Chapter 5: Page Segmentation Techniques in Document Analysis."},{"key":"e_1_3_3_26_2","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1109\/ICDM.2002.1183910","volume-title":"2002 IEEE International Conference on Data Mining, 2002 (ICDM'03)","author":"Kovacevic Milos","year":"2002","unstructured":"Milos Kovacevic, Michelangelo Diligenti, Marco Gori, and Veljko Milutinovic. 2002. Recognition of common areas in a web page using visual information: A possible application in a page classification. In 2002 IEEE International Conference on Data Mining, 2002 (ICDM'03). IEEE, 250\u2013257."},{"key":"e_1_3_3_27_2","first-page":"374","volume-title":"International Conference on Web Engineering","author":"Kreuzer Robert","year":"2015","unstructured":"Robert Kreuzer, Jurriaan Hage, and Ad Feelders. 2015. A quantitative comparison of semantic web page segmentation approaches. In International Conference on Web Engineering. Springer, 374\u2013391."},{"key":"e_1_3_3_28_2","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1145\/2034691.2034721","volume-title":"11th ACM Symposium on Document Engineering (DocEng\u201911)","author":"Kr\u00fcpl-Sypien Bernhard","year":"2011","unstructured":"Bernhard Kr\u00fcpl-Sypien, Ruslan R. Fayzrakhmanov, Wolfgang Holzinger, Mathias Panzenb\u00f6ck, and Robert Baumgartner. 2011. A versatile model for web page representation, information extraction and content re-packaging. In 11th ACM Symposium on Document Engineering (DocEng\u201911). ACM, New York, 129\u2013138."},{"doi-asserted-by":"publisher","key":"e_1_3_3_29_2","DOI":"10.1016\/j.intcom.2005.11.006"},{"doi-asserted-by":"publisher","key":"e_1_3_3_30_2","DOI":"10.2307\/2529310"},{"doi-asserted-by":"publisher","key":"e_1_3_3_31_2","DOI":"10.1145\/989863.989875"},{"key":"e_1_3_3_32_2","first-page":"31","volume-title":"16th International Conference on World Wide Web, WWW 2007","author":"Mahmud J. U.","year":"2007","unstructured":"J. U. Mahmud, Y. Borodin, and I. V. Ramakrishnan. 2007. Csurf: A context-driven non-visual webbrowser. In 16th International Conference on World Wide Web, WWW 2007. ACM, New York, 31\u201340."},{"key":"e_1_3_3_33_2","first-page":"416","volume-title":"8th International Conference on Computer Vision","volume":"2","author":"Martin D.","year":"2001","unstructured":"D. Martin, C. Fowlkes, D. Tal, and J. Malik. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In 8th International Conference on Computer Vision, Vol. 2. 416\u2013423."},{"key":"e_1_3_3_34_2","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1109\/ICDAR.2017.75","volume-title":"2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR)","volume":"1","author":"Meier Benjamin","year":"2017","unstructured":"Benjamin Meier, Thilo Stadelmann, Jan Stampfli, Marek Arnold, and Mark Cieliebak. 2017. Fully convolutional neural networks for newspaper article segmentation. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Vol. 1. IEEE, 414\u2013419."},{"key":"e_1_3_3_35_2","first-page":"347","volume-title":"ICPR","author":"Nagy G.","year":"1984","unstructured":"G. Nagy and S. Seth. 1984. Hierarchical representation of optically scanned documents. In ICPR. 347\u2013349."},{"doi-asserted-by":"publisher","key":"e_1_3_3_36_2","DOI":"10.5555\/52121"},{"key":"e_1_3_3_37_2","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1145\/1054972.1055078","volume-title":"SIGCHI Conference on Human Factors in Computing Systems (CHI\u201905)","author":"Rosenholtz Ruth","year":"2005","unstructured":"Ruth Rosenholtz, Yuanzhen Li, Jonathan Mansfield, and Zhenlan Jin. 2005. Feature congestion: A measure of display clutter. In SIGCHI Conference on Human Factors in Computing Systems (CHI\u201905). ACM, New York, 761\u2013770."},{"doi-asserted-by":"publisher","key":"e_1_3_3_38_2","DOI":"10.1109\/ICMCS.2014.6911249"},{"key":"e_1_3_3_39_2","first-page":"81","volume-title":"ASSETS 2002","author":"Takagi H.","year":"2002","unstructured":"H. Takagi, C. Asakawa, K. Fukuda, and J. Maeda. 2002. Site-wide annotation: Reconstructing existing pages to be accessible. In ASSETS 2002. ACM Press, 81\u201388."},{"unstructured":"TUWIEN Database and Artificial Intelligence Group. [n.d.]. TUWIEN Project ABBA: Web Accessibility. http:\/\/www.dbai.tuwien.ac.at\/proj\/ABBA\/. Accessed June 4 2014.","key":"e_1_3_3_40_2"},{"doi-asserted-by":"publisher","key":"e_1_3_3_41_2","DOI":"10.1007\/11581062_9"},{"unstructured":"W3C. 2010. WAI-ARIA 1.0 Primer: An introduction to rich Internet application accessibility challenges and solutions. https:\/\/www.w3.org\/TR\/wai-aria-primer\/. (accessed Feb 16 2016). (2010).","key":"e_1_3_3_42_2"},{"unstructured":"W3C. 2014. Accessible Rich Internet Applications (WAI-ARIA) 1.0. W3C Recommendation. http:\/\/www.w3.org\/TR\/wai-aria. (accessed Feb 16 2016). (2014).","key":"e_1_3_3_43_2"},{"key":"e_1_3_3_44_2","volume-title":"ICWE 2004 (LNCS)","author":"Yesilada Y.","year":"2004","unstructured":"Y. Yesilada, S. Harper, C. A. Goble, and R. Stevens. 2004. Screen readers cannot see (ontology based semantic annotation for visually impaired web travellers). In ICWE 2004 (LNCS), N. Koch, P. Fraternali, and M. Wirsing (Eds.), Vol. 3140. Springer, Berlin."},{"doi-asserted-by":"publisher","key":"e_1_3_3_45_2","DOI":"10.1145\/1279700.1279704"},{"doi-asserted-by":"publisher","key":"e_1_3_3_46_2","DOI":"10.1016\/j.ipm.2017.02.002"},{"doi-asserted-by":"publisher","key":"e_1_3_3_47_2","DOI":"10.1145\/3372117"}],"container-title":["ACM Transactions on the Web"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580519","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3580519","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:46:35Z","timestamp":1750178795000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580519"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,21]]},"references-count":46,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,8,31]]}},"alternative-id":["10.1145\/3580519"],"URL":"https:\/\/doi.org\/10.1145\/3580519","relation":{},"ISSN":["1559-1131","1559-114X"],"issn-type":[{"type":"print","value":"1559-1131"},{"type":"electronic","value":"1559-114X"}],"subject":[],"published":{"date-parts":[[2024,8,21]]},"assertion":[{"value":"2021-01-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-12-06","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}