{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,22]],"date-time":"2025-12-22T22:14:54Z","timestamp":1766441694450,"version":"3.41.0"},"reference-count":22,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2013,10,1]],"date-time":"2013-10-01T00:00:00Z","timestamp":1380585600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2013,10]]},"abstract":"<jats:p>Efficient learning from Web resources can depend on accurately assessing the quality of each resource. We present a methodology for developing computational models of quality that can assist users in assessing Web resources. The methodology consists of four steps: 1) a meta-analysis of previous studies to decompose quality into high-level dimensions and low-level indicators, 2) an expert study to identify the key low-level indicators of quality in the target domain, 3) human annotation to provide a collection of example resources where the presence or absence of quality indicators has been tagged, and 4) training of a machine learning model to predict quality indicators based on content and link features of Web resources. We find that quality is a multifaceted construct, with different aspects that may be important to different users at different times. We show that machine learning models can predict this multifaceted nature of quality, both in the context of aiding curators as they evaluate resources submitted to digital libraries, and in the context of aiding teachers as they develop online educational resources. Finally, we demonstrate how computational models of quality can be provided as a service, and embedded into applications such as Web search.<\/jats:p>","DOI":"10.1145\/2533670.2533673","type":"journal-article","created":{"date-parts":[[2013,11,6]],"date-time":"2013-11-06T14:09:19Z","timestamp":1383746959000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Characterizing and Predicting the Multifaceted Nature of Quality in Educational Web Resources"],"prefix":"10.1145","volume":"3","author":[{"given":"Philipp","family":"Wetzler","sequence":"first","affiliation":[{"name":"University of Colorado Boulder"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Steven","family":"Bethard","sequence":"additional","affiliation":[{"name":"University of Colorado Boulder"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Heather","family":"Leary","sequence":"additional","affiliation":[{"name":"University of Colorado Boulder"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kirsten","family":"Butcher","sequence":"additional","affiliation":[{"name":"University of Utah"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Soheil Danesh","family":"Bahreini","sequence":"additional","affiliation":[{"name":"University of Colorado Boulder"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jin","family":"Zhao","sequence":"additional","affiliation":[{"name":"National University of Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James H.","family":"Martin","sequence":"additional","affiliation":[{"name":"University of Colorado Boulder"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tamara","family":"Sumner","sequence":"additional","affiliation":[{"name":"University of Colorado Boulder"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2013,10]]},"reference":[{"volume-title":"Proceedings of the 3rd International Conference on Educational Data Mining. 267--268","author":"Aleahmad T.","key":"e_1_2_1_1_1"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1555400.1555436"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1367497.1367673"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-7552(98)00110-X"},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Custard M. and Sumner T. 2005. Using machine learning to support quality judgments. D-Lib Magazine 11.  Custard M. and Sumner T. 2005. Using machine learning to support quality judgments. D-Lib Magazine 11 .","DOI":"10.1045\/october2005-custard"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.21437"},{"volume-title":"Proceedings of the AAAI Workshop on Wikipedia and Artificial Intelligence: An Evolving Synergy.","year":"2008","author":"Druck G.","key":"e_1_2_1_7_1"},{"volume-title":"Protocol Analysis: Verbal Reports as Data","year":"1984","author":"Ericsson K. A.","key":"e_1_2_1_8_1"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0057532"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/365024.365037"},{"volume-title":"A Standard Sample of Present-Day English for Use with Digital Computers. Report to the U.S Office of Education on Cooperative Research Project No. E-007","author":"Francis W. N.","key":"e_1_2_1_11_1"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03195564"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/365024.365035"},{"volume-title":"Proceedings of the 23rd International Conference on Computational Linguistics. 546--554","author":"Kate R. J.","key":"e_1_2_1_14_1"},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Kincaid J. P. Fishburne R. P. Rogers R. and Chissom B. S. 1975. Derivation of new readability formulas (automated readability index fog count and flesch reading ease formula) for Navy enlisted personnel. Research branch rep. 8-75 Naval Technical Training U. S. Naval Air Station Memphis TN.  Kincaid J. P. Fishburne R. P. Rogers R. and Chissom B. S. 1975. Derivation of new readability formulas (automated readability index fog count and flesch reading ease formula) for Navy enlisted personnel. Research branch rep. 8-75 Naval Technical Training U. S. Naval Air Station Memphis TN.","DOI":"10.21236\/ADA006655"},{"volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing. 186--195","author":"Pitler E.","key":"e_1_2_1_16_1"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2009.08.006"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.10017.abs"},{"volume-title":"Proceedings of the International Conference on Knowledge Management. 101--113","author":"Stvilia B.","key":"e_1_2_1_19_1"},{"volume-title":"Proceedings of the 3rd ACM\/IEEE-CS Joint Conference on Digital Libraries. 269--279","author":"Sumner T.","key":"e_1_2_1_20_1"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1501434.1501445"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/345508.345602"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2533670.2533673","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2533670.2533673","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T07:34:05Z","timestamp":1750232045000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2533670.2533673"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,10]]},"references-count":22,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2013,10]]}},"alternative-id":["10.1145\/2533670.2533673"],"URL":"https:\/\/doi.org\/10.1145\/2533670.2533673","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"type":"print","value":"2160-6455"},{"type":"electronic","value":"2160-6463"}],"subject":[],"published":{"date-parts":[[2013,10]]},"assertion":[{"value":"2012-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-12-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-10-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}