{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T05:09:46Z","timestamp":1784783386438,"version":"3.55.0"},"reference-count":37,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2020,6,24]],"date-time":"2020-06-24T00:00:00Z","timestamp":1592956800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Librarianship and Information Science"],"published-print":{"date-parts":[[2021,3]]},"abstract":"<jats:p>Evaluation criteria and appropriate measures are critical to the success of digital library evaluation. The key problem is a lack of specific measures, especially from a heuristic perspective, corresponding to diverse dimensions and criteria. This study explored a variety of measures for digital library evaluation focusing on their appropriateness. It also compared the similarities and differences in perceptions of the appropriateness of digital library evaluation measures of two groups. Sixty-one participants were recruited representing scholars and academic digital librarians. The participants were instructed to fill in an in-depth survey consisting of 174 measures associated with 10 evaluation dimensions and 82 criteria. The findings of this study highlight the most appropriate measures in each dimension and show significant agreement in identifying appropriate measures by digital library scholars and librarians. Differences in the rating of the measures were found between the two groups in the dimensions of \u201cinterface design,\u201d \u201csystem and technology,\u201d \u201ceffects on users,\u201d \u201cadministration,\u201d \u201cuser engagement,\u201d and \u201ccontext.\u201d The significance of this study lies not only in the integration of the perspectives on measures from both researchers and practitioners, but also in providing an inclusive list of measures to guide practitioners to effectively evaluate digital libraries in academic settings.<\/jats:p>","DOI":"10.1177\/0961000620935505","type":"journal-article","created":{"date-parts":[[2020,6,24]],"date-time":"2020-06-24T23:19:49Z","timestamp":1593040789000},"page":"130-152","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":11,"title":["Digital library evaluation measures in academic settings: Perspectives from scholars and practitioners"],"prefix":"10.1177","volume":"53","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5901-4894","authenticated-orcid":false,"given":"Iris","family":"Xie","sequence":"first","affiliation":[{"name":"School of Information Studies, University of Wisconsin\u2013Milwaukee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6770-5118","authenticated-orcid":false,"given":"Soohyung","family":"Joo","sequence":"additional","affiliation":[{"name":"School of Information Science, University of Kentucky, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Krystyna K.","family":"Matusiak","sequence":"additional","affiliation":[{"name":"Research Methods and Information Science Department, University of Denver, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2020,6,24]]},"reference":[{"key":"bibr1-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1108\/NLW-07-2014-0087"},{"key":"bibr2-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1045\/march2007-castelli"},{"key":"bibr3-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1007\/s00799-014-0116-0"},{"key":"bibr4-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-58706-6_14"},{"key":"bibr5-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-44796-2_17"},{"key":"bibr6-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1007\/s00799-007-0011-z"},{"key":"bibr7-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1108\/PROG-09-2014-0070"},{"key":"bibr8-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1007\/s00799-008-0033-1"},{"key":"bibr9-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1145\/984321.984325"},{"key":"bibr10-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2006.11.010"},{"key":"bibr11-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1108\/AJIM-09-2017-0216"},{"key":"bibr12-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1177\/0961000616668572"},{"key":"bibr13-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1515\/LIBR.2005.96"},{"key":"bibr14-0961000620935505","doi-asserted-by":"publisher","DOI":"10.4018\/978-1-4666-2991-2.ch008"},{"key":"bibr15-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1080\/19322909.2014.954740"},{"key":"bibr16-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1080\/19322909.2017.1391733"},{"key":"bibr17-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1145\/3197026.3203871"},{"key":"bibr18-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1108\/EL-05-2012-0046"},{"key":"bibr19-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2018.10.012"},{"key":"bibr20-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1108\/EL-09-2018-0185"},{"key":"bibr21-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1080\/1941126X.2016.1200927"},{"key":"bibr22-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1016\/j.acalib.2009.11.005"},{"key":"bibr23-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1108\/07378831111189787"},{"key":"bibr24-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1108\/00012531211215169"},{"key":"bibr25-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1108\/PMM-03-2018-0012"},{"key":"bibr26-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1080\/01639370902737240"},{"key":"bibr27-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1080\/19322909.2014.901211"},{"key":"bibr28-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1080\/19322909.2016.1223573"},{"key":"bibr29-0961000620935505","unstructured":"Saracevic T (2004) Evaluation of digital libraries. In: Agosti M, F\u00fchr N (eds) Notes of the DELOS WP7 Workshop on the Evaluation of Digital Libraries, Padua, Italy. Available at: http:\/\/dlib.ionio.gr\/wp7\/workshop2004_program.html"},{"key":"bibr30-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1515\/abitech-2018-0006"},{"key":"bibr31-0961000620935505","volume-title":"Notes of the DELOS WP7 Workshop on the Evaluation of Digital Libraries","author":"Tsakonas G","year":"2004"},{"key":"bibr32-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1016\/j.lisr.2006.06.002"},{"key":"bibr33-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1016\/j.acalib.2018.09.002"},{"key":"bibr34-0961000620935505","volume-title":"Discovery Digital Libraries: Theory and Practice","author":"Xie I","year":"2016"},{"key":"bibr35-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1108\/LHT-04-2013-0042"},{"key":"bibr36-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1109\/ETTLIS.2018.8485273"},{"key":"bibr37-0961000620935505","doi-asserted-by":"publisher","DOI":"10.1002\/asi.21220"}],"container-title":["Journal of Librarianship and Information Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0961000620935505","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0961000620935505","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0961000620935505","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T22:58:14Z","timestamp":1777676294000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0961000620935505"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,24]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,3]]}},"alternative-id":["10.1177\/0961000620935505"],"URL":"https:\/\/doi.org\/10.1177\/0961000620935505","relation":{},"ISSN":["0961-0006","1741-6477"],"issn-type":[{"value":"0961-0006","type":"print"},{"value":"1741-6477","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,24]]}}}