{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T05:21:09Z","timestamp":1784092869470,"version":"3.55.0"},"reference-count":32,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2011,2,3]],"date-time":"2011-02-03T00:00:00Z","timestamp":1296691200000},"content-version":"vor","delay-in-days":94,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc of Assoc for Info"],"published-print":{"date-parts":[[2010,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Self\u2010assessment of topic\/task knowledge is a human metacognitive capacity that impacts information behavior, for example through selection of learning and search strategies. It is often used as a measure in experiments for evaluation of results and those measurements are taken to be generally reliable. We conducted a user study (n=40) to test this by constructing a concept\u2010based topic knowledge representation for each participant and then comparing it with the participant judgment of their topic knowledge elicited with Likert\u2010scale questions. The tasks were in the genomics domain and knowledge representations were constructed from the MeSH thesaurus terms that indexed relevant documents for five topics. The participants rated their familiarity with the topic, the anticipated task difficulty, the amount of learning gained during the task, and made other knowledge\u2010related judgments associated with the task. Although there is considerable variability over individuals, the results provide evidence that these self\u2010assessed topic knowledge measures are correlated in the expected way with the independently\u2010constructed topic knowledge measure. We argue the results provide evidence for the general validity of topic knowledge self\u2010assessment and discuss ways to further explore knowledge self\u2010assessment and its reliability for prediction of individual knowledge levels.<\/jats:p>","DOI":"10.1002\/meet.14504701285","type":"journal-article","created":{"date-parts":[[2011,2,3]],"date-time":"2011-02-03T16:35:46Z","timestamp":1296750946000},"page":"1-10","source":"Crossref","is-referenced-by-count":8,"title":["Are self\u2010assessments reliable indicators of topic knowledge?"],"prefix":"10.1002","volume":"47","author":[{"given":"Michael J.","family":"Cole","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangmin","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinging","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chang","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicholas J.","family":"Belkin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ralf","family":"Bierig","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacek","family":"Gwizdka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2011,2,3]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0277(83)90012-4"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1394251.1394261"},{"key":"e_1_2_8_4_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026722"},{"key":"e_1_2_8_5_1","doi-asserted-by":"crossref","unstructured":"Bierig R. Cole M.J. &Gwizdka J.(2010).A data analysis and modelling framework for the evaluation of interactive information retrieval. InProceedings of ECIR 2010 (Milton Keynes UK 2009) Lecture Notes in Computer Science. Berlin: Springer.","DOI":"10.1007\/978-3-642-12275-0_76"},{"key":"e_1_2_8_6_1","volume-title":"Applying the Rasch model: Fundamental measurement in the human sciences","author":"Bond T.G.","year":"2007"},{"issue":"3","key":"e_1_2_8_7_1","article-title":"The IIR evaluation model: A framework for evaluation of interactive information retrieval systems","volume":"8","author":"Borlund P.","year":"2003","journal-title":"Information Research"},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00138746"},{"key":"e_1_2_8_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.system.2005.08.004"},{"key":"e_1_2_8_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.system.2008.03.001"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03200643"},{"key":"e_1_2_8_12_1","volume-title":"The language of thought","author":"Fodor J.","year":"1975"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.system.2009.09.006"},{"key":"e_1_2_8_14_1","doi-asserted-by":"crossref","unstructured":"Heilman M.&Eskenazi M.(2008).Self\u2010assessment in vocabulary tutoring.In Proceedings of Intelligent Tutoring Systems 9th International Conference ITS 2008 Montreal Canada June 23\u201327 2008 volume 5091\/2008 of LNCS 656\u2013658 Berlin. Springer.","DOI":"10.1007\/978-3-540-69132-7_68"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-008-9076-6"},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199304)44:3<161::AID-ASI5>3.0.CO;2-8"},{"key":"e_1_2_8_17_1","volume-title":"Semantic theory","author":"Katz J.","year":"1972"},{"key":"e_1_2_8_18_1","doi-asserted-by":"crossref","unstructured":"Kelly D.&Cool C.(2002).The effects of topic familiarity on information search behavior. InProceedings of the Second ACM\/IEEE Joint Conference on Digital Libraries (JCDL \u203202) Portland Oregon 74\u201375. New York. ACM.","DOI":"10.1145\/544220.544232"},{"key":"e_1_2_8_19_1","doi-asserted-by":"crossref","unstructured":"Kelly D. Kantor P. Morse E. Scholtz J. &Sun Y.(2006).User\u2010centered evaluation of interactive question answering systems. InProceedings of the Interactive Question Answering Workshop at HLT\u2010NAACL 2006 June 9 2006 New York New York 49\u201356. Stroudsburg PA. Association for Computational Linguistics.","DOI":"10.3115\/1641579.1641586"},{"key":"e_1_2_8_20_1","first-page":"87","volume-title":"Automated Essay Scoring: A cross\u2010disciplinary perspective","author":"Landauer T.K.","year":"2003"},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0079-7421(02)80004-4"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1191\/0265532205lt297oa"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1177\/026553228700400202"},{"key":"e_1_2_8_24_1","doi-asserted-by":"crossref","unstructured":"Mitrovic A.(2001).Investigating students' self\u2010assessment skills. InProceedings of the 8th International Conference for User Modeling Sonthofen Germany July 13\u201317 2001 volume 2109\/2010 of LNCS 247\u2013250 Berlin. Springer.","DOI":"10.1007\/3-540-44566-8_35"},{"issue":"2","key":"e_1_2_8_25_1","doi-asserted-by":"crossref","first-page":"121","DOI":"10.3233\/IRG-2007-17(2)03","article-title":"Evaluating the Effect of Open Student Models on Self\u2010Assessment","volume":"17","author":"Mitrovic A.","year":"2007","journal-title":"International Journal of Artificial Intelligence in Education"},{"key":"e_1_2_8_26_1","unstructured":"Pirrone R. Azevedo R.&Biswas G.(2009).Why metacognition in modern education systems?InProceedings of AAAI Fall Symposium on Cognitive and Metacognitive Educational Systems Alexandria Virginia USA November 8 2009. vii\u2010viii Menlo Park CA. AAAI Press"},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-008-9072-x"},{"issue":"10","key":"e_1_2_8_28_1","first-page":"1","article-title":"The reliability, validity, and utility of self\u2010assessment","volume":"11","author":"Ross J.A.","year":"2006","journal-title":"Practical Assessment Research & Evaluation"},{"key":"e_1_2_8_29_1","unstructured":"Toms E. Mackenzie T. Jordan C. O'Brien H. Freund L. Toze S. Dawe E. &MacNutt A.(2007).How task affects information search. InWorkshop Pre\u2010Proceedings in Initiative for the Evaluation of XML Retrieval (INEX).337\u2013341."},{"key":"e_1_2_8_30_1","doi-asserted-by":"crossref","unstructured":"Wen L. Ruthven I. &Borlund P.(2006).The effects on topic familiarity on online search behaviour and use of relevance criteria. InAdvances in Information Retrieval 28th European Conference on IR Research ECIR 2006 (London UK) 456\u2013459 London. Springer","DOI":"10.1007\/11735106_40"},{"key":"e_1_2_8_31_1","doi-asserted-by":"crossref","unstructured":"White R.W. Dumais S.T. &Teevan J.(2009).Characterizing the influence of domain expertise on web search behavior. In Proceedings of the Second ACM International Conference on Web Search and Data Mining pages132\u2013141. ACM New York NY.","DOI":"10.1145\/1498759.1498819"},{"key":"e_1_2_8_32_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.10367"},{"issue":"2","key":"e_1_2_8_33_1","article-title":"Domain knowledge, search behaviour, and search effectiveness of engineering and science students: An exploratory study","volume":"10","author":"Zhang X.","year":"2005","journal-title":"Information Research"}],"container-title":["Proceedings of the American Society for Information Science and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fmeet.14504701285","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fmeet.14504701285","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/meet.14504701285","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,20]],"date-time":"2025-10-20T14:54:06Z","timestamp":1760972046000},"score":1,"resource":{"primary":{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/10.1002\/meet.14504701285"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,11]]},"references-count":32,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2010,11]]}},"alternative-id":["10.1002\/meet.14504701285"],"URL":"https:\/\/doi.org\/10.1002\/meet.14504701285","archive":["Portico"],"relation":{},"ISSN":["0044-7870","1550-8390"],"issn-type":[{"value":"0044-7870","type":"print"},{"value":"1550-8390","type":"electronic"}],"subject":[],"published":{"date-parts":[[2010,11]]}}}