{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,17]],"date-time":"2026-01-17T03:06:56Z","timestamp":1768619216181,"version":"3.49.0"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2017,8,16]],"date-time":"2017-08-16T00:00:00Z","timestamp":1502841600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"ICT program of the EC under project OPTIMUM","award":["H2020-MG-636160"],"award-info":[{"award-number":["H2020-MG-636160"]}]},{"DOI":"10.13039\/501100004329","name":"Slovenian Research Agency","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004329","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2017,10,31]]},"abstract":"<jats:p>Scaled acquisition of high-quality structured knowledge has been a longstanding goal of Artificial Intelligence research. Recent advances in crowdsourcing, the sheer number of Internet and mobile users, and the commercial availability of supporting platforms offer new tools for knowledge acquisition. This article applies context-aware knowledge acquisition that simultaneously satisfies users\u2019 immediate information needs while extending its own knowledge using crowdsourcing. The focus is on knowledge acquisition on a mobile device, which makes the approach practical and scalable; in this context, we propose and implement a new KA approach that exploits an existing knowledge base to drive the KA process, communicate with the right people, and check for consistency of the user-provided answers. We tested the viability of the approach in experiments using our platform with real users around the world, and an existing large source of common-sense background knowledge. These experiments show that the approach is promising: the knowledge is estimated to be true and useful for users 95% of the time. Using context to proactively drive knowledge acquisition increased engagement and effectiveness (the number of new assertions\/day\/user increased for 175%). Using pre-existing and newly acquired knowledge also proved beneficial.<\/jats:p>","DOI":"10.1145\/3086686","type":"journal-article","created":{"date-parts":[[2017,8,24]],"date-time":"2017-08-24T11:49:04Z","timestamp":1503575344000},"page":"1-46","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["Curious Cat--Mobile, Context-Aware Conversational Crowdsourcing Knowledge Acquisition"],"prefix":"10.1145","volume":"35","author":[{"given":"Luka","family":"Brade\u0161ko","sequence":"first","affiliation":[{"name":"Jo\u017eef Stefan Institute and Jo\u017eef Stefan Postgraduate School, Ljubljana Slovenia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Witbrock","sequence":"additional","affiliation":[{"name":"IBM Thomas J Watson Research Center, NY"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Janez","family":"Starc","sequence":"additional","affiliation":[{"name":"Jo\u017eef Stefan Institute and Jo\u017eef Stefan Postgraduate School, Ljubljana Slovenia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zala","family":"Herga","sequence":"additional","affiliation":[{"name":"Jo\u017eef Stefan Institute and Jo\u017eef Stefan Postgraduate School, Ljubljana Slovenia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marko","family":"Grobelnik","sequence":"additional","affiliation":[{"name":"Jo\u017eef Stefan Institute, Ljubljana Slovenia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dunja","family":"Mladeni\u0107","sequence":"additional","affiliation":[{"name":"Jo\u017eef Stefan Institute and Jo\u017eef Stefan Postgraduate School, Ljubljana Slovenia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,8,16]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the AAAI 2005: International Symposium on Explanation-Aware Computing.","author":"Baxter D.","unstructured":"D. Baxter , B. Shepard , N. Siegel , B. Gottesman , and D. Schneider . 2005. Interactive natural language explanations of cyc inferences . In Proceedings of the AAAI 2005: International Symposium on Explanation-Aware Computing. D. Baxter, B. Shepard, N. Siegel, B. Gottesman, and D. Schneider. 2005. Interactive natural language explanations of cyc inferences. In Proceedings of the AAAI 2005: International Symposium on Explanation-Aware Computing."},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the 5th International Workshop on Semantic Sensor Networks. 97","author":"Brade\u0161ko Luka","year":"2012","unstructured":"Luka Brade\u0161ko , Alexandra Moraru , Blaz Fortuna , Carolina Fortuna , and Dunja Mladenic . 2012 . A framework for acquiring semantic sensor descriptions (short paper) . In Proceedings of the 5th International Workshop on Semantic Sensor Networks. 97 . Luka Brade\u0161ko, Alexandra Moraru, Blaz Fortuna, Carolina Fortuna, and Dunja Mladenic. 2012. A framework for acquiring semantic sensor descriptions (short paper). In Proceedings of the 5th International Workshop on Semantic Sensor Networks. 97."},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the 8th Conference on Language Technologies. Ljubljana: Institut Jo\u017eef Stefan.","author":"Brade\u0161ko Luka","year":"2012","unstructured":"Luka Brade\u0161ko and Dunja Mladeni\u0107 . 2012 . A survey of chatbot systems through a loebner prize competition . In Proceedings of the 8th Conference on Language Technologies. Ljubljana: Institut Jo\u017eef Stefan. Luka Brade\u0161ko and Dunja Mladeni\u0107. 2012. A survey of chatbot systems through a loebner prize competition. In Proceedings of the 8th Conference on Language Technologies. Ljubljana: Institut Jo\u017eef Stefan."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/IS.2016.7737428"},{"key":"e_1_2_1_5_1","unstructured":"Luka Brade\u0161ko Ricardo Vitorino Zala Herga Hasibur Rahman Stephane Junique Kalaboukas Kostantinos and Margherita Forcolin. 2015. Integrated mobility decision support (Mobis D4.5). EU Project Deliverable.  Luka Brade\u0161ko Ricardo Vitorino Zala Herga Hasibur Rahman Stephane Junique Kalaboukas Kostantinos and Margherita Forcolin. 2015. Integrated mobility decision support (Mobis D4.5). EU Project Deliverable."},{"key":"e_1_2_1_6_1","unstructured":"Kino Coursey. 2004. Living in cyn: Mating AIML and cyc together with program n. http:\/\/www.daxtron.com\/pdf\/cyn_description.pdf.  Kino Coursey. 2004. Living in cyn: Mating AIML and cyc together with program n. http:\/\/www.daxtron.com\/pdf\/cyn_description.pdf."},{"key":"e_1_2_1_7_1","unstructured":"Cycorp. 2012. Foundations of Knowledge Representation in cyc -- Microtheories. Cyc 101 Tutorial. cycorp corporation. http:\/\/www.cyc.com\/doc\/tut\/ppoint\/MoreContentAreas files\/v3 document.htm.  Cycorp. 2012. Foundations of Knowledge Representation in cyc -- Microtheories. Cyc 101 Tutorial. cycorp corporation. http:\/\/www.cyc.com\/doc\/tut\/ppoint\/MoreContentAreas files\/v3 document.htm."},{"key":"e_1_2_1_8_1","volume-title":"Proceedimgs of the 22nd National Conference on Artificial Intelligence.","author":"Forbus K.","unstructured":"K. Forbus , C. Riesback , L. Birnbaum , K. Livingston , A. Sharma , and L. Ureel . 2007. Integrating natural language, knowledge representation and reasoning and analogical processing to learn by reading . In Proceedimgs of the 22nd National Conference on Artificial Intelligence. K. Forbus, C. Riesback, L. Birnbaum, K. Livingston, A. Sharma, and L. Ureel. 2007. Integrating natural language, knowledge representation and reasoning and analogical processing to learn by reading. In Proceedimgs of the 22nd National Conference on Artificial Intelligence."},{"key":"e_1_2_1_9_1","volume-title":"Pedro","author":"Hruschka Estevam Raphael","year":"2012","unstructured":"Estevam Raphael Hruschka and Saulo D. S . Pedro . 2012 . Conversing learning: Active learning and active social interaction for human supervision in never-ending learning systems. In Advances in Artificial Intelligence -- IBERAMIA 2012: Proceedings of the 13th Ibero-American Conference on AI (Cartagena de Indias, Colombia, November 13--16, 2012). Springer , Berlin. Estevam Raphael Hruschka and Saulo D. S. Pedro. 2012. Conversing learning: Active learning and active social interaction for human supervision in never-ending learning systems. In Advances in Artificial Intelligence -- IBERAMIA 2012: Proceedings of the 13th Ibero-American Conference on AI (Cartagena de Indias, Colombia, November 13--16, 2012). Springer, Berlin."},{"key":"e_1_2_1_10_1","unstructured":"Johan de Klee. 2010. Method and apparatus for maintaining groupings and relationships of propositions associated with a knowledge base. Patent US 8401988 B2.  Johan de Klee. 2010. Method and apparatus for maintaining groupings and relationships of propositions associated with a knowledge base. Patent US 8401988 B2."},{"key":"e_1_2_1_11_1","volume-title":"The AAAI Fall Symposiym Series","author":"Kuo Y.","year":"2010","unstructured":"Y. Kuo and J. Hsu . 2010. Goal-oriented knowledge collection . The AAAI Fall Symposiym Series , Nov. 2010 . Y. Kuo and J. Hsu. 2010. Goal-oriented knowledge collection. The AAAI Fall Symposiym Series, Nov. 2010."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/219717.219745"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1463434.1463477"},{"key":"e_1_2_1_14_1","volume-title":"Proceedings of the 2006 AAAI Spring Symposium on Formalizing and Compiling Background Knowledge and Its Applications to Knowledge Representation and Question Answering.","author":"Matuszek Cynthia","year":"2006","unstructured":"Cynthia Matuszek , John Cabral , Michael Witbrock , and John DeOliviera . 2006 . An introduction to the syntax and content of cyc . In Proceedings of the 2006 AAAI Spring Symposium on Formalizing and Compiling Background Knowledge and Its Applications to Knowledge Representation and Question Answering. Cynthia Matuszek, John Cabral, Michael Witbrock, and John DeOliviera. 2006. An introduction to the syntax and content of cyc. In Proceedings of the 2006 AAAI Spring Symposium on Formalizing and Compiling Background Knowledge and Its Applications to Knowledge Representation and Question Answering."},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI-15)","author":"Mitchell Tom","unstructured":"Tom Mitchell , W. Cohen , E. Hruschka , P. Talukdar , J. Betteridge , A. Carlson , and B. Dalvi et al. 2015. Never-ending learning . In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI-15) . Tom Mitchell, W. Cohen, E. Hruschka, P. Talukdar, J. Betteridge, A. Carlson, and B. Dalvi et al. 2015. Never-ending learning. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI-15)."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2189736.2189744"},{"key":"e_1_2_1_17_1","unstructured":"Saulo D. S. Pedro A. Appel and Estevam Hruschka. 2013. Autonomously reviewing and validating the knowledge base of a never-ending learning system.  Saulo D. S. Pedro A. Appel and Estevam Hruschka. 2013. Autonomously reviewing and validating the knowledge base of a never-ending learning system."},{"key":"e_1_2_1_18_1","volume-title":"Cleverbot Chat Engine Is Learning from the Internet to Talk Like a Human. Jan 13. Accessed","author":"Saenz Aaron","year":"2016","unstructured":"Aaron Saenz . 2010. Cleverbot Chat Engine Is Learning from the Internet to Talk Like a Human. Jan 13. Accessed 2016 . http:\/\/singularityhub.com\/2010\/01\/13\/cleverbot-chat-engine-is-learning-from-the-internet-to-talk-like-a-human\/. Aaron Saenz. 2010. Cleverbot Chat Engine Is Learning from the Internet to Talk Like a Human. Jan 13. Accessed 2016. http:\/\/singularityhub.com\/2010\/01\/13\/cleverbot-chat-engine-is-learning-from-the-internet-to-talk-like-a-human\/."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2740908.2741710"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the AAAI Fall Symposium Series.","author":"Sharma A.","unstructured":"A. Sharma and K. Forbus . 2010. Graph based reasoning and reinforcement learning for improving Q\/A performance in large knowledge-based systems . In Proceedings of the AAAI Fall Symposium Series. A. Sharma and K. Forbus. 2010. Graph based reasoning and reinforcement learning for improving Q\/A performance in large knowledge-based systems. In Proceedings of the AAAI Fall Symposium Series."},{"key":"e_1_2_1_21_1","doi-asserted-by":"crossref","unstructured":"P. Singh T. Lin E. Mueller G. Lim T. Perkins and W. Zhu. 2002. Open mind common sense: Knowledge acquisition from the general public. Cooperative Information Systems Oct. 30-Nov. 1 2002.   P. Singh T. Lin E. Mueller G. Lim T. Perkins and W. Zhu. 2002. Open mind common sense: Knowledge acquisition from the general public. Cooperative Information Systems Oct. 30-Nov. 1 2002.","DOI":"10.1007\/3-540-36124-3_77"},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the Workshop on Common Sense and Interactive Applications.","author":"Speer Robert","year":"2007","unstructured":"Robert Speer . 2007 . Open mind commons: An inquisitive approach to learning common sense . In Proceedings of the Workshop on Common Sense and Interactive Applications. Robert Speer. 2007. Open mind commons: An inquisitive approach to learning common sense. In Proceedings of the Workshop on Common Sense and Interactive Applications."},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the 23rd National Conference on Artificial Intelligence (AAAI\u201908)","volume":"1","author":"Speer Robert","year":"2008","unstructured":"Robert Speer , Catherine Havasi , and Henry Lieberman . 2008 . AnalogySpace: Reducing the dimensionality of common sense knowledge . In Proceedings of the 23rd National Conference on Artificial Intelligence (AAAI\u201908) . Vol. 1 . AAAI Press, Chicago, Illinois, 548--553. http:\/\/dl.acm.org\/citation.cfm?id&equals;16 19995.1620084. Robert Speer, Catherine Havasi, and Henry Lieberman. 2008. AnalogySpace: Reducing the dimensionality of common sense knowledge. In Proceedings of the 23rd National Conference on Artificial Intelligence (AAAI\u201908). Vol. 1. AAAI Press, Chicago, Illinois, 548--553. http:\/\/dl.acm.org\/citation.cfm?id&equals;1619995.1620084."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1502650.1502672"},{"key":"e_1_2_1_25_1","unstructured":"Richard Wallace. 2013. AIML 2.0 Working Draft. https:\/\/docs.google.com\/document\/d\/1wNT25hJRyupcG51aO89UcQEiG-HkXRXusukADpFnDs4\/pub.  Richard Wallace. 2013. AIML 2.0 Working Draft. https:\/\/docs.google.com\/document\/d\/1wNT25hJRyupcG51aO89UcQEiG-HkXRXusukADpFnDs4\/pub."},{"key":"e_1_2_1_26_1","volume-title":"The Elements of AIML Style","author":"Wallace Richard","unstructured":"Richard Wallace . 2003. The Elements of AIML Style . Alice AI Foundation . Richard Wallace. 2003. The Elements of AIML Style. Alice AI Foundation."},{"key":"e_1_2_1_27_1","unstructured":"Bruce Wilcox. 2011. \u201cBeyond Fa\u00e7ade: Pattern Matching for Natural Language Applications.\u201d Telltale Games. http:\/\/chatscript.sourceforge.net\/Documentation\/Pattern_Matching_for_Natural_Language_Applications.pdf.  Bruce Wilcox. 2011. \u201cBeyond Fa\u00e7ade: Pattern Matching for Natural Language Applications.\u201d Telltale Games. http:\/\/chatscript.sourceforge.net\/Documentation\/Pattern_Matching_for_Natural_Language_Applications.pdf."},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 18th International Joint Conference on Artificial Intelligence.","author":"Witbrock Michael","year":"2003","unstructured":"Michael Witbrock , David Baxter , Jon Curtis , Dave Schneider , Robert Kahlert , Perluigi Miraglia , Peter Wagner , Kathy Panton , Gavin Matthews , and Amanda Vizedom . 2003 . An interactive dialogue system for knowledge acquisition in cyc . In Proceedings of the 18th International Joint Conference on Artificial Intelligence. Michael Witbrock, David Baxter, Jon Curtis, Dave Schneider, Robert Kahlert, Perluigi Miraglia, Peter Wagner, Kathy Panton, Gavin Matthews, and Amanda Vizedom. 2003. An interactive dialogue system for knowledge acquisition in cyc. In Proceedings of the 18th International Joint Conference on Artificial Intelligence."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-013-1369-6"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3086686","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3086686","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:30:13Z","timestamp":1750217413000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3086686"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,8,16]]},"references-count":29,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2017,10,31]]}},"alternative-id":["10.1145\/3086686"],"URL":"https:\/\/doi.org\/10.1145\/3086686","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,8,16]]},"assertion":[{"value":"2016-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-08-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}