{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T18:58:00Z","timestamp":1764961080604,"version":"3.46.0"},"reference-count":50,"publisher":"Walter de Gruyter GmbH","issue":"1","funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["03IO1617"],"award-info":[{"award-number":["03IO1617"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,4,25]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Semantic technologies provide meaning to information resources in the form of machine-accessible structured data. Research over the past two decades has commonly focused on tools and interfaces for technical experts, leading to various usability problems regarding users unfamiliar with the underlying technologies \u2013 so-called nontechnical experts. Existing approaches to semantic technologies consider mostly consumers of structured data and leave out the creation perspective. In this work, we focus on the usability of creating structured data from textual resources, especially the creation of relations between entities. The research was conducted in collaboration with scholars from the humanities. We review existing research on the usability of semantic technologies and the state of the art of annotation tools to identify shortcomings. Subsequently we use the knowledge gained to propose a new interaction design for the creation of relations between entities to create structured data in the subject-predicate-object form. We implemented our interaction design and conducted a user study which showed that the proposal performed well, making it a contribution to enhance the overall usability in this field. However, this research provides an example of how technically sophisticated technology needs to be \u201ctranslated\u201d to make it usable for nontechnical experts. We need to extend this perspective in the future by providing more insight into the internal functioning of semantic technologies.<\/jats:p>","DOI":"10.1515\/icom-2018-0005","type":"journal-article","created":{"date-parts":[[2018,4,12]],"date-time":"2018-04-12T04:08:02Z","timestamp":1523506082000},"page":"25-40","source":"Crossref","is-referenced-by-count":4,"title":["Generating Structured Data by Nontechnical Experts in Research Settings"],"prefix":"10.1515","volume":"17","author":[{"given":"Andre","family":"Breitenfeld","sequence":"first","affiliation":[{"name":"Freie Universit\u00e4t Berlin , Human-Centered Computing , Berlin , Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Florian","family":"Berger","sequence":"additional","affiliation":[{"name":"Freie Universit\u00e4t Berlin , Human-Centered Computing , Berlin , Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ming-Tung","family":"Hong","sequence":"additional","affiliation":[{"name":"Freie Universit\u00e4t Berlin , Human-Centered Computing , Berlin , Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maximilian","family":"Mackeprang","sequence":"additional","affiliation":[{"name":"Freie Universit\u00e4t Berlin , Human-Centered Computing , Berlin , Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Claudia","family":"M\u00fcller-Birn","sequence":"additional","affiliation":[{"name":"Freie Universit\u00e4t Berlin , Human-Centered Computing , Berlin , Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"374","published-online":{"date-parts":[[2018,3,27]]},"reference":[{"key":"2025120518123280490_j_icom-2018-0005_ref_001","unstructured":"W3C DATA ACTIVITY building the web of data. https:\/\/www.w3.org\/2013\/data\/."},{"key":"2025120518123280490_j_icom-2018-0005_ref_002","unstructured":"T. Berners-Lee. Linked Data. https:\/\/www.w3.org\/DesignIssues\/LinkedData.html."},{"key":"2025120518123280490_j_icom-2018-0005_ref_003","doi-asserted-by":"crossref","unstructured":"C.\u2009L. Borgman. Big Data, Little Data, No Data. MIT Press, 2015.","DOI":"10.7551\/mitpress\/9963.001.0001"},{"key":"2025120518123280490_j_icom-2018-0005_ref_004","unstructured":"M. Burghardt. Usability recommendations for annotation tools. In Proceedings of the Sixth Linguistic Annotation Workshop, pages 104\u2013112, Jeju, Republic of Korea, July 2012. Association for Computational Linguistics."},{"key":"2025120518123280490_j_icom-2018-0005_ref_005","unstructured":"M. Burghardt. Engineering annotation usability \u2013 toward usability patterns for linguistic annotation tools, September 2014. URL http:\/\/epub.uni-regensburg.de\/30768\/. Pattern wiki: http:\/\/www.annotation-usability.net."},{"key":"2025120518123280490_j_icom-2018-0005_ref_006","unstructured":"J.-P. Cahier and M. Zacklad. Socio-semantic web applications: towards a methodology based on the theory of the communities of action. resource, 8 (400), 2004."},{"key":"2025120518123280490_j_icom-2018-0005_ref_007","doi-asserted-by":"crossref","unstructured":"J.\u2009J. Carroll, I. Dickinson, C. Dollin, D. Reynolds, A. Seaborne, and K. Wilkinson. Jena: Implementing the semantic web recommendations. In Proc. of the WWW, pages 74\u201383. ACM, 2004.","DOI":"10.1145\/1010432.1010446"},{"key":"2025120518123280490_j_icom-2018-0005_ref_008","doi-asserted-by":"crossref","unstructured":"P. Ciccarese, M. Ocana, and T. Clark. Open semantic annotation of scientific publications using DOMEO. J. Biomedical Semantics, 3 (S-1): S1, 2012a. URL http:\/\/www.jbiomedsem.com\/content\/3\/S1\/S1.","DOI":"10.1186\/2041-1480-3-S1-S1"},{"key":"2025120518123280490_j_icom-2018-0005_ref_009","doi-asserted-by":"crossref","unstructured":"P. Ciccarese, M. Ocana, and T. Clark. Open semantic annotation of scientific publications using DOMEO. Journal of Biomedical Semantics, 3 (Suppl 1): S1, 2012b. ISSN 2041-1480. doi:10.1186\/2041-1480-3-S1-S1. URL http:\/\/www.jbiomedsem.com\/supplements\/3\/S1\/S1.","DOI":"10.1186\/2041-1480-3-S1-S1"},{"key":"2025120518123280490_j_icom-2018-0005_ref_010","unstructured":"R. Crawford. The Techniques of Creative Thinking: How to Use Your Ideas to Achieve Success. Hawthorn Books, 1954. URL https:\/\/books.google.co.uk\/books?id=BCAsAQAAMAAJ."},{"key":"2025120518123280490_j_icom-2018-0005_ref_011","doi-asserted-by":"crossref","unstructured":"A.\u2009S. Dadzie, M. Rowe, and D. Petrelli. Hide the stack: Toward usable linked data. In Lecture Notes in Computer Science, pages 93\u2013107, 2011.","DOI":"10.1007\/978-3-642-21034-1_7"},{"key":"2025120518123280490_j_icom-2018-0005_ref_012","unstructured":"J. Damerow, B.\u2009E. Peirson, and M.\u2009D. Laubichler. Don\u2019t panic! a research system for network-based digital history of science. In Future of Historical Network Research Conference, 2013."},{"key":"2025120518123280490_j_icom-2018-0005_ref_013","unstructured":"P. Di Maio. Toward global user models for semantic technologies: Emergent perspectives. In Proc. of ASWC, volume 8, pages 141\u2013152, 2008."},{"key":"2025120518123280490_j_icom-2018-0005_ref_014","doi-asserted-by":"crossref","unstructured":"O. Gilson, N. Silva, P.\u2009W. Grant, and M. Chen. From web data to visualization via ontology mapping. In Computer Graphics Forum, volume 27, pages 959\u2013966, 2008.","DOI":"10.1111\/j.1467-8659.2008.01230.x"},{"key":"2025120518123280490_j_icom-2018-0005_ref_015","doi-asserted-by":"crossref","unstructured":"E. Goodman, M. Kuniavsky, and A. Moed. Observing the user experience: A practitioner\u2019s guide to user research. IEEE TPC, 56 (3): 260\u2013261, 2013.","DOI":"10.1109\/TPC.2013.2274110"},{"key":"2025120518123280490_j_icom-2018-0005_ref_016","doi-asserted-by":"crossref","unstructured":"M. Grassi, C. Morbidoni, M. Nucci, S. Fonda, and F. Piazza. Pundit: Augmenting web contents with semantics. Literary and Linguistic Computing, 28 (4): 640\u2013659, 2013.","DOI":"10.1093\/llc\/fqt060"},{"key":"2025120518123280490_j_icom-2018-0005_ref_017","unstructured":"O. Hartig. Provenance information in the web of data, 2009."},{"key":"2025120518123280490_j_icom-2018-0005_ref_018","doi-asserted-by":"crossref","unstructured":"S.\u2009R. Herring, B.\u2009R. Jones, and B.\u2009P. Bailey. Idea generation techniques among creative professionals. In System Sciences, 2009. HICSS \u201909. 42nd Hawaii International Conference on, pages 1\u201310, Jan 2009. doi:10.1109\/HICSS.2009.241.","DOI":"10.1109\/HICSS.2009.241"},{"key":"2025120518123280490_j_icom-2018-0005_ref_019","unstructured":"T. Hey, S. Tansley, K.\u2009M. Tolle, et al.The fourth paradigm: data-intensive scientific discovery, volume 1. Microsoft research Redmond, WA, 2009."},{"key":"2025120518123280490_j_icom-2018-0005_ref_020","doi-asserted-by":"crossref","unstructured":"A. Hinze, R. Heese, M. Luczak-R\u00f6sch, and A. Paschke. Semantic enrichment by non-experts: usability of manual annotation tools. In ISWC, pages 165\u2013181. Springer, 2012.","DOI":"10.1007\/978-3-642-35176-1_11"},{"key":"2025120518123280490_j_icom-2018-0005_ref_021","unstructured":"A. Hoffmeister, F. Berger, M. Pogorzhelskiy, G. Zhang, C. Zwick, and C. M\u00fcller-Birn. Toward cyber-physical research practice based on mixed reality. In M. Burghardt, R. Wimmer, C. Wolff, and C. Womser-Hacker, editors, Mensch und Computer 2017 \u2013 Workshopband, Regensburg, 2017. Gesellschaft f\u00fcr Informatik e.V."},{"key":"2025120518123280490_j_icom-2018-0005_ref_022","doi-asserted-by":"crossref","unstructured":"J. Huang, R. White, and G. Buscher. User see, user point: gaze and cursor alignment in web search. In Proc. of the SIGCHI Conference, pages 1341\u20131350. ACM, 2012.","DOI":"10.1145\/2207676.2208591"},{"key":"2025120518123280490_j_icom-2018-0005_ref_023","doi-asserted-by":"crossref","unstructured":"N. Ide and L. Romary. International standard for a linguistic annotation framework. Nat. Lang. Eng., 10 (3\u20134): 211\u2013225, Sept. 2004. ISSN 1351-3249. URL http:\/\/dx.doi.org\/10.1017\/S135132490400350X.","DOI":"10.1017\/S135132490400350X"},{"key":"2025120518123280490_j_icom-2018-0005_ref_024","doi-asserted-by":"crossref","unstructured":"A. Jameson. Usability and the Semantic Web. In ESWC, pages 3. Springer, 2006.","DOI":"10.1007\/11762256_3"},{"key":"2025120518123280490_j_icom-2018-0005_ref_025","doi-asserted-by":"crossref","unstructured":"N. Karam, C. M\u00fcller-Birn, M. Gleisberg, D. Fichtm\u00fcller, R. Tolksdorf, and A. G\u00fcntsch. A Terminology Service Supporting Semantic Annotation, Integration, Discovery and Analysis of Interdisciplinary Research Data. Datenbank-Spektrum, 16 (3): 195\u2013205, Nov. 2016. ISSN 1610-1995. URL https:\/\/doi.org\/10.1007\/s13222-016-0231-8.","DOI":"10.1007\/s13222-016-0231-8"},{"key":"2025120518123280490_j_icom-2018-0005_ref_026","doi-asserted-by":"crossref","unstructured":"D.\u2009R. Karger. The semantic web and end users: What\u2019s wrong and how to fix it. IEEE Internet Computing, 18 (6): 64\u201370, 2014.","DOI":"10.1109\/MIC.2014.124"},{"key":"2025120518123280490_j_icom-2018-0005_ref_027","doi-asserted-by":"crossref","unstructured":"A. Khalili, S. Auer, and D. Hladky. The rdfa content editor \u2013 from wysiwyg to wysiwym. In Proceedings of COMPSAC 2012 \u2013 Trustworthy Software Systems for the Digital Society, July 16\u201320, 2012, Izmir, Turkey, 2012. URL http:\/\/svn.aksw.org\/papers\/2012\/COMPSAC_RDFaCE\/public.pdf.","DOI":"10.1109\/COMPSAC.2012.72"},{"key":"2025120518123280490_j_icom-2018-0005_ref_028","doi-asserted-by":"crossref","unstructured":"A. Kiryakov, B. Popov, I. Terziev, D. Manov, and D. Ognyanoff. Semantic annotation, indexing, and retrieval. Web Semantics: Science, Services and Agents on the World Wide Web, 2 (1): 49\u201379, 2004. ISSN 1570-8268. https:\/\/doi.org\/10.1016\/j.websem.2004.07.005. URL http:\/\/www.sciencedirect.com\/science\/article\/pii\/S1570826804000162.","DOI":"10.1016\/j.websem.2004.07.005"},{"key":"2025120518123280490_j_icom-2018-0005_ref_029","doi-asserted-by":"crossref","unstructured":"S. Lohmann, S. Negru, F. Haag, and T. Ertl. Visualizing ontologies with vowl. Semantic Web, 7 (4): 399\u2013419, 2016.","DOI":"10.3233\/SW-150200"},{"key":"2025120518123280490_j_icom-2018-0005_ref_030","doi-asserted-by":"crossref","unstructured":"V. Lopez, M. Fern\u00e1ndez, E. Motta, and N. Stieler. PowerAqua: Supporting users in querying and exploring the Semantic Web. Semantic Web, 3 (3): 249\u2013265, 2012.","DOI":"10.3233\/SW-2011-0030"},{"key":"2025120518123280490_j_icom-2018-0005_ref_031","unstructured":"M. Marci\u0144czuk, J. Koco\u0144, and B. Broda. Inforex \u2013 a web-based tool for text corpus management and semantic annotation. In N.\u2009C.\u2009C. Chair, K. Choukri, T. Declerck, M.\u2009U. Do\u011fan, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, and S. Piperidis, editors, Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC\u201912), Istanbul, Turkey, may 2012. European Language Resources Association (ELRA). ISBN 978-2-9517408-7-7."},{"key":"2025120518123280490_j_icom-2018-0005_ref_032","doi-asserted-by":"crossref","unstructured":"M.\u2009J. McGuffin and R. Balakrishnan. Fitts\u2019 law and expanding targets: Experimental studies and designs for user interfaces. TOCHI, 12 (4): 388\u2013422, 2005.","DOI":"10.1145\/1121112.1121115"},{"key":"2025120518123280490_j_icom-2018-0005_ref_033","doi-asserted-by":"crossref","unstructured":"E.\u2009T. Meyer and R. Schroeder. The world wide web of research and access to knowledge. Knowledge Management Research & Practice, 7 (3): 218\u2013233, Sept. 2009. ISSN 1477-8246. URL https:\/\/doi.org\/10.1057\/kmrp.2009.13.","DOI":"10.1057\/kmrp.2009.13"},{"key":"2025120518123280490_j_icom-2018-0005_ref_034","doi-asserted-by":"crossref","unstructured":"C. M\u00fcller-Birn, T. Kl\u00fcwer, A. Breitenfeld, A. Schlegel, and L. Benedix. Neonion: Combining human and machine intelligence. In Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work &#38; Social Computing, CSCW\u201915 Companion, pages 223\u2013226, New York, NY, USA, 2015. ACM. ISBN 978-1-4503-2946-0. URL http:\/\/doi.acm.org\/10.1145\/2685553.2699012.","DOI":"10.1145\/2685553.2699012"},{"key":"2025120518123280490_j_icom-2018-0005_ref_035","doi-asserted-by":"crossref","unstructured":"J.\u2009L. Navarro-Galindo and J. Samos. The FLERSA tool: adding semantics to a web content management system. IJWIS, 8 (1): 73\u2013126, 2012. URL http:\/\/dx.doi.org\/10.1108\/17440081211222609.","DOI":"10.1108\/17440081211222609"},{"key":"2025120518123280490_j_icom-2018-0005_ref_036","doi-asserted-by":"crossref","unstructured":"M. Neves and U. Leser. A survey on annotation tools for the biomedical literature. Briefings in Bioinformatics, 2012.","DOI":"10.1093\/bib\/bbs084"},{"key":"2025120518123280490_j_icom-2018-0005_ref_037","unstructured":"J. Nielsen. 10 usability heuristics for user interface design. Fremont: Nielsen Norman Group. [Consult. 20 maio 2014]. Dispon\u00edvel na Internet, 1995."},{"key":"2025120518123280490_j_icom-2018-0005_ref_038","doi-asserted-by":"crossref","unstructured":"K. Ohara, T. Berners-Lee, W. Hall, and N. Shadbolt. Use of the semantic web in e-research. World Wide Research: Reshaping the Sciences and Humanities, page 130, 2010.","DOI":"10.7551\/mitpress\/9780262014397.003.0039"},{"key":"2025120518123280490_j_icom-2018-0005_ref_039","doi-asserted-by":"crossref","unstructured":"D. Oldman, M. Doerr, and S. Gradmann. Zen and the art of linked data. In R.\u2009S.\u2009a.\u2009U. Susan Schreibman, editor, A New Companion to Digital Humanities, pages 251\u2013273. John Wiley & Sons, Ltd, 2015. doi:10.1002\/9781118680605.ch18.","DOI":"10.1002\/9781118680605.ch18"},{"key":"2025120518123280490_j_icom-2018-0005_ref_040","doi-asserted-by":"crossref","unstructured":"J. Pratt, P.\u2009V. Radulescu, R.\u2009M. Guo, and R.\u2009A. Abrams. It\u2019s alive! animate motion captures visual attention. Psychological Science, 21 (11): 1724\u20131730, 2010.","DOI":"10.1177\/0956797610387440"},{"key":"2025120518123280490_j_icom-2018-0005_ref_041","doi-asserted-by":"crossref","unstructured":"C. Preist. A conceptual architecture for semantic web services. In ISWC, pages 395\u2013409. Springer, 2004.","DOI":"10.1007\/978-3-540-30475-3_28"},{"key":"2025120518123280490_j_icom-2018-0005_ref_042","unstructured":"G. Rizzo and R. Troncy. Nerd: Evaluating named entity recognition tools in the web of data. 2011."},{"key":"2025120518123280490_j_icom-2018-0005_ref_043","doi-asserted-by":"crossref","unstructured":"A. Sears. Heuristic walkthroughs: Finding the problems without the noise. Int. J. Hum. Comput. Interaction, 9 (3): 213\u2013234, 1997.","DOI":"10.1207\/s15327590ijhc0903_2"},{"key":"2025120518123280490_j_icom-2018-0005_ref_044","doi-asserted-by":"crossref","unstructured":"L. Serafini and A. Tamilin. Drago: Distributed reasoning architecture for the semantic web. In ESWC, pages 361\u2013376. Springer, 2005.","DOI":"10.1007\/11431053_25"},{"key":"2025120518123280490_j_icom-2018-0005_ref_045","doi-asserted-by":"crossref","unstructured":"H.\u2009 N. Talantikite, D. Aissani, and N. Boudjlida. Semantic annotations for web services discovery and composition. Comput. Stand. Interfaces, 31 (6): 1108\u20131117, Nov. 2009. ISSN 0920-5489.","DOI":"10.1016\/j.csi.2008.09.041"},{"key":"2025120518123280490_j_icom-2018-0005_ref_046","doi-asserted-by":"crossref","unstructured":"V. Uren, P. Cimiano, J. Iria, S. Handschuh, M. Vargas-Vera, E. Motta, and F. Ciravegna. Semantic annotation for knowledge management: Requirements and a survey of the state of the art. Web Semant., 4 (1): 14\u201328, Jan. 2006. ISSN 1570-8268.","DOI":"10.1016\/j.websem.2005.10.002"},{"key":"2025120518123280490_j_icom-2018-0005_ref_047","doi-asserted-by":"crossref","unstructured":"K. van Deemter and R. Kibble. What is coreference, and what should coreference annotation be? In Proceedings of the Workshop on Coreference and Its Applications, CorefApp \u201999, pages 90\u201396, Stroudsburg, PA, USA, 1999. Association for Computational Linguistics. URL http:\/\/dl.acm.org\/citation.cfm?id=1608810.1608828.","DOI":"10.3115\/1608810.1608828"},{"key":"2025120518123280490_j_icom-2018-0005_ref_048","doi-asserted-by":"crossref","unstructured":"D. Vrande\u010di\u0107 and M. Kr\u00f6tzsch. Wikidata: A Free Collaborative Knowledgebase. Commun. ACM, 2014.","DOI":"10.1145\/2629489"},{"key":"2025120518123280490_j_icom-2018-0005_ref_049","doi-asserted-by":"crossref","unstructured":"A. Widl\u00f6cher and Y. Mathet. The glozz platform: A corpus annotation and mining tool. In Proc. of the DocEng, pages 171\u2013180. ACM, 2012.","DOI":"10.1145\/2361354.2361394"},{"key":"2025120518123280490_j_icom-2018-0005_ref_050","unstructured":"M. Wilkinson and et al.The fair guiding principles for scientific data management and stewardship. Nature Scientific Data, (160018), 2016. URL http:\/\/www.nature.com\/articles\/sdata201618."}],"container-title":["i-com"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.degruyter.com\/view\/journals\/icom\/17\/1\/article-p25.xml","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/icom-2018-0005\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/icom-2018-0005\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T18:40:38Z","timestamp":1764960038000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/icom-2018-0005\/html"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,3,27]]},"references-count":50,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2018,3,27]]},"published-print":{"date-parts":[[2018,4,25]]}},"alternative-id":["10.1515\/icom-2018-0005"],"URL":"https:\/\/doi.org\/10.1515\/icom-2018-0005","relation":{},"ISSN":["2196-6826","1618-162X"],"issn-type":[{"type":"electronic","value":"2196-6826"},{"type":"print","value":"1618-162X"}],"subject":[],"published":{"date-parts":[[2018,3,27]]}}}