{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T12:12:10Z","timestamp":1781093530572,"version":"3.54.1"},"reference-count":64,"publisher":"Emerald","issue":"3","license":[{"start":{"date-parts":[[2016,8,8]],"date-time":"2016-08-08T00:00:00Z","timestamp":1470614400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JSIT"],"published-print":{"date-parts":[[2016,8,8]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title><jats:p>The purpose of this paper is to explore implicit crowdsourcing, leveraging social media in real-time scenarios for intelligent systems.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title><jats:p>A case study using an illustrative example system, which systematically used a custom social media platform for automated financial news analysis and summarisation was developed, evaluated and discussed. Literature review related to crowdsourcing and collective intelligence in intelligent systems was also conducted to provide context and to further explore the case study.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Findings<\/jats:title><jats:p>It was shown how, and that useful intelligent systems can be constructed from appropriately engineered custom social media platforms which are integrated with intelligent automated processes. A recent inter-rater agreement measure for evaluating quality of implicit crowd contributions was also explored and found to be of value.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Practical implications<\/jats:title><jats:p>This paper argues that when social media platforms are closely integrated with other automated processes into a single system, this may provide a highly worthwhile online and real-time approach to intelligent systems through implicit crowdsourcing. Key practical issues, such as achieving high-quality crowd contributions, challenges of efficient workflows and real-time crowd integration into intelligent systems, were discussed. Important ethical and related considerations were also covered.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title><jats:p>A contribution to existing theory was made by proposing how social media Web platforms may benefit crowdsourcing. As opposed to traditional crowdsourcing platforms, the presented approach and example system has a set of social elements that encourages implicit crowdsourcing. Instances of crowdsourcing with existing social media, such as Twitter, often also called crowd piggybacking, have been used in the past; however, using an entirely custom-built social media system for implicit crowdsourcing is relatively novel and has several advantages. Some of the discussion in context of intelligent systems construction are novel and contribute to the existing body of literature in this field.<\/jats:p><\/jats:sec>","DOI":"10.1108\/jsit-03-2016-0019","type":"journal-article","created":{"date-parts":[[2016,8,3]],"date-time":"2016-08-03T06:19:47Z","timestamp":1470205187000},"page":"255-276","source":"Crossref","is-referenced-by-count":5,"title":["Engineering social media driven intelligent systems through crowdsourcing"],"prefix":"10.1108","volume":"18","author":[{"given":"Martin","family":"Sykora","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"140","reference":[{"key":"key2020121400073909400_ref001","article-title":"Assigning trust to wikipedia content","year":"2008"},{"key":"key2020121400073909400_ref002","article-title":"Measuring author contributions to the Wikipedia","year":"2008"},{"key":"key2020121400073909400_ref003","article-title":"Finding high-quality content in social media","year":"2008"},{"issue":"307","key":"key2020121400073909400_ref004","first-page":"307","article-title":"A model of investor sentiment","volume":"49","year":"1998","journal-title":"Journal of Financial Economics"},{"key":"key2020121400073909400_ref005","article-title":"Crowdsourced real-world sensing: sentiment analysis and the real-time web","year":"2010"},{"issue":"2","key":"key2020121400073909400_ref006","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1108\/13287261211232144","article-title":"Using social media to sample ideas: lessons from a Slate-Twitter contest","volume":"14","year":"2012","journal-title":"Journal of Systems and Information Technology"},{"issue":"1","key":"key2020121400073909400_ref007","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1177\/1354856507084420","article-title":"Crowdsourcing as a model for problem solving: an introduction and cases","volume":"14","year":"2008","journal-title":"International Journal of Research into New Media Technologies"},{"key":"key2020121400073909400_ref008","article-title":"Creating speech and language data with Amazon\u2019s Mechanical Turk","year":"2010"},{"issue":"2","key":"key2020121400073909400_ref009","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1016\/S0304-405X(03)00146-6","article-title":"Stock price reaction to news and no-news: drift and reversal after headlines","volume":"70","year":"2003","journal-title":"Journal of Financial Economics"},{"issue":"2","key":"key2020121400073909400_ref010","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1108\/13287261211232135","article-title":"Factors determining the flow of information among the online community users","volume":"14","year":"2012","journal-title":"Journal of Systems and Information Technology"},{"issue":"1","key":"key2020121400073909400_ref011","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","year":"1960","journal-title":"Educational and Psychological Measurement"},{"issue":"3","key":"key2020121400073909400_ref012","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/BF02310555","article-title":"Coefficient alpha and the internal structure of tests","volume":"16","year":"1951","journal-title":"Psychometrica"},{"key":"key2020121400073909400_ref013","first-page":"1745","article-title":"Working within a black box: transparency in the collection and production of Big Twitter data","volume":"8","year":"2014","journal-title":"International Journal of Communication"},{"issue":"4","key":"key2020121400073909400_ref014","doi-asserted-by":"crossref","first-page":"1161","DOI":"10.1111\/j.1540-6261.1993.tb04750.x","article-title":"How markets process information: news releases and volatility","volume":"48","year":"1993","journal-title":"Journal of Finance"},{"issue":"2","key":"key2020121400073909400_ref015","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1177\/0165551512437638","article-title":"Towards an integrated crowdsourcing definition","volume":"38","year":"2012","journal-title":"Journal of Information Science"},{"issue":"5","key":"key2020121400073909400_ref016","first-page":"55","article-title":"Random walks in stock market prices","volume":"51","year":"1965","journal-title":"Financial Analysts Journal"},{"key":"key2020121400073909400_ref017","article-title":"Annotating named entities in Twitter data with crowdsourcing","year":"2010"},{"issue":"1","key":"key2020121400073909400_ref018","first-page":"378","article-title":"Measuring nominal scale agreement among many raters","volume":"76","year":"1971","journal-title":"Psychological Bulletin"},{"key":"key2020121400073909400_ref019","article-title":"Why do people write for Wikipedia? Incentives to contribute to open-content publishing","year":"2005"},{"issue":"1","key":"key2020121400073909400_ref020","first-page":"1","article-title":"The predicting power of textual information on financial markets","volume":"5","year":"2005","journal-title":"IEEE Intelligent Informatics Bulletin"},{"key":"key2020121400073909400_ref021","doi-asserted-by":"crossref","first-page":"900","DOI":"10.1038\/438900a","article-title":"Internet encyclopaedias go head to head","volume":"438","year":"2005","journal-title":"Nature"},{"key":"key2020121400073909400_ref022","article-title":"Piggyback prototyping: using existing, large-scale social computing systems to prototype new ones","year":"2015"},{"issue":"11","key":"key2020121400073909400_ref023","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1109\/2.881692","article-title":"The challenges of automatic summarization","volume":"33","year":"2000","journal-title":"Computer"},{"issue":"1","key":"key2020121400073909400_ref024","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1080\/19312450709336664","article-title":"Answering the call for a standard reliability measure for coding data","volume":"1","year":"2007","journal-title":"Communication Methods and Measures"},{"key":"key2020121400073909400_ref025","volume-title":"G\u00f6del, Escher, Bach: An Eternal Golden Braid","year":"1979"},{"key":"key2020121400073909400_ref026","unstructured":"Howe, J. (2006), \u201cCrowdsourcing: a definition\u201d, available at: http:\/\/crowdsourcing.typepad.com\/cs\/2006\/06\/crowdsourcing_a.html (accessed 20 January 2015)."},{"issue":"1","key":"key2020121400073909400_ref027","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1108\/JSIT-04-2015-0031","article-title":"Achieving adoption and effective usage of web 2.0 among employees within Australian Government Organizations","volume":"18","year":"2016","journal-title":"Journal of Systems and Information Technology"},{"key":"key2020121400073909400_ref028","unstructured":"Kambil, A. (2003), \u201cYou can bet on idea markets\u201d, available at: http:\/\/hbswk.hbs.edu\/archive\/3808.html (accessed 1 April 2011)."},{"key":"key2020121400073909400_ref029","volume-title":"Making Markets: How Firms Can Design and Profit from Online Auctions and Exchanges","year":"2002"},{"key":"key2020121400073909400_ref030","unstructured":"Kanefsky, B., Barlow, N.G. and Gulick, V.C. (2001), \u201cCan distributed volunteers accomplish massive data analysis tasks?\u201d, Technical Report, available at: http:\/\/bit.ly\/9mtG1i (accessed 1 August 2010)."},{"key":"key2020121400073909400_ref031","volume-title":"The Cult of the Amateur: How the Democratization of the Digital World is Assaulting Our Economy, Our Culture, and Our Values","year":"2007"},{"key":"key2020121400073909400_ref032","first-page":"297","article-title":"Computing attitude and affect in text: theory and applications","volume-title":"Good News or Bad News? Let the Market Decide","year":"2006"},{"key":"key2020121400073909400_ref033","volume-title":"Content Analysis: An Introduction to its Methodology","year":"2004"},{"issue":"2","key":"key2020121400073909400_ref034","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1215942.1215943","article-title":"Motivation of Contributors to Wikipedia","volume":"36","year":"2006","journal-title":"ACM SIGCAS Journal of Computers and Society"},{"key":"key2020121400073909400_ref035","article-title":"Crowd-powered intelligent systems","year":"2014","journal-title":"Human Computation Journal"},{"key":"key2020121400073909400_ref036","volume-title":"L\u2019intelligence collective. Pour une anthropologie du cyberspace","year":"1994"},{"key":"key2020121400073909400_ref037","article-title":"Heuristics for supporting social interaction in online communities","year":"2009"},{"issue":"1","key":"key2020121400073909400_ref038","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3758\/s13428-011-0124-6","article-title":"Conducting behavioral research on Amazon\u2019s Mechanical Turk","volume":"44","year":"2012","journal-title":"Behavior Research Methods"},{"key":"key2020121400073909400_ref039","unstructured":"Mittermayer, M.A. and Knolmayer, G. (2006), \u201cText mining systems for market response to news: a survey\u201d, Working Paper, Bern University, available at: www2.ie.iwi.unibe.ch\/publikationen\/berichte\/resource\/WP-184.pdf"},{"key":"key2020121400073909400_ref040","unstructured":"Nagar, Y. and Malone, T. (2011), \u201cCombining human and machine intelligence for making predictions\u201d, CCI Working Paper No 2011-02, available at: http:\/\/cci.mit.edu\/publications\/CCIwp2011-02.pdf"},{"key":"key2020121400073909400_ref041","article-title":"Real-time detection, tracking, and monitoring of automatically discovered events in social media","year":"2014"},{"issue":"1\/2","key":"key2020121400073909400_ref042","first-page":"1","article-title":"Opinion mining and sentiment analysis","volume":"2","year":"2008","journal-title":"Foundations and Trends in Information Retrieval"},{"issue":"5","key":"key2020121400073909400_ref043","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1017\/S1930297500002205","article-title":"Running experiments on Amazon Mechanical Turk","volume":"5","year":"2010","journal-title":"Journal of Judgment and Decision Making"},{"issue":"3","key":"key2020121400073909400_ref044","article-title":"Loser generated content: from participation to exploitation","volume":"13","year":"2008","journal-title":"Internet Journal - First Monday"},{"key":"key2020121400073909400_ref045","volume-title":"A Computational Model of Crowds for Collective Intelligence, Collective Intelligence 2014","year":"2014"},{"key":"key2020121400073909400_ref046","first-page":"43","article-title":"The politics of Twitter data","volume-title":"Twitter and Society","year":"2014"},{"key":"key2020121400073909400_ref047","article-title":"A real-time social media aggregation tool: reflections from five large-scale events","year":"2011"},{"key":"key2020121400073909400_ref048","volume-title":"The Global Brain Awakens","year":"1983"},{"issue":"3","key":"key2020121400073909400_ref049","article-title":"Market ideology and the myths of Web 2.0","volume":"13","year":"2008","journal-title":"Internet Journal \u2013 First Monday"},{"key":"key2020121400073909400_ref050","volume-title":"Market Wizards: Interviews with Top Traders","year":"1993"},{"issue":"3","key":"key2020121400073909400_ref051","first-page":"243","article-title":"Prediction market, does money matter?","volume":"4","year":"2004","journal-title":"Electronic Markets"},{"key":"key2020121400073909400_ref052","volume-title":"Cognitive Surplus: Creativity and Generosity in a Connected Age","year":"2010"},{"key":"key2020121400073909400_ref053","article-title":"Power of Web 2.0 mass collaboration in computational intelligence and its\u2019 uses, an example from Finance","year":"2009"},{"key":"key2020121400073909400_ref054","volume-title":"Neural Networks and the Financial Markets: Predicting, Combining and Portfolio Optimisation","year":"2002"},{"issue":"4","key":"key2020121400073909400_ref055","doi-asserted-by":"crossref","first-page":"402","DOI":"10.1177\/0894439310386557","article-title":"Election forecasts with Twitter: how 140 characters reflect the political landscape","volume":"29","year":"2011","journal-title":"Social Science Computer Review"},{"key":"key2020121400073909400_ref056","unstructured":"Ushahidi (2015), \u201cYour tools for changing the world\u201d, available at: www.ushahidi.com\/product\/ushahidi\/ (accessed 21 January 2015)."},{"issue":"2","key":"key2020121400073909400_ref057","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1108\/13287261211232171","article-title":"Exploring uses of social media in a global corporation","volume":"14","year":"2012","journal-title":"Journal of Systems and Information Technology"},{"key":"key2020121400073909400_ref058","volume-title":"World Brain: The Idea of A Permanent World Encyclopedia","year":"1994"},{"key":"key2020121400073909400_ref059","unstructured":"Westwood, S.J. and Messing, S. (2011), \u201cSelective exposure in the age of social media: endorsements trump partisan source affiliation when selecting Online News Media\u201d, Working Paper, Stanford University, available at: http:\/\/stanford.edu\/\u223cseanjw\/papers\/socialNews.pdf"},{"key":"key2020121400073909400_ref061","article-title":"Daily stock market forecast from textual web data systems","year":"1998"},{"key":"key2020121400073909400_ref062","volume-title":"The Handbook of Data Mining","year":"2003"},{"key":"key2020121400073909400_ref063","unstructured":"Zemke, S. (2003), \u201cData mining for prediction: financial series case\u201d, PhD thesis, The Royal Institute of Technology, Department of Computer and Systems Sciences, Sweden."},{"key":"key2020121400073909400_ref064","volume-title":"Toward A Model of Literary Comprehension, Models of Understanding Text","year":"1993"},{"issue":"1","key":"key2020121400073909400_ref065","first-page":"189","article-title":"Using geolocated Twitter data to monitor the prevalence of healthy and unhealthy food references across the US","volume":"54","year":"2014","journal-title":"Applied Geography"}],"container-title":["Journal of Systems and Information Technology"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/JSIT-03-2016-0019","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/JSIT-03-2016-0019\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/JSIT-03-2016-0019\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T22:23:41Z","timestamp":1753395821000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jsit\/article\/18\/3\/255-276\/248421"}},"subtitle":["Insights from a financial news summarisation system"],"short-title":[],"issued":{"date-parts":[[2016,8,8]]},"references-count":64,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2016,8,8]]}},"alternative-id":["10.1108\/JSIT-03-2016-0019"],"URL":"https:\/\/doi.org\/10.1108\/jsit-03-2016-0019","relation":{},"ISSN":["1328-7265"],"issn-type":[{"value":"1328-7265","type":"print"}],"subject":[],"published":{"date-parts":[[2016,8,8]]}}}