{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T09:52:29Z","timestamp":1761126749349,"version":"3.37.3"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T00:00:00Z","timestamp":1573171200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T00:00:00Z","timestamp":1573171200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n              <jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>The paper introduces a deep learning-based approach for real-time detection and insights generation about one of the most prevalent chronic conditions in Australia - Pollen allergy. The popular social media platform is used for data collection as cost-effective and unobtrusive alternative for public health monitoring to complement the traditional survey-based approaches.<\/jats:p>\n              <\/jats:sec>\n              <jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>The data was extracted from Twitter based on pre-defined keywords (i.e. \u2019hayfever\u2019 OR \u2019hay fever\u2019) throughout the period of 6 months, covering the high pollen season in Australia. The following deep learning architectures were adopted in the experiments: CNN, RNN, LSTM and GRU. Both default (GloVe) and domain-specific (HF) word embeddings were used in training the classifiers. Standard evaluation metrics (i.e. Accuracy, Precision and Recall) were calculated for the results validation. Finally, visual correlation with weather variables was performed.<\/jats:p>\n              <\/jats:sec>\n              <jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The neural networks-based approach was able to correctly identify the implicit mentions of the symptoms and treatments, even unseen previously (accuracy up to 87.9% for GRU with GloVe embeddings of 300 dimensions).<\/jats:p>\n              <\/jats:sec>\n              <jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The system addresses the shortcomings of the conventional machine learning techniques with manual feature-engineering that prove limiting when exposed to a wide range of non-standard expressions relating to medical concepts. The case-study presented demonstrates an application of \u2019black-box\u2019 approach to the real-world problem, along with its internal workings demonstration towards more transparent, interpretable and reproducible decision-making in health informatics domain.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-019-0921-x","type":"journal-article","created":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T00:04:07Z","timestamp":1573171447000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Deep learning for pollen allergy surveillance from twitter in Australia"],"prefix":"10.1186","volume":"19","author":[{"given":"Jia","family":"Rong","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3732-1882","authenticated-orcid":false,"given":"Sandra","family":"Michalska","sequence":"additional","affiliation":[]},{"given":"Sudha","family":"Subramani","sequence":"additional","affiliation":[]},{"given":"Jiahua","family":"Du","sequence":"additional","affiliation":[]},{"given":"Hua","family":"Wang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,11,8]]},"reference":[{"key":"921_CR1","unstructured":"Australian Institute of Health and Welfare (AIHW). Allergic rhinitis (\u2019hay fever\u2019). 2016. \n                    https:\/\/www.aihw.gov.au\/reports\/chronic-respiratory-conditions\/allergic-r%hinitis-hay-fever\/contents\/ allergic-rhinitis-by-the-numbers\n                    \n                  . Accessed 30 Jan 2019."},{"issue":"1","key":"921_CR2","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1093\/jamia\/ocx148","volume":"25","author":"M Vigo","year":"2017","unstructured":"Vigo M, Hassan L, Vance W, Jay C, Brass A, Cruickshank S. Britain breathing: using the experience sampling method to collect the seasonal allergy symptoms of a country. J Am Med Informa Assoc. 2017; 25(1):88\u201392.","journal-title":"J Am Med Informa Assoc"},{"issue":"1","key":"921_CR3","first-page":"1","volume":"8","author":"G D\u2019Amato","year":"2015","unstructured":"D\u2019Amato G, Holgate ST, Pawankar R, Ledford DK, Cecchi L, Al-Ahmad M, Al-Enezi F, Al-Muhsen S, Ansotegui I, Baena-Cagnani CE, et al. Meteorological conditions, climate change, new emerging factors, and asthma and related allergic disorders. a statement of the world allergy organization. World Allergy Org J. 2015; 8(1):1.","journal-title":"World Allergy Org J"},{"key":"921_CR4","volume-title":"International Conference on Smart Health","author":"L Xia","year":"2017","unstructured":"Xia L, Wang GA, Fan W. A deep learning based named entity recognition approach for adverse drug events identification and extraction in health social media. In: International Conference on Smart Health. Hong Kong: Springer: 2017. p. 237\u201348."},{"issue":"3","key":"921_CR5","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1093\/jamia\/ocu041","volume":"22","author":"A Nikfarjam","year":"2015","unstructured":"Nikfarjam A, Sarker A, O\u2019connor K, Ginn R, Gonzalez G. Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features. J Am Med Informa Assoc. 2015; 22(3):671\u201381.","journal-title":"J Am Med Informa Assoc"},{"key":"921_CR6","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1016\/j.jbi.2014.11.002","volume":"53","author":"A Sarker","year":"2015","unstructured":"Sarker A, Gonzalez G. Portable automatic text classification for adverse drug reaction detection via multi-corpus training. J Biomed Informa. 2015; 53:196\u2013207.","journal-title":"J Biomed Informa"},{"key":"921_CR7","first-page":"1","volume":"2014","author":"A Patki","year":"2014","unstructured":"Patki A, Sarker A, Pimpalkhute P, Nikfarjam A, Ginn R, O\u2019Connor K, Smith K, Gonzalez G. Mining adverse drug reaction signals from social media: going beyond extraction. Proc BioLinkSig. 2014; 2014:1\u20138.","journal-title":"Proc BioLinkSig"},{"key":"921_CR8","volume-title":"Proceedings of the Social Media Mining Shared Task Workshop at the Pacific Symposium on Biocomputing, Big Island, HI, USA","author":"J Jonnagaddala","year":"2016","unstructured":"Jonnagaddala J, Jue TR, Dai H-J. Binary classification of twitter posts for adverse drug reactions. In: Proceedings of the Social Media Mining Shared Task Workshop at the Pacific Symposium on Biocomputing, Big Island, HI, USA. Big Island, HI: PSB: 2016. p. 4\u20138."},{"issue":"3","key":"921_CR9","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1016\/j.ajic.2009.11.004","volume":"38","author":"D Scanfeld","year":"2010","unstructured":"Scanfeld D, Scanfeld V, Larson EL. Dissemination of health information through social networks: Twitter and antibiotics. Am J Infect Cont. 2010; 38(3):182\u20138.","journal-title":"Am J Infect Cont"},{"key":"921_CR10","volume-title":"Proceedings of the International Workshop on Software Engineering in Healthcare Systems","author":"K Byrd","year":"2016","unstructured":"Byrd K, Mansurov A, Baysal O. Mining twitter data for influenza detection and surveillance. In: Proceedings of the International Workshop on Software Engineering in Healthcare Systems. Austin: ACM: 2016. p. 43\u20139."},{"key":"921_CR11","volume-title":"Proceedings of the First Workshop on Social Media Analytics","author":"A Culotta","year":"2010","unstructured":"Culotta A. Towards detecting influenza epidemics by analyzing twitter messages. In: Proceedings of the First Workshop on Social Media Analytics. Washington DC: ACM: 2010. p. 115\u201322."},{"key":"921_CR12","volume-title":"Proceedings of the International Workshop on Digital Disease Detection Using Social Media 2017 (DDDSM-2017)","author":"C-K Wang","year":"2017","unstructured":"Wang C-K, Singh O, Tang Z-L, Dai H-J. Using a recurrent neural network model for classification of tweets conveyed influenza-related information. In: Proceedings of the International Workshop on Digital Disease Detection Using Social Media 2017 (DDDSM-2017). Taipei: Asian Federation of Natural Language Processing: 2017. p. 33\u201338."},{"key":"921_CR13","volume-title":"2015 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)","author":"K Lee","year":"2015","unstructured":"Lee K, Agrawal A, Choudhary A. Mining social media streams to improve public health allergy surveillance. In: 2015 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). Paris: IEEE: 2015. p. 815\u201322."},{"issue":"4","key":"921_CR14","first-page":"293","volume":"2","author":"E de Quincey","year":"2014","unstructured":"de Quincey E. Potential of social media to determine hay fever seasons and drug efficacy. Planet@ Risk. 2014; 2(4):293\u201397.","journal-title":"Planet@ Risk"},{"key":"921_CR15","volume-title":"Proceedings of the 6th International Conference on Digital Health Conference","author":"E de Quincey","year":"2016","unstructured":"de Quincey E, Kyriacou T, Pantin T. # hayfever; a longitudinal study into hay fever related tweets in the uk. In: Proceedings of the 6th International Conference on Digital Health Conference. Montreal: ACM: 2016. p. 85\u20139."},{"issue":"7","key":"921_CR16","doi-asserted-by":"publisher","first-page":"0133706","DOI":"10.1371\/journal.pone.0133706","volume":"10","author":"F Gesualdo","year":"2015","unstructured":"Gesualdo F, Stilo G, D\u2019Ambrosio A, Carloni E, Pandolfi E, Velardi P, Fiocchi A, Tozzi AE. Can twitter be a source of information on allergy? correlation of pollen counts with tweets reporting symptoms of allergic rhinoconjunctivitis and names of antihistamine drugs. PloS One. 2015; 10(7):0133706.","journal-title":"PloS One"},{"issue":"12","key":"921_CR17","doi-asserted-by":"publisher","first-page":"4434","DOI":"10.3390\/s18124434","volume":"18","author":"S Cowie","year":"2018","unstructured":"Cowie S, Arthur R, Williams H. @ choo: Tracking pollen and hayfever in the uk using social media. Sensors. 2018; 18(12):4434.","journal-title":"Sensors"},{"key":"921_CR18","volume-title":"Proceedings of the 2010 Workshop on Biomedical Natural Language Processing","author":"R Leaman","year":"2010","unstructured":"Leaman R, Wojtulewicz L, Sullivan R, Skariah A, Yang J, Gonzalez G. Towards internet-age pharmacovigilance: extracting adverse drug reactions from user posts to health-related social networks. In: Proceedings of the 2010 Workshop on Biomedical Natural Language Processing. Uppsala: Association for Computational Linguistics: 2010. p. 117\u201325."},{"issue":"4","key":"921_CR19","doi-asserted-by":"publisher","first-page":"267","DOI":"10.2165\/11590720-000000000-00000","volume":"34","author":"IR Edwards","year":"2011","unstructured":"Edwards IR, Lindquist M. Social media and networks in pharmacovigilance. Drug Saf. 2011; 34(4):267\u2013271.","journal-title":"Drug Saf"},{"issue":"Aug","key":"921_CR20","first-page":"2493","volume":"12","author":"R Collobert","year":"2011","unstructured":"Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P. Natural language processing (almost) from scratch. J Mach Learn Res. 2011; 12(Aug):2493\u2013537.","journal-title":"J Mach Learn Res"},{"key":"921_CR21","unstructured":"Australasian Society of Clinical Immunology and Allergy (ASCIA). Pollen allergy. 2017. \n                    https:\/\/www.allergy.org.au\/images\/pcc\/ASCIA_PCC_Pollen_allergy_2017.pdf\n                    \n                  . Accessed: 2019 Jan 30."},{"key":"921_CR22","unstructured":"World Allergy Organization (WAO). World Allergy Week 2016. 2016. \n                    https:\/\/www.worldallergy.org\/UserFiles\/file\/WorldAllergyWeek2016FactSheet%.pdf\n                    \n                  . Accessed: 2019 Jan 30."},{"issue":"10","key":"921_CR23","doi-asserted-by":"publisher","first-page":"4248","DOI":"10.1073\/pnas.1014107108","volume":"108","author":"L Ziska","year":"2011","unstructured":"Ziska L, Knowlton K, Rogers C, Dalan D, Tierney N, Elder MA, Filley W, Shropshire J, Ford LB, Hedberg C, et al. Recent warming by latitude associated with increased length of ragweed pollen season in central north america. Proc Nat Acad Sci. 2011; 108(10):4248\u201351.","journal-title":"Proc Nat Acad Sci"},{"key":"921_CR24","unstructured":"Australian Bureau of Statistics (ABS). Migration, Australia, 2014-15. 2016. \n                    http:\/\/www.abs.gov.au\/AUSSTATS\/abs@.nsf\/Lookup\/3412.0Main+Features12014-1%5?OpenDocument\n                    \n                  . Accessed: 2019 Jan 30."},{"issue":"1","key":"921_CR25","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1038\/s41533-018-0071-0","volume":"28","author":"B Cvetkovski","year":"2018","unstructured":"Cvetkovski B, Kritikos V, Yan K, Bosnic-Anticevich S. Tell me about your hay fever: a qualitative investigation of allergic rhinitis management from the perspective of the patient. NPJ Primary Care Respiratory Med. 2018; 28(1):3.","journal-title":"NPJ Primary Care Respiratory Med"},{"key":"921_CR26","unstructured":"Ginn R, Pimpalkhute P, Nikfarjam A, Patki A, O\u2019Connor K, Sarker A, Smith K, Gonzalez G. Mining twitter for adverse drug reaction mentions: a corpus and classification benchmark. In: Proceedings of the Fourth Workshop on Building and Evaluating Resources for Health and Biomedical Text Processing. Citeseer: 2014."},{"issue":"2","key":"921_CR27","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1037\/0003-066X.55.2.205","volume":"55","author":"KP Davison","year":"2000","unstructured":"Davison KP, Pennebaker JW, Dickerson SS. Who talks?the social psychology of illness support groups. Am Psych. 2000; 55(2):205.","journal-title":"Am Psych"},{"key":"921_CR28","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1016\/j.jbi.2014.03.005","volume":"49","author":"S Tuarob","year":"2014","unstructured":"Tuarob S, Tucker CS, Salathe M, Ram N. An ensemble heterogeneous classification methodology for discovering health-related knowledge in social media messages. J Biomed Informa. 2014; 49:255\u201368.","journal-title":"J Biomed Informa"},{"key":"921_CR29","volume-title":"International Conference on Health Information Science","author":"S Subramani","year":"2018","unstructured":"Subramani S, Michalska S, Wang H, Whittaker F, Heyward B. Text mining and real-time analytics of twitter data: A case study of australian hay fever prediction. In: International Conference on Health Information Science. Cairns: Springer: 2018. p. 134\u201345."},{"issue":"3","key":"921_CR30","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1093\/jamia\/ocx131","volume":"25","author":"S Gao","year":"2017","unstructured":"Gao S, Young MT, Qiu JX, Yoon H-J, Christian JB, Fearn PA, Tourassi GD, Ramanthan A. Hierarchical attention networks for information extraction from cancer pathology reports. J Am Med Informa Assoc. 2017; 25(3):321\u201330.","journal-title":"J Am Med Informa Assoc"},{"key":"921_CR31","volume-title":"Eleventh International AAAI Conference on Web and Social Media","author":"DT Nguyen","year":"2017","unstructured":"Nguyen DT, Al Mannai KA, Joty S, Sajjad H, Imran M, Mitra P. Robust classification of crisis-related data on social networks using convolutional neural networks. In: Eleventh International AAAI Conference on Web and Social Media. Montreal: AAAI: 2017."},{"issue":"2","key":"921_CR32","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1109\/MIS.2017.23","volume":"32","author":"N Majumder","year":"2017","unstructured":"Majumder N, Poria S, Gelbukh A, Cambria E. Deep learning-based document modeling for personality detection from text. IEEE Intell Syst. 2017; 32(2):74\u20139.","journal-title":"IEEE Intell Syst"},{"key":"921_CR33","unstructured":"Poria S, Cambria E, Hazarika D, Vij P. A deeper look into sarcastic tweets using deep convolutional neural networks. arXiv preprint arXiv:1610.08815. 2016."},{"key":"921_CR34","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.knosys.2016.06.009","volume":"108","author":"S Poria","year":"2016","unstructured":"Poria S, Cambria E, Gelbukh A. Aspect extraction for opinion mining with a deep convolutional neural network. Knowl-Based Syst. 2016; 108:42\u201349.","journal-title":"Knowl-Based Syst"},{"key":"921_CR35","volume-title":"2016 IEEE 16th International Conference on Data Mining (ICDM)","author":"S Poria","year":"2016","unstructured":"Poria S, Chaturvedi I, Cambria E, Hussain A. Convolutional mkl based multimodal emotion recognition and sentiment analysis. In: 2016 IEEE 16th International Conference on Data Mining (ICDM). Barcelona: IEEE: 2016. p. 439\u201348."},{"key":"921_CR36","volume-title":"Proceedings of International Conference on Neural Networks (ICNN\u201996), vol 1","author":"C Goller","year":"1996","unstructured":"Goller C, Kuchler A. Learning task-dependent distributed representations by backpropagation through structure. In: Proceedings of International Conference on Neural Networks (ICNN\u201996), vol 1. Washington DC: IEEE: 1996. p. 347\u201352."},{"key":"921_CR37","volume-title":"9th International Conference on Artificial Neural Networks: ICANN \u201999","author":"FA Gers","year":"1999","unstructured":"Gers FA, Schmidhuber J, Cummins F. Learning to forget: Continual prediction with lstm. In: 9th International Conference on Artificial Neural Networks: ICANN \u201999. Edinburgh: IET: 1999. p. 850\u201355."},{"key":"921_CR38","doi-asserted-by":"crossref","unstructured":"Cho K, Van Merri\u00ebnboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078. 2014.","DOI":"10.3115\/v1\/D14-1179"},{"key":"921_CR39","unstructured":"Chung J, Gulcehre C, Cho K, Bengio Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR. 2014; abs\/1412.3555. http:\/\/arxiv.org\/abs\/1412.3555. \n                    https:\/\/dblp.org\/rec\/bib\/journals\/corr\/ChungGCB14\n                    \n                  ."},{"issue":"8","key":"921_CR40","doi-asserted-by":"publisher","first-page":"1009","DOI":"10.2105\/AJPH.2018.304497","volume":"108","author":"JB Colditz","year":"2018","unstructured":"Colditz JB, Chu K-H, Emery SL, Larkin CR, James AE, Welling J, Primack BA. Toward real-time infoveillance of twitter health messages. Am J Publ Health. 2018; 108(8):1009\u201314.","journal-title":"Am J Publ Health"},{"issue":"2","key":"921_CR41","first-page":"249","volume":"22","author":"J Carletta","year":"1996","unstructured":"Carletta J. Assessing agreement on classification tasks: the kappa statistic. Comput Linguistics. 1996; 22(2):249\u201354.","journal-title":"Comput Linguistics"},{"issue":"5","key":"921_CR42","first-page":"360","volume":"37","author":"AJ Viera","year":"2005","unstructured":"Viera AJ, Garrett JM, et al. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005; 37(5):360\u20133.","journal-title":"Fam Med"},{"issue":"3","key":"921_CR43","doi-asserted-by":"publisher","first-page":"1166","DOI":"10.1016\/j.ipm.2018.04.011","volume":"56","author":"O Serban","year":"2019","unstructured":"Serban O, Thapen N, Maginnis B, Hankin C, Foot V. Real-time processing of social media with sentinel: a syndromic surveillance system incorporating deep learning for health classification. Inf Process Manag. 2019; 56(3):1166\u201384.","journal-title":"Inf Process Manag"},{"issue":"3","key":"921_CR44","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1016\/j.artmed.2014.01.002","volume":"61","author":"P Velardi","year":"2014","unstructured":"Velardi P, Stilo G, Tozzi AE, Gesualdo F. Twitter mining for fine-grained syndromic surveillance. Artif Intell Med. 2014; 61(3):153\u201363.","journal-title":"Artif Intell Med"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-019-0921-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12911-019-0921-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-019-0921-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,7]],"date-time":"2020-11-07T00:23:55Z","timestamp":1604708635000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-019-0921-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,8]]},"references-count":44,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["921"],"URL":"https:\/\/doi.org\/10.1186\/s12911-019-0921-x","relation":{},"ISSN":["1472-6947"],"issn-type":[{"type":"electronic","value":"1472-6947"}],"subject":[],"published":{"date-parts":[[2019,11,8]]},"assertion":[{"value":"17 March 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 September 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 November 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"This research is not human research and did not require IRB approval.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"208"}}