{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T01:00:45Z","timestamp":1792803645515,"version":"4.5.0"},"reference-count":56,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2021,3,1]],"date-time":"2021-03-01T00:00:00Z","timestamp":1614556800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Korea Meteorological Institute","award":["KMI2018-01410"],"award-info":[{"award-number":["KMI2018-01410"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJERPH"],"abstract":"<jats:p>Many countries are operating a heatwave warning system (HWWS) to mitigate the impact of heatwaves on human health. The level of heatwave warning is normally determined by using the threshold temperature of heat-related morbidity or mortality. However, morbidity and mortality threshold temperatures have not been used together to account for the severity of health impacts. In this study, we developed a heatwave warning system with two different warning levels: Level-1 and Level-2, by analyzing the severity and likelihood of heat-related morbidity and mortality using the generalized additive model. The study particularly focuses on the cases in Seoul, South Korea, between 2011 and 2018. The study found that the threshold temperature for heat-related morbidity and mortality are 30 \u00b0C and 33 \u00b0C, respectively. Approximately 73.1% of heat-related patients visited hospitals when temperature was between 30 \u00b0C and 33 \u00b0C. We validated the developed HWWS by using both the threshold temperatures of morbidity and mortality. The area under curves (AUCs) of the proposed model were 0.74 and 0.86 at Level-1 and Level-2, respectively. On the other hand, the AUCs of the model using only the mortality threshold were 0.60 and 0.86 at Level-1 and Level-2, respectively. The AUCs of the model using only the morbidity threshold were 0.73 and 0.78 at Level-1 and Level-2, respectively. The results suggest that the updated HWWS can help to reduce the impact of heatwaves, particularly on vulnerable groups, by providing the customized information. This also indicates that the HWWS could effectively mitigate the risk of morbidity and mortality.<\/jats:p>","DOI":"10.3390\/ijerph18052380","type":"journal-article","created":{"date-parts":[[2021,3,1]],"date-time":"2021-03-01T10:25:18Z","timestamp":1614594318000},"page":"2380","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Analysis on Effectiveness of Impact Based Heatwave Warning Considering Severity and Likelihood of Health Impacts in Seoul, Korea"],"prefix":"10.3390","volume":"18","author":[{"given":"Yeora","family":"Chae","sequence":"first","affiliation":[{"name":"Korea Environment Institute, 370 Sicheong-daero, Sejong 30147, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jongchul","family":"Park","sequence":"additional","affiliation":[{"name":"Korea Environment Institute, 370 Sicheong-daero, Sejong 30147, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,1]]},"reference":[{"key":"ref_1","unstructured":"KMA (Korea Meteorological Administration) (2018). Comparison of heatwaves in 2018 and 1994 (Press release 2018.08.17.)."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1002\/ams2.387","article-title":"Severe heatwave in Japan","volume":"6","author":"Hayashida","year":"2019","journal-title":"Acute Med. Surg."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Li, M., Yao, Y., Luo, D., and Zhong, L. (2019). The Linkage of the Large-Scale Circulation Pattern to a Long-Lived Heatwave over Mideastern China in 2018. Atmosphere, 10.","DOI":"10.3390\/atmos10020089"},{"key":"ref_4","unstructured":"Bureau of Meteorology, AUS (2020, June 01). Climate Change\u2014Trends and Extremes, Available online: http:\/\/www.bom.gov.au\/climate\/change."},{"key":"ref_5","unstructured":"Opernicus and Climate Change Service (2020, March 03). Observed Temperature Extremes. Available online: https:\/\/surfobs.climate.copernicus.eu\/stateoftheclimate\/observed_extremes.php."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1324","DOI":"10.1038\/s41467-018-03732-9","article-title":"Longer and more frequent marine heatwaves over the past century","volume":"9","author":"Oliver","year":"2018","journal-title":"Nat. Commun."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1007\/s13143-017-0059-7","article-title":"Projected heat wave characteristics over the Korean Peninsula during the twenty-first Century","volume":"54","author":"Shin","year":"2018","journal-title":"Asia-Pac. J. Atmos. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Lee, J.Y., Kim, E., Lee, W.S., Chae, Y., and Kim, H. (2018). Projection of future mortality due to temperature and population changes under Representative Concentration Pathways and Shared Socioeconomic Pathways. Int. J. Environ. Res. Public Health, 15.","DOI":"10.3390\/ijerph15040822"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Park, J., Choi, S.H., and Chae, Y. (2019). Analysis of mortality change rate from temperature in summer by age, occupation, household type, and chronic diseases in 229 Korean municipalities from 2007 to 2016. Int. J. Environ. Res. Public Health, 16.","DOI":"10.3390\/ijerph16091561"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"109993","DOI":"10.1016\/j.envres.2020.109993","article-title":"The impact of heat waves on daily mortality in districts in Madrid: The effect of sociodemographic factors","volume":"190","author":"Julio","year":"2020","journal-title":"Environ. Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"614","DOI":"10.1016\/j.envres.2009.03.010","article-title":"Estimating the mortality effect of the July 2006 California heat wave","volume":"109","author":"Ostro","year":"2009","journal-title":"Environ. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1136\/jech.2008.085167","article-title":"A study of intracity variation of temperature-related mortality and socioeconomic status among the Chinese population in Hong Kong","volume":"66","author":"Chan","year":"2010","journal-title":"J. Epidemiol. Community Health"},{"key":"ref_13","unstructured":"Met Office (2020, June 01). Heat-Health Watch, Available online: https:\/\/www.metoffice.gov.uk\/public\/weather\/heat-health\/?tab=heatHealth&season=normal."},{"key":"ref_14","unstructured":"M\u00e9t\u00e9o-France (2020, June 01). Vigilance Meteo France, Available online: https:\/\/vigilance.meteofrance.fr\/fr."},{"key":"ref_15","unstructured":"NWS (2020, June 01). Heat Watch vs. Warning, Available online: https:\/\/www.weather.gov\/safety\/heat-ww."},{"key":"ref_16","unstructured":"PHU (2020, June 01). Heat Warning and Information System, Available online: https:\/\/www2.health.vic.gov.au\/public-health\/environmental-health\/climate-weather-and-public-health\/heatwaves-and-extreme-heat\/heat-health-alert-status."},{"key":"ref_17","unstructured":"HHS (2020, June 01). Heat-Health Related Alert System, Available online: http:\/\/www.health.gov.on.ca\/en\/common\/ministry\/publications\/reports\/heat_warning_information_system\/heat_warning_information_system.aspx."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4623","DOI":"10.3390\/ijerph8124623","article-title":"Heatwave early warning systems and adaptation advice to reduce human health consequences of heatwaves","volume":"8","author":"Lowe","year":"2011","journal-title":"Int. J. Environ. Res. Public Health"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1175\/BAMS-85-8-1067","article-title":"Heat watch\/warning systems save lives: Estimated costs and benefits for Philadelphia 1995\u201398","volume":"85","author":"Ebi","year":"2004","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1093\/ije\/dym253","article-title":"Has the impact of heat waves on mortality changed in France since the European heat wave of summer 2003? A study of the 2006 heat wave","volume":"37","author":"Fouillet","year":"2008","journal-title":"Int. J. Epidemiol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1016\/j.envres.2014.04.002","article-title":"The effects of summer temperature and heat waves on heat-related illness in a coastal city of China, 2011\u20132013","volume":"132","author":"Bai","year":"2014","journal-title":"Environ. Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"986","DOI":"10.1093\/aje\/kwm396","article-title":"Effect modification by community characteristics on the short-term effects of ozone exposure and mortality in 98 US communities","volume":"167","author":"Bell","year":"2008","journal-title":"Am. J. Epidemiol."},{"key":"ref_23","unstructured":"WMO (2013). WMO Risk Management Framework, Communications and Public Affairs Office."},{"key":"ref_24","unstructured":"PHE (2018). Heatwave Plan for England."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Casanueva, A., Burgstall, A., Kotlarski, S., Messeri, A., Morabito, M., Flouris, A.D., Nybo, L., Spirig, C., and Schwierz, C. (2019). Overview of existing heat-health warning systems in Europe. Int. J. Environ. Res. Public Health, 16.","DOI":"10.3390\/ijerph16152657"},{"key":"ref_26","unstructured":"Sant\u00e9 publique France (2020, July 01). Bulletin de sant\u00e9 publique. Available online: https:\/\/www.santepubliquefrance.fr\/determinants-de-sante\/climat\/fortes-chaleurs-canicule\/documents\/bulletin-national\/bulletin-de-sante-publique-canicule.-bilan-ete-2019."},{"key":"ref_27","unstructured":"Laaidi, A., Ung, A., Wagner, V., Beaudeau, P., and Pascal, M. (2013). The French Heat and Health Watch Warning System: Principles, Fundamentals and Assessment, French Institute for Public Health Surveillance, Department of Environmental Health."},{"key":"ref_28","unstructured":"Japan ME (Ministry of the Environment) (2020, March 03). HRI Prevention Information, Available online: www.wbgt.env.go.jp."},{"key":"ref_29","unstructured":"Williams, L., Erens, B., Ettelt, S., Hajat, S., Manacorda, T., and Mays, N. (2019). Evaluation of the Heatwave Plan for the UK: Final Report, Policy Innovation Evaluation Research Unit, London School of Hygiene and Tropical Medicine."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1016\/S0140-6736(14)62114-0","article-title":"Mortality risk attributable to high and low ambient temperature: A multicountry observational study","volume":"386","author":"Gasparrini","year":"2015","journal-title":"Lancet"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hajat, S., Kovats, R.S., Atkinson, R.W., and Haines, A. (2002). Impact of hot temperatures on death in London: A time series approach. J. Epidemiol. Community Health, 56.","DOI":"10.1136\/jech.56.5.367"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sun, X., Sun, Q., Yang, M., Zhou, X., Li, X., Yu, A., Geng, F., and Guo, Y. (2014). Effects of temperature and heat waves on emergency department visits and emergency ambulance dispatches in Pudong New Area, China: A time series analysis. Environ. Health, 13.","DOI":"10.1186\/1476-069X-13-76"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.envres.2014.07.007","article-title":"Temperature-related mortality in 17 large Chinese cities: How heat and cold affect mortality in China","volume":"134","author":"Ma","year":"2014","journal-title":"Environ. Res."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.envres.2013.03.009","article-title":"Ambient temperature and emergency department visits for heat-related illness in North Carolina, 2007\u20132008","volume":"124","author":"Lippmann","year":"2013","journal-title":"Environ. Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1016\/j.envres.2016.08.029","article-title":"Study on the association between ambient temperature and mortality using spatially resolved exposure data","volume":"151","author":"Lee","year":"2016","journal-title":"Environ. Res."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1050","DOI":"10.1038\/nclimate2022","article-title":"Attributing mortality from extreme temperatures to climate change in Stockholm, Sweden","volume":"3","author":"Forsberg","year":"2013","journal-title":"Nat. Clim. Chang."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"523","DOI":"10.5572\/KOSAE.2008.24.5.523","article-title":"A study on the influence of extreme heat on daily mortality","volume":"24","author":"Park","year":"2008","journal-title":"J. Korean Soc. Atmos. Environ."},{"key":"ref_38","first-page":"767","article-title":"Study on the establishment of threshold criteria for heat health watch warning system in Korea; Part I: Establishment of criteria and verification","volume":"18","author":"Park","year":"2009","journal-title":"J. Environ. Sci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"277","DOI":"10.5322\/JESI.2019.28.2.277","article-title":"Investigation of operation and improvement for heat watch","volume":"28","author":"Hwang","year":"2019","journal-title":"J. Environ. Sci. Int."},{"key":"ref_40","unstructured":"(2020, August 20). KMA. Available online: https:\/\/www.weather.go.kr\/weather\/warning\/heatwave.jsp."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.envint.2018.04.017","article-title":"Temporal changes in mortality impacts of heat wave and cold spell in Korea and Japan","volume":"116","author":"Lee","year":"2018","journal-title":"Environ. Int."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/j.healthplace.2018.08.017","article-title":"Heatwave and health impact research: A global review","volume":"53","author":"Campbell","year":"2018","journal-title":"Health Place"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1080\/09603123.2017.1379056","article-title":"Spatial variability in threshold temperatures of heat wave mortality: Impact assessment on prevention plans","volume":"27","author":"Carmona","year":"2017","journal-title":"Int. J. Environ. Health Res."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"7973519","DOI":"10.1155\/2018\/7973519","article-title":"Building resilience to climate change: Pilot evaluation of the impact of India\u2019s first heat action plan on all-cause mortality","volume":"2018","author":"Hess","year":"2018","journal-title":"J. Environ. Public Health"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.scitotenv.2006.08.014","article-title":"A vulnerability study of the low-income elderly in the context of high temperature and mortality in Seoul, Korea","volume":"371","author":"Kim","year":"2006","journal-title":"Sci. Total Environ."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"109027","DOI":"10.1016\/j.envres.2019.109027","article-title":"Heat wave intensity and daily mortality in four of the largest cities of Spain","volume":"182","author":"Codesido","year":"2020","journal-title":"Environ. Res."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"19","DOI":"10.35149\/jakpg.2019.29.4.002","article-title":"Comparison of statistical interpolation methods for downscaling of summer temperature data from the Korea Meteorological Administration\u2019s digital forecasts","volume":"29","author":"Yang","year":"2019","journal-title":"J. Assoc. Korean Photo Geogr."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2224","DOI":"10.1002\/sim.3940","article-title":"Distributed lag non-linear models","volume":"29","author":"Gasparrini","year":"2010","journal-title":"Stat. Med."},{"key":"ref_49","first-page":"245","article-title":"Study on the vulnerability regarding high temperature related mortality in Korea","volume":"49","author":"Jung","year":"2014","journal-title":"J. Korean Geogr. Soc."},{"key":"ref_50","unstructured":"(2020, April 28). FluMOMO v4.2. Available online: https:\/\/www.euromomo.eu\/how-it-works\/flumomo\/."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An introduction to ROC analysis","volume":"27","author":"Fawcett","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., and M\u00fcller, M. (2011). pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform., 12.","DOI":"10.1186\/1471-2105-12-77"},{"key":"ref_53","unstructured":"Nairn, J.R., and Fawcett, R.J. (2013). Defining Heatwaves: Heatwave Defined as a Heat-Impact Event Servicing All Community and Business Sectors in Australia, CAWCR Technical Report."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"227","DOI":"10.3390\/ijerph120100227","article-title":"Excess heat factor: A metric for heatwave intensity and its use in classifying heatwave severity","volume":"12","author":"Nairn","year":"2015","journal-title":"Int. J. Environ. Res. Public Health"},{"key":"ref_55","unstructured":"Department of Health (2020). Heat Health Alert System: Information for Local Government and Other Stakeholders."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1007\/s00484-005-0003-x","article-title":"France\u2019s heat health watch warning system","volume":"50","author":"Pascal","year":"2006","journal-title":"Int. J. Biometeorol."}],"container-title":["International Journal of Environmental Research and Public Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/5\/2380\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:30:54Z","timestamp":1760160654000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/5\/2380"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,1]]},"references-count":56,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["ijerph18052380"],"URL":"https:\/\/doi.org\/10.3390\/ijerph18052380","relation":{},"ISSN":["1660-4601"],"issn-type":[{"value":"1660-4601","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,1]]}}}