{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T15:48:19Z","timestamp":1784648899655,"version":"3.55.0"},"reference-count":40,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2023,5,27]],"date-time":"2023-05-27T00:00:00Z","timestamp":1685145600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSF","award":["1541349"],"award-info":[{"award-number":["1541349"]}]},{"name":"NSF","award":["2100237"],"award-info":[{"award-number":["2100237"]}]},{"name":"NSF","award":["2120019"],"award-info":[{"award-number":["2120019"]}]},{"name":"NSF","award":["1826967"],"award-info":[{"award-number":["1826967"]}]},{"name":"NSF","award":["2112167"],"award-info":[{"award-number":["2112167"]}]},{"name":"SDG&amp;E","award":["1541349"],"award-info":[{"award-number":["1541349"]}]},{"name":"SDG&amp;E","award":["2100237"],"award-info":[{"award-number":["2100237"]}]},{"name":"SDG&amp;E","award":["2120019"],"award-info":[{"award-number":["2120019"]}]},{"name":"SDG&amp;E","award":["1826967"],"award-info":[{"award-number":["1826967"]}]},{"name":"SDG&amp;E","award":["2112167"],"award-info":[{"award-number":["2112167"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Research has shown that climate change creates warmer temperatures and drier conditions, leading to longer wildfire seasons and increased wildfire risks in the United States. These factors have, in turn, led to increases in the frequency, extent, and severity of wildfires in recent years. Given the danger posed by wildland fires to people, property, wildlife, and the environment, there is an urgent need to provide tools for effective wildfire management. Early detection of wildfires is essential to minimizing potentially catastrophic destruction. To that end, in this paper, we present our work on integrating multiple data sources into SmokeyNet, a deep learning model using spatiotemporal information to detect smoke from wildland fires. We present Multimodal SmokeyNet and SmokeyNet Ensemble for multimodal wildland fire smoke detection using satellite-based fire detections, weather sensor measurements, and optical camera images. An analysis is provided to compare these multimodal approaches to the baseline SmokeyNet in terms of accuracy metrics, as well as time-to-detect, which is important for the early detection of wildfires. Our results show that incorporating weather data in SmokeyNet improves performance numerically in terms of both F1 and time-to-detect over the baseline with a single data source. With a time-to-detect of only a few minutes, SmokeyNet can be used for automated early notification of wildfires, providing a useful tool in the fight against destructive wildfires.<\/jats:p>","DOI":"10.3390\/rs15112790","type":"journal-article","created":{"date-parts":[[2023,5,27]],"date-time":"2023-05-27T16:17:33Z","timestamp":1685204253000},"page":"2790","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["Multimodal Wildland Fire Smoke Detection"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-2494-2027","authenticated-orcid":false,"given":"Jaspreet Kaur","family":"Bhamra","sequence":"first","affiliation":[{"name":"Computer Science & Engineering, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shreyas","family":"Anantha Ramaprasad","sequence":"additional","affiliation":[{"name":"Computer Science & Engineering, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1104-0748","authenticated-orcid":false,"given":"Siddhant","family":"Baldota","sequence":"additional","affiliation":[{"name":"Computer Science & Engineering, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shane","family":"Luna","sequence":"additional","affiliation":[{"name":"Jacobs School of Engineering, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eugene","family":"Zen","sequence":"additional","affiliation":[{"name":"Jacobs School of Engineering, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ravi","family":"Ramachandra","sequence":"additional","affiliation":[{"name":"Jacobs School of Engineering, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Harrison","family":"Kim","sequence":"additional","affiliation":[{"name":"Jacobs School of Engineering, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7230-4482","authenticated-orcid":false,"given":"Chris","family":"Schmidt","sequence":"additional","affiliation":[{"name":"Space Science and Engineering Center, University of Wisconsin-Madison, Madison, WI 53706, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chris","family":"Arends","sequence":"additional","affiliation":[{"name":"San Diego Gas & Electric, San Diego, CA 92123, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jessica","family":"Block","sequence":"additional","affiliation":[{"name":"San Diego Supercomputer Center, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ismael","family":"Perez","sequence":"additional","affiliation":[{"name":"San Diego Supercomputer Center, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Crawl","sequence":"additional","affiliation":[{"name":"San Diego Supercomputer Center, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ilkay","family":"Altintas","sequence":"additional","affiliation":[{"name":"San Diego Supercomputer Center, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7538-1715","authenticated-orcid":false,"given":"Garrison W.","family":"Cottrell","sequence":"additional","affiliation":[{"name":"Computer Science & Engineering, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1444-648X","authenticated-orcid":false,"given":"Mai H.","family":"Nguyen","sequence":"additional","affiliation":[{"name":"San Diego Supercomputer Center, University of California San Diego, La Jolla, CA 92093, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,27]]},"reference":[{"key":"ref_1","unstructured":"Reidmiller, D.R., Avery, C.W., Easterling, D.R., Kunkel, K.E., Lewis, K.L., Maycock, T.K., and Stewart, B.C. (2023, March 15). Impacts, Risks, and Adaptation in the United States: Fourth National Climate Assessment, Available online: https:\/\/repository.library.noaa.gov\/view\/noaa\/19487;."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"20150178","DOI":"10.1098\/rstb.2015.0178","article-title":"Increasing western US forest wildfire activity: Sensitivity to changes in the timing of spring","volume":"371","author":"Westerling","year":"2016","journal-title":"Philos. Trans. R. Soc. Biol. Sci."},{"key":"ref_3","unstructured":"Agency, U.S.E.P. (2023, March 15). Climate Change Indicators: Wildfires, Available online: https:\/\/www.epa.gov\/climate-indicators\/climate-change-indicators-wildfires."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wuebbles, D., Fahey, D., Hibbard, K., Kokken, D., Stewart, B., and Maycock, T. (2023, March 15). Climate Science Special Report: Fourth National Climate Assessment, Available online: https:\/\/science2017.globalchange.gov\/.","DOI":"10.7930\/J0J964J6"},{"key":"ref_5","unstructured":"(2023, March 15). National Oceanic and Atmospheric Administration (NOAA) Billion-Dollar Weather and Climate Disasters, Available online: https:\/\/www.ncei.noaa.gov\/access\/billions."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"11770","DOI":"10.1073\/pnas.1607171113","article-title":"Impact of anthropogenic climate change on wildfire across western US forests","volume":"113","author":"Abatzoglou","year":"2016","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Dewangan, A., Pande, Y., Braun, H.W., Vernon, F., Perez, I., Altintas, I., Cottrell, G.W., and Nguyen, M.H. (2022). FIgLib & SmokeyNet: Dataset and Deep Learning Model for Real-Time Wildland Fire Smoke Detection. Remote Sens., 14.","DOI":"10.3390\/rs14041007"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Park, M., Tran, D.Q., Jung, D., and Park, S. (2020). Wildfire-Detection Method Using DenseNet and CycleGAN Data Augmentation-Based Remote Camera Imagery. Remote Sens., 12.","DOI":"10.3390\/rs12223715"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"115125","DOI":"10.1016\/j.eswa.2021.115125","article-title":"DeepSmoke: Deep learning model for smoke detection and segmentation in outdoor environments","volume":"182","author":"Khan","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"154732","DOI":"10.1109\/ACCESS.2019.2946712","article-title":"An Attention Enhanced Bidirectional LSTM for Early Forest Fire Smoke Recognition","volume":"7","author":"Cao","year":"2019","journal-title":"IEEE Access"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1134\/S1054661818040168","article-title":"Detection of Wildfires along Transmission Lines Using Deep Time and Space Features","volume":"28","author":"Yuan","year":"2018","journal-title":"Pattern Recognit. Image Anal."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Jeong, M., Park, M., Nam, J., and Ko, B.C. (2020). Light-Weight Student LSTM for Real-Time Wildfire Smoke Detection. Sensors, 20.","DOI":"10.3390\/s20195508"},{"key":"ref_13","unstructured":"Jindal, P., Gupta, H., Pachauri, N., Sharma, V., and Verma, O.P. (2021). Soft Computing: Theories and Applications, Springer."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2419","DOI":"10.1007\/s10694-019-00872-2","article-title":"Dual Deep Learning Model for Image Based Smoke Detection","volume":"55","author":"Pundir","year":"2019","journal-title":"Fire Technol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gupta, T., Liu, H., and Bhanu, B. (2021, January 10\u201315). Early Wildfire Smoke Detection in Videos. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR-20), Milan, Italy.","DOI":"10.1109\/ICPR48806.2021.9413231"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"017208","DOI":"10.1117\/1.OE.51.1.017208","article-title":"Wildfire smoke detection using temporospatial features and random forest classifiers","volume":"51","author":"Ko","year":"2012","journal-title":"Opt. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, T., Zhao, E., Zhang, J., and Hu, C. (2019). Detection of Wildfire Smoke Images Based on a Densely Dilated Convolutional Network. Electronics, 8.","DOI":"10.3390\/electronics8101131"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1007\/s11042-017-5561-5","article-title":"Recurrent convolutional network for video-based smoke detection","volume":"78","author":"Yin","year":"2019","journal-title":"Multimed. Tools Appl."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1016\/j.proeng.2017.12.034","article-title":"Wildland Forest Fire Smoke Detection Based on Faster R-CNN using Synthetic Smoke Images","volume":"211","author":"Zhang","year":"2018","journal-title":"Procedia Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1016\/j.neucom.2019.05.011","article-title":"Deep smoke segmentation","volume":"357","author":"Yuan","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Govil, K., Welch, M.L., Ball, J.T., and Pennypacker, C.R. (2020). Preliminary Results from a Wildfire Detection System Using Deep Learning on Remote Camera Images. Remote Sens., 12.","DOI":"10.3390\/rs12010166"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"105029","DOI":"10.1016\/j.compag.2019.105029","article-title":"Real-time forest smoke detection using hand-designed features and deep learning","volume":"167","author":"Peng","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_23","first-page":"5522","article-title":"Smoke detection in video sequences based on dynamic texture using volume local binary patterns","volume":"11","author":"Lin","year":"2017","journal-title":"KSII Trans. Internet Inf. Syst. TIIS"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Mukhiddinov, M., Abdusalomov, A.B., and Cho, J. (2022). A Wildfire Smoke Detection System Using Unmanned Aerial Vehicle Images Based on the Optimized YOLOv5. Sensors, 22.","DOI":"10.3390\/s22239384"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"da Penha, O.S., and Nakamura, E.F. (2010, January 22\u201325). Fusing light and temperature data for fire detection. Proceedings of the IEEE Symposium on Computers and Communications, Riccione, Italy.","DOI":"10.1109\/ISCC.2010.5546519"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.protcy.2012.03.008","article-title":"Wireless sensor networks and fusion information methods for forest fire detection","volume":"3","author":"Tafoya","year":"2012","journal-title":"Procedia Technol."},{"key":"ref_27","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Advances in Neural Information Processing Systems (NeurIPS-12), MIT Press."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_29","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 3\u20137). An Image is Worth 16 \u00d7 16 Words: Transformers for Image Recognition at Scale. Proceedings of the International Conference on Learning Representations (ICLR-21), Virtual."},{"key":"ref_30","unstructured":"San Diego Supercomputer Center, and Scripps Institution of Oceanography (2023, March 15). The High Performance Wireless Research and Education Network. Available online: http:\/\/hpwren.ucsd.edu\/."},{"key":"ref_31","unstructured":"San Diego Supercomputer Center, and Scripps Institution of Oceanography (2023, March 15). HPWREN Weather Readings. Available online: http:\/\/hpwren.ucsd.edu\/Sensors\/."},{"key":"ref_32","unstructured":"San Diego Gas & Electric (2023, March 15). SDG&E Weather Awareness System. Available online: https:\/\/weather.sdgeweather.com\/."},{"key":"ref_33","unstructured":"Southern California Edison (2023, March 15). Weather and Fire Detection. Available online: https:\/\/www.sce.com\/wildfire\/situational-awareness."},{"key":"ref_34","unstructured":"Synoptic Data (2023, March 15). Mesonet API. Available online: https:\/\/developers.synopticdata.com\/mesonet\/."},{"key":"ref_35","unstructured":"U.S. Department of Commerce (2023, March 15). National Oceanic and Atmospheric Administration, Available online: https:\/\/https:\/\/www.noaa.gov\/\/."},{"key":"ref_36","unstructured":"(2023, March 15). Geostationary Operational Environmental Satellites\u2014R Series, Available online: https:\/\/www.goes-r.gov\/."},{"key":"ref_37","unstructured":"Losos, D. (2023, March 15). Geostationary Operational Environmental Satellites\u2014R Series, Available online: https:\/\/www.goes-r.gov\/downloads\/resources\/documents\/Beginners_Guide_to_GOES-R_Series_Data.pdf."},{"key":"ref_38","unstructured":"Schmidt, C.C., Hoffman, J., Prins, E., and Lindstrom, S. (2023, March 15). GOES-R Advanced Baseline Imager (ABI) Algorithm Theoretical Basis Document For Fire\/Hot Spot Characterization, Available online: https:\/\/www.star.nesdis.noaa.gov\/goesr\/documents\/ATBDs\/Baseline\/ATBD_GOES-R_FIRE_v2.6_Oct2013.pdf."},{"key":"ref_39","unstructured":"(2023, March 15). Wildfire Automated Biomass Burning Algorithm. Available online: http:\/\/wfabba.ssec.wisc.edu\/index.html."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/11\/2790\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:43:21Z","timestamp":1760125401000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/11\/2790"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,27]]},"references-count":40,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["rs15112790"],"URL":"https:\/\/doi.org\/10.3390\/rs15112790","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,27]]}}}