{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,6]],"date-time":"2026-07-06T22:33:39Z","timestamp":1783377219043,"version":"3.54.6"},"reference-count":42,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2020,4,4]],"date-time":"2020-04-04T00:00:00Z","timestamp":1585958400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2018R1C1B5084417"],"award-info":[{"award-number":["2018R1C1B5084417"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002553","name":"Seoul National University of Science and Technology","doi-asserted-by":"publisher","award":["Research Program"],"award-info":[{"award-number":["Research Program"]}],"id":[{"id":"10.13039\/501100002553","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Driving is a task that puts heavy demands on visual information, thereby the human visual system plays a critical role in making proper decisions for safe driving. Understanding a driver\u2019s visual attention and relevant behavior information is a challenging but essential task in advanced driver-assistance systems (ADAS) and efficient autonomous vehicles (AV). Specifically, robust prediction of a driver\u2019s attention from images could be a crucial key to assist intelligent vehicle systems where a self-driving car is required to move safely interacting with the surrounding environment. Thus, in this paper, we investigate a human driver\u2019s visual behavior in terms of computer vision to estimate the driver\u2019s attention locations in images. First, we show that feature representations at high resolution improves visual attention prediction accuracy and localization performance when being fused with features at low-resolution. To demonstrate this, we employ a deep convolutional neural network framework that learns and extracts feature representations at multiple resolutions. In particular, the network maintains the feature representation with the highest resolution at the original image resolution. Second, attention prediction tends to be biased toward centers of images when neural networks are trained using typical visual attention datasets. To avoid overfitting to the center-biased solution, the network is trained using diverse regions of images. Finally, the experimental results verify that our proposed framework improves the prediction accuracy of a driver\u2019s attention locations.<\/jats:p>","DOI":"10.3390\/s20072030","type":"journal-article","created":{"date-parts":[[2020,4,7]],"date-time":"2020-04-07T03:58:39Z","timestamp":1586231919000},"page":"2030","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["High-Resolution Neural Network for Driver Visual Attention Prediction"],"prefix":"10.3390","volume":"20","author":[{"given":"Byeongkeun","family":"Kang","sequence":"first","affiliation":[{"name":"Department of Electronic and IT Media Engineering, Seoul National University of Science and Technology, Seoul 139-743, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yeejin","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Electrical and Information Engineering, Seoul National University of Science and Technology, Seoul 139-743, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1016\/j.patcog.2016.08.029","article-title":"Are all objects equal? Deep spatio-temporal importance prediction in driving videos","volume":"64","author":"Trivedi","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"5424","DOI":"10.1109\/TVT.2015.2487826","article-title":"How Much of Driving Is Preattentive?","volume":"64","author":"Pugeault","year":"2015","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kim, I.H., Bong, J.H., Park, J., and Park, S. (2017). Prediction of driver\u2019s intention of lane change by augmenting sensor information using machine learning techniques. Sensors, 17.","DOI":"10.3390\/s17061350"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1038\/35058500","article-title":"Computational modelling of visual attention","volume":"2","author":"Itti","year":"2001","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2348","DOI":"10.1016\/j.visres.2010.05.021","article-title":"Vision and driving","volume":"50","author":"Owsley","year":"2010","journal-title":"Vis. Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1167\/17.1.6","article-title":"Scene perception from central to peripheral vision","volume":"17","author":"Loschky","year":"2017","journal-title":"J. Vis."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"De Lumen, A.B., Lim, R.A., Orosa, K., Paralleon, D., and Sedilla, K. (2018). Driver Distraction: Determining the Ideal Location of a Navigation Device for Transportation Network Vehicle Services (TNVS) Drivers in Metro Manila. International Conference on Applied Human Factors and Ergonomics, Springer.","DOI":"10.1007\/978-3-319-93885-1_50"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"802","DOI":"10.1109\/TIP.2011.2162740","article-title":"Combining Head Pose and Eye Location Information for Gaze Estimation","volume":"21","author":"Valenti","year":"2012","journal-title":"IEEE Trans. Image Process."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1109\/MIS.2016.47","article-title":"Driver gaze region estimation without use of eye movement","volume":"31","author":"Fridman","year":"2016","journal-title":"IEEE Intell. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tawari, A., and Kang, B. (2017, January 11\u201314). A computational framework for driver\u2019s visual attention using a fully convolutional architecture. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995828"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1720","DOI":"10.1109\/TPAMI.2018.2845370","article-title":"Predicting the Driver\u2019s Focus of Attention: The DR (eye) VE Project","volume":"41","author":"Palazzi","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_12","unstructured":"Xia, Y., Zhang, D., Kim, J., Nakayama, K., Zipser, K., and Whitney, D. (2018). Predicting driver attention in critical situations. Asian Conference on Computer Vision, Springer."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chang, S., Zhang, Y., Zhang, F., Zhao, X., Huang, S., Feng, Z., and Wei, Z. (2020). Spatial Attention Fusion for Obstacle Detection Using MmWave Radar and Vision Sensor. Sensors, 20.","DOI":"10.3390\/s20040956"},{"key":"ref_14","unstructured":"Bylinskii, Z., Judd, T., Borji, A., Itti, L., Durand, F., Oliva, A., and Torralba, A. (2015). MIT Saliency Benchmark, MIT."},{"key":"ref_15","unstructured":"Judd, T., Durand, F., and Torralba, A. (2012). A Benchmark of Computational Models of Saliency to Predict Human Fixations, MIT. MIT Technical Report."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1254","DOI":"10.1109\/34.730558","article-title":"A model of saliency-based visual attention for rapid scene analysis","volume":"20","author":"Itti","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","unstructured":"Sch\u00f6lkopf, B., Platt, J.C., and Hoffman, T. (2007). Graph-Based Visual Saliency. Advances in Neural Information Processing Systems (NIPS), MIT Press."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Dobnikar, A., Lotri\u010d, U., and \u0160ter, B. (2011). Using Pattern Recognition to Predict Driver Intent. Adaptive and Natural Computing Algorithms, Springer.","DOI":"10.1007\/978-3-642-20267-4"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Tawari, A., and Kang, B. (2018). Systems and Methods of a Computational Framework for a Driver\u2019s Visual Attention Using a Fully Convolutional Architecture. (US20180225554A1), US Patent.","DOI":"10.1109\/IVS.2017.7995828"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1109\/TPAMI.2016.2572683","article-title":"Fully Convolutional Networks for Semantic Segmentation","volume":"39","author":"Shelhamer","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","unstructured":"Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems (NIPS), Curran Associates, Inc."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kim, J., and Canny, J. (2017, January 22\u201329). Interpretable learning for self-driving cars by visualizing causal attention. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.320"},{"key":"ref_24","unstructured":"Yu, F., and Koltun, V. (2016, January 2\u20134). Multi-Scale Context Aggregation by Dilated Convolutions. Proceedings of the 4th International Conference on Learning Representations (ICLR), San Juan, Puerto Rico."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Springer International Publishing.","DOI":"10.1007\/978-3-319-24553-9"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Noh, H., Hong, S., and Han, B. (2015, January 7\u201313). Learning Deconvolution Network for Semantic Segmentation. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.178"},{"key":"ref_28","unstructured":"Sun, K., Zhao, Y., Jiang, B., Cheng, T., Xiao, B., Liu, D., Mu, Y., Wang, X., Liu, W., and Wang, J. (2019). High-Resolution Representations for Labeling Pixels and Regions. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs","volume":"40","author":"Chen","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","unstructured":"Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Computer Vision\u2014ECCV 2018, Springer International Publishing."},{"key":"ref_31","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid Scene Parsing Network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lin, G., Milan, A., Shen, C., and Reid, I. (2017, January 21\u201326). RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation. Proceedings of the 2017 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.549"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Pohlen, T., Hermans, A., Mathias, M., and Leibe, B. (2017, January 21\u201326). Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.353"},{"key":"ref_35","unstructured":"Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018). ICNet for Real-Time Semantic Segmentation on High-Resolution Images. Computer Vision\u2014ECCV 2018, Springer International Publishing."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Bruce, N.D., Catton, C., and Janjic, S. (2016, January 27\u201330). A deeper look at saliency: Feature contrast, semantics, and beyond. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.62"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1167\/7.14.4","article-title":"The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions","volume":"7","author":"Tatler","year":"2007","journal-title":"J. Vis."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1167\/9.7.4","article-title":"Quantifying center bias of observers in free viewing of dynamic natural scenes","volume":"9","author":"Tseng","year":"2009","journal-title":"J. Vis."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_40","unstructured":"Williams, R.H. (2003). Probability, Statistics, and Random Processes for Engineers, Cl-Engineering."},{"key":"ref_41","unstructured":"Leon-Garcia, A. (2017). Probability, Statistics, and Random Processes for Electrical Engineering, Pearson."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1109\/TPAMI.2018.2815601","article-title":"What do different evaluation metrics tell us about saliency models?","volume":"41","author":"Bylinskii","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/7\/2030\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:15:29Z","timestamp":1760174129000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/7\/2030"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,4]]},"references-count":42,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["s20072030"],"URL":"https:\/\/doi.org\/10.3390\/s20072030","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,4]]}}}