{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T10:39:28Z","timestamp":1774694368276,"version":"3.50.1"},"reference-count":24,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2022,9,6]],"date-time":"2022-09-06T00:00:00Z","timestamp":1662422400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>To provide analytic materials for business management for smart retail solutions, it is essential to recognize various customer behaviors (CB) from video footage acquired by in-store cameras. Along with frequent changes in needs and environments, such as promotion plans, product categories, in-store layouts, etc., the targets of customer behavior recognition (CBR) also change frequently. Therefore, one of the requirements of the CBR method is the flexibility to adapt to changes in recognition targets. However, existing approaches, mostly based on machine learning, usually take a great deal of time to re-collect training data and train new models when faced with changing target CBs, reflecting their lack of flexibility. In this paper, we propose a CBR method to achieve flexibility by considering CB in combination with primitives. A primitive is a unit that describes an object\u2019s motion or multiple objects\u2019 relationships. The combination of different primitives can characterize a particular CB. Since primitives can be reused to define a wide range of different CBs, our proposed method is capable of flexibly adapting to target CB changes in retail stores. In experiments undertaken, we utilized both our collected laboratory dataset and the public MERL dataset. We changed the combination of primitives to cope with the changes in target CBs between different datasets. As a result, our proposed method achieved good flexibility with acceptable recognition accuracy.<\/jats:p>","DOI":"10.3390\/s22186740","type":"journal-article","created":{"date-parts":[[2022,9,8]],"date-time":"2022-09-08T04:18:32Z","timestamp":1662610712000},"page":"6740","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["A Customer Behavior Recognition Method for Flexibly Adapting to Target Changes in Retail Stores"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7384-0957","authenticated-orcid":false,"given":"Jiahao","family":"Wen","sequence":"first","affiliation":[{"name":"Graduate School of Information Sciences, Tohoku University, 6-3-09 Aramaki-Aza-Aoba, Aoba-ku, Sendai 980-8579, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3786-0122","authenticated-orcid":false,"given":"Toru","family":"Abe","sequence":"additional","affiliation":[{"name":"Cyberscience Center, Tohoku University, 6-3 Aramaki-Aza-Aoba, Aoba-ku, Sendai 980-8578, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5798-5125","authenticated-orcid":false,"given":"Takuo","family":"Suganuma","sequence":"additional","affiliation":[{"name":"Cyberscience Center, Tohoku University, 6-3 Aramaki-Aza-Aoba, Aoba-ku, Sendai 980-8578, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,6]]},"reference":[{"key":"ref_1","unstructured":"Data Bridge Market Research (2022, June 18). Global Smart Retail Market\u2014Industry Trends and Forecast to 2029. Available online: https:\/\/www.databridgemarketresearch.com\/reports\/global-smart-retail-market."},{"key":"ref_2","unstructured":"Hernandez, D.A.M., Nalbach, O., and Werth, D. (2019, January 15\u201317). How computer vision provides physical retail with a better view on customers. Proceedings of the 2019 IEEE 21st Conference on Business Informatics, Moscow, Russia."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00138-020-01118-w","article-title":"Deep understanding of shopper behaviours and interactions using RGB-D vision","volume":"31","author":"Paolanti","year":"2020","journal-title":"Mach. Vision Appl."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Popa, M.C., Gritti, T., Rothkrantz, L.J.M., Shan, C., and Wiggers, P. (2011, January 29\u201331). Detecting customers\u2019 buying events on a real-life database. Proceedings of the  14th International Conference on Computer Analysis of Images and Patterns, Seville, Spain.","DOI":"10.1007\/978-3-642-23672-3_3"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1879","DOI":"10.1016\/j.patrec.2012.11.015","article-title":"Shopping behavior recognition using a language modeling analogy","volume":"34","author":"Popa","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.patrec.2016.04.008","article-title":"Purchase behavior analysis through gaze and gesture observation","volume":"81","author":"Merad","year":"2016","journal-title":"Pattern Recognit. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Singh, B., Marks, T.K., Jones, M., Tuzel, O., and Shao, M. (2016, January 27\u201330). A multi-stream bi-directional recurrent neural network for fine-grained action detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognitiont, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.216"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_9","first-page":"63","article-title":"Customer Activity Recognition System using Image Processing","volume":"21","author":"WaqasMaria","year":"2021","journal-title":"Int. J. Comput. Sci. Netw. Secur."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liu, J., Gu, Y., and Kamijo, S. (2015, January 14\u201316). Customer behavior recognition in retail store from surveillance camera. Proceedings of the 2015 IEEE International Symposium on Multimedia (ISM), Miami, FL, USA.","DOI":"10.1109\/ISM.2015.52"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yamamoto, J., Inoue, K., and Yoshioka, M. (2017, January 24\u201331). Investigation of customer behavior analysis based on top-view depth camera. Proceedings of the 2017 IEEE Winter Applications of Computer Vision Workshops, Santa Rosa, CA, USA.","DOI":"10.1109\/WACVW.2017.18"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Frontoni, E., Raspa, P., Mancini, A., Zingaretti, P., and Placidi, V. (2013, January 9\u201313). Customers\u2019 activity recognition in intelligent retail environments. Proceedings of the 2013 Image Analysis and Processing, Naples, Italy.","DOI":"10.1007\/978-3-642-41190-8_55"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Liciotti, D., Contigiani, M., Frontoni, E., Mancini, A., Zingaretti, P., and Placidi, V. (2014, January 24). Shopper analytics: A customer activity recognition system using a distributed RGB-D camera network. Proceedings of the International Workshop on Video Analytics for Audience Measurement in Retail and Digital Signage, Stockholm, Sweden.","DOI":"10.1007\/978-3-319-12811-5_11"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, H., Hartmann, Y., and Schultz, T. (2021, January 23\u201327). Motion Units: Generalized Sequence Modeling of Human Activities for Sensor-Based Activity Recognition. Proceedings of the 2021 29th European Signal Processing Conference (EUSIPCO), Dublin, Ireland.","DOI":"10.23919\/EUSIPCO54536.2021.9616298"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hartmann., Y., Liu., H., Lahrberg., S., and Schultz., T. (2022, January 9\u201311). Interpretable High-level Features for Human Activity Recognition. Proceedings of the 15th International Joint Conference on Biomedical Engineering Systems and Technologies\u2014BIOSIGNALS, Online.","DOI":"10.5220\/0010840500003123"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Rai, N., Chen, H., Ji, J., Desai, R., Kozuka, K., Ishizaka, S., Adeli, E., and Niebles, J.C. (2021, January 20\u201325). Home Action Genome: Cooperative Compositional Action Understanding. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01103"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Mansour, R.F., Escorcia-Gutierrez, J., Gamarra, M., Villanueva, J.A., and Leal, N. (2021). Intelligent video anomaly detection and classification using faster RCNN with deep reinforcement learning model. Image Vis. Comput., 112.","DOI":"10.1016\/j.imavis.2021.104229"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Generosi, A., Ceccacci, S., and Mengoni, M. (2018, January 2\u20135). A deep learning-based system to track and analyze customer behavior in retail store. Proceedings of the 2018 IEEE 8th International Conference on Consumer Electronics, Berlin, Germany.","DOI":"10.1109\/ICCE-Berlin.2018.8576169"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Lee, J.G., Han, J., and Whang, K.Y. (2007, January 12\u201314). Trajectory clustering: A partition-and-group framework. Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, Beijing, China.","DOI":"10.1145\/1247480.1247546"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Leal, E., and Gruenwald, L. (2018, January 10\u201313). DynMDL: A parallel trajectory segmentation algorithm. Proceedings of the 2018 IEEE International Conference on Big Data, Seattle, WA, USA.","DOI":"10.1109\/BigDataCongress.2018.00036"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wen, J., Guillen, L., Abe, T., and Suganuma, T. (2021). A Hierarchy-Based System for Recognizing Customer Activity in Retail Environments. Sensors, 21.","DOI":"10.3390\/s21144712"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. (2016). Simple Online and Realtime Tracking. arXiv.","DOI":"10.1109\/ICIP.2016.7533003"},{"key":"ref_23","unstructured":"Cheng, B., Xiao, B., Wang, J., Shi, H., Huang, T.S., and Zhang, L. (2022, July 12). Bottom-up Higher-Resolution Networks for Multi-Person Pose Estimation. Available online: https:\/\/www.catalyzex.com\/paper\/arxiv:1908.10357."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Doll\u00e1r, P. (2014). Microsoft COCO: Common Objects in Context. Computer Vision\u2014ECCV 2014, Springer.","DOI":"10.1007\/978-3-319-10602-1_48"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6740\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:24:30Z","timestamp":1760142270000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6740"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,6]]},"references-count":24,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22186740"],"URL":"https:\/\/doi.org\/10.3390\/s22186740","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,6]]}}}