{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T07:43:01Z","timestamp":1773301381025,"version":"3.50.1"},"reference-count":49,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,4,16]],"date-time":"2021-04-16T00:00:00Z","timestamp":1618531200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Education, Taiwan","award":["the Higher Education Sprout project"],"award-info":[{"award-number":["the Higher Education Sprout project"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this study, an Artificial Intelligence of Things (AIoT)-based automated picking system was proposed for the development of an online shop and the services for automated shipping systems. Speed and convenience are two key points in Industry 4.0 and Society 5.0. In the context of online shopping, speed and convenience can be provided by integrating e-commerce platforms with AIoT systems and robots that are following consumers\u2019 needs. Therefore, this proposed system diverts consumers who are moved by AIoT, while robotic manipulators replace human tasks to pick. To prove this idea, we implemented a modified YOLO (You Only Look Once) algorithm as a detection and localization tool for items purchased by consumers. At the same time, the modified YOLOv2 with data-driven mode was used for the process of taking goods from unstructured shop shelves. Our system performance is proven by experiments to meet the expectations in evaluating efficiency, speed, and convenience of the system in Society 5.0\u2019s context.<\/jats:p>","DOI":"10.3390\/s21082813","type":"journal-article","created":{"date-parts":[[2021,4,19]],"date-time":"2021-04-19T06:35:53Z","timestamp":1618814153000},"page":"2813","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["An Artificial Intelligence of Things-Based Picking Algorithm for Online Shop in the Society 5.0\u2019s Context"],"prefix":"10.3390","volume":"21","author":[{"given":"Muslikhin","family":"Muslikhin","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Southern Taiwan University of Science and Technology, Tainan 710, Taiwan"},{"name":"Department of Electronics Engineering Education, Universitas Negeri Yogyakarta, Yogyakarta 55281, Indonesia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jenq-Ruey","family":"Horng","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Southern Taiwan University of Science and Technology, Tainan 710, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2812-4089","authenticated-orcid":false,"given":"Szu-Yueh","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Southern Taiwan University of Science and Technology, Tainan 710, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ming-Shyan","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Southern Taiwan University of Science and Technology, Tainan 710, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-2005-6404","authenticated-orcid":false,"given":"Baiti-Ahmad","family":"Awaluddin","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Southern Taiwan University of Science and Technology, Tainan 710, Taiwan"},{"name":"Department of Electronics Engineering Education, Universitas Negeri Yogyakarta, Yogyakarta 55281, Indonesia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.jretai.2019.12.004","article-title":"Autonomous Shopping Systems: Identifying and Overcoming Barriers to Consumer Adoption","volume":"96","author":"Johar","year":"2020","journal-title":"J. Retail."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Loke, K. (2017, January 12\u201314). Automatic recognition of clothes pattern and motifs empowering online fashion shopping. Proceedings of the 2017 IEEE International Conference on Consumer Electronics\u2014Taiwan (ICCE-TW), Taipei, Taiwan.","DOI":"10.1109\/ICCE-China.2017.7991152"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Mantha, A., Arora, Y., Gupta, S., Kanumala, P., Liu, Z., Guo, S., and Achan, K. (2020, January 4\u20138). A Large-Scale Deep Architecture for Personalized Grocery Basket Recommendations. Proceedings of the ICASSP 2020\u20142020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9053091"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Matsuda, K., Uesugi, S., Naruse, K., and Morita, M. (2019, January 28\u201330). Technologies of Production with Society 5.0. Proceedings of the 2019 6th International Conference on Behavioral, Economic and Socio-Cultural Computing (BESC), Beijing, China.","DOI":"10.1109\/BESC48373.2019.8963541"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Martynov, V.V., Shavaleeva, D.N., and Zaytseva, A.A. (2019, January 23\u201327). Information Technology as the Basis for Transformation into a Digital Society and Industry 5.0. Proceedings of the 2019 International Conference \u201cQuality Management, Transport and Information Security, Information Technologies\u201d (IT&QM&IS), Sochi, Russia.","DOI":"10.1109\/ITQMIS.2019.8928305"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Shibata, M., Ohtsuka, Y., Okamoto, K., and Takahashi, M. (2017, January 9\u201313). Toward an Efficient Search Method to Capture the Future MOT Curriculum Based on the Society 5.0. Proceedings of the 2017 Portland International Conference on Management of Engineering and Technology (PICMET), Portland, OR, USA.","DOI":"10.23919\/PICMET.2017.8125333"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"107460","DOI":"10.1016\/j.ijpe.2019.07.033","article-title":"Science, technology and innovation ecosystem transformation toward society 5.0","volume":"220","author":"Fukuda","year":"2020","journal-title":"Int. J. Prod. Econ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/S1361-3723(19)30087-9","article-title":"The future is Society 5.0","volume":"2019","author":"Grunwitz","year":"2019","journal-title":"Comput. Fraud. Secur."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Tripathy, A.K., Carvalho, R., Pawaskar, K., Yadav, S., and Yadav, V. (2015, January 4\u20136). Mobile based healthcare management using artificial intelligence. Proceedings of the 2015 International Conference on Technologies for Sustainable Development (ICTSD), Mumbai, India.","DOI":"10.1109\/ICTSD.2015.7095895"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1016\/j.isatra.2019.06.026","article-title":"An experimental setup of multi-intelligent control system (MICS) of water management using the Internet of Things (IoT)","volume":"96","author":"Hadipour","year":"2020","journal-title":"ISA Trans."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"44441","DOI":"10.1109\/ACCESS.2019.2908843","article-title":"A Deep Learning-Based Intelligent Medicine Recognition System for Chronic Patients","volume":"7","author":"Chang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"134826","DOI":"10.1109\/ACCESS.2020.3010847","article-title":"ScalpEye: A Deep Learning-Based Scalp Hair Inspection and Diagnosis System for Scalp Health","volume":"8","author":"Chang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"17013","DOI":"10.1109\/ACCESS.2020.2967400","article-title":"MedGlasses: A Wearable Smart-Glasses-Based Drug Pill Recognition System Using Deep Learning for Visually Impaired Chronic Patients","volume":"8","author":"Chang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"148163","DOI":"10.1109\/ACCESS.2019.2946468","article-title":"DeepCrash: A Deep Learning-Based Internet of Vehicles System for Head-On and Single-Vehicle Accident Detection with Emergency Notification","volume":"7","author":"Chang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"62032","DOI":"10.1109\/ACCESS.2020.2984383","article-title":"Enabling Intelligent Environment by the Design of Emotionally Aware Virtual Assistant: A Case of Smart Campus","volume":"8","author":"Chiu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"175487","DOI":"10.1109\/ACCESS.2019.2955834","article-title":"Fast and Efficient Non-Contact Ball Detector for Picking Robots","volume":"7","author":"Mao","year":"2019","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"180998","DOI":"10.1109\/ACCESS.2019.2958614","article-title":"Tender Tea Shoots Recognition and Positioning for Picking Robot Using Improved YOLO-V3 Model","volume":"7","author":"Yang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"158035","DOI":"10.1109\/ACCESS.2019.2946267","article-title":"Grasping Point Detection of Randomly Placed Fruit Cluster Using Adaptive Morphology Segmentation and Principal Component Classification of Multiple Features","volume":"7","author":"Zhang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4224","DOI":"10.1109\/TII.2018.2822828","article-title":"Object Classification Using CNN-Based Fusion of Vision and LIDAR in Autonomous Vehicle Environment","volume":"14","author":"Gao","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"121765","DOI":"10.1109\/ACCESS.2020.3006843","article-title":"Object Localization and Depth Estimation for Eye-in-Hand Manipulator Using Mono Camera","volume":"8","author":"Muslikhin","year":"2020","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"147642","DOI":"10.1109\/ACCESS.2019.2946369","article-title":"Fruit Localization and Environment Perception for Strawberry Harvesting Robots","volume":"7","author":"Ge","year":"2019","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1109\/LRA.2018.2885579","article-title":"Inferring 3D Shapes of Unknown Rigid Objects in Clutter through Inverse Physics Reasoning","volume":"4","author":"Song","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1109\/TASE.2014.2361346","article-title":"Using Manipulation Primitives for Object Sorting in Cluttered Environments","volume":"12","author":"Gupta","year":"2014","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"113342","DOI":"10.1016\/j.eswa.2020.113342","article-title":"Predicting online shopping behaviour from clickstream data using deep learning","volume":"150","author":"Koehn","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yamamoto, Y., Kawabe, T., Tsuruta, S., Damiani, E., Yoshitaka, A., Mizuno, Y., Sakurai, Y., and Knauf, R. (December, January 28). Enhanced IoT-Aware Online Shopping System. Proceedings of the 2016 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), Naples, Italy.","DOI":"10.1109\/SITIS.2016.14"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Rashid, Z., Peig, E., and Pous, R. (2015, January 26\u201328). Bringing online shopping experience to offline retail through augmented reality and RFID. Proceedings of the 2015 5th International Conference on the Internet of Things (IOT), Seoul, Korea.","DOI":"10.1109\/IOT.2015.7356547"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"11246","DOI":"10.1109\/ACCESS.2019.2892310","article-title":"When Time Matters: Predictive Mission Planning in Cyber-Physical Scenarios","volume":"7","author":"Gaggero","year":"2019","journal-title":"IEEE Access"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1109\/TNSM.2016.2516946","article-title":"Design Patterns and Extensibility of REST API for Networking Applications","volume":"13","author":"Li","year":"2016","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Du, Y.-C., Muslikhin, M., Hsieh, T.-H., and Wang, M.-S. (2020). Stereo Vision-Based Object Recognition and Manipulation by Regions with Convolutional Neural Network. Electronics, 9.","DOI":"10.3390\/electronics9020210"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sriyakul, H., Koolpiruck, D., Songkasiri, W., and Nuratch, S. (2017, January 21\u201323). Cyber-Physical System Based Production Monitoring for Tapioca Starch Production. Proceedings of the 2017 4th International Conference on Information Science and Control Engineering (ICISCE), Changsha, China.","DOI":"10.1109\/ICISCE.2017.196"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1109\/MC.2018.3011041","article-title":"Society 5.0: For Human Security and Well-Being","volume":"51","author":"Shiroishi","year":"2018","journal-title":"Computer"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1764","DOI":"10.1109\/TII.2017.2759319","article-title":"CPS-Based Smart Control Model for Shopfloor Material Handling","volume":"14","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1016\/j.protcy.2012.03.022","article-title":"Developing Social Networks Mashups: An Overview of REST-Based APIs","volume":"3","year":"2012","journal-title":"Procedia Technol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"9188","DOI":"10.1109\/ACCESS.2017.2702715","article-title":"A Three-Dimensional Adaptive PSO-Based Packing Algorithm for an IoT-Based Automated e-Fulfillment Packaging System","volume":"5","author":"Li","year":"2017","journal-title":"IEEE Access"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"139635","DOI":"10.1109\/ACCESS.2019.2942144","article-title":"Cucumber Fruits Detection in Greenhouses Based on Instance Segmentation","volume":"7","author":"Liu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Taryudi, and Wang, M.-S. (2018). Eye to hand calibration using ANFIS for stereo vision-based object manipulation system. Microsyst. Technol., 24, 305\u2013317.","DOI":"10.1007\/s00542-017-3315-y"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cviu.2007.09.014","article-title":"Speeded-Up Robust Features (SURF)","volume":"110","author":"Bay","year":"2008","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"273","DOI":"10.3182\/20090916-3-BR-3001.0048","article-title":"Speeded up Robust Features for vision-based underwater motion estimation and SLAM: Comparison with correlation-based techniques","volume":"42","author":"Ferreira","year":"2009","journal-title":"IFAC Proc. Vol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.image.2015.09.007","article-title":"Sequential block-based disparity map estimation algorithm for stereoscopic image coding","volume":"39","author":"Kadaikar","year":"2015","journal-title":"Signal Process. Image Commun."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1109\/TSMCB.2004.842249","article-title":"Multicriteria Meta-Heuristics for AGV Dispatching Control Based on Computational Intelligence","volume":"35","author":"Naso","year":"2005","journal-title":"IEEE Trans. Syst. Man Cybern. Part B"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"113900","DOI":"10.1016\/j.eswa.2020.113900","article-title":"Estimation of disparity maps through an evolutionary algorithm and global image features as descriptors","volume":"165","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"9370","DOI":"10.1109\/JSEN.2018.2870957","article-title":"Visual Object Recognition and Pose Estimation Based on a Deep Semantic Segmentation Network","volume":"18","author":"Lin","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Ji, W., Meng, X., Qian, Z., Xu, B., and Zhao, D. (2017). Branch localization method based on the skeleton feature extraction and stereo matching for apple harvesting robot. Int. J. Adv. Robot. Syst., 14.","DOI":"10.1177\/1729881417705276"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/j.comnet.2018.12.008","article-title":"Internet of Things applications: A systematic review","volume":"148","author":"Asghari","year":"2019","journal-title":"Comput. Netw."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1199","DOI":"10.1016\/j.procs.2019.11.233","article-title":"A Review of Usability and Security Evaluation Model of Ecommerce Website","volume":"161","author":"Mohd","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1016\/j.apergo.2013.03.031","article-title":"Evaluating ecommerce websites cognitive efficiency: An integrative framework based on data envelopment analysis","volume":"44","author":"Storto","year":"2013","journal-title":"Appl. Ergon."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/8\/2813\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:48:57Z","timestamp":1760161737000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/8\/2813"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,16]]},"references-count":49,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["s21082813"],"URL":"https:\/\/doi.org\/10.3390\/s21082813","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,16]]}}}