{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T00:43:18Z","timestamp":1771634598857,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2021,5,21]],"date-time":"2021-05-21T00:00:00Z","timestamp":1621555200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>As many as 40% to 50% of patients do not adhere to long-term medications for managing chronic conditions, such as diabetes or hypertension. Limited opportunity for medication monitoring is a major problem from the perspective of health professionals. The availability of prompt medication error reports can enable health professionals to provide immediate interventions for patients. Furthermore, it can enable clinical researchers to modify experiments easily and predict health levels based on medication compliance. This study proposes a method in which videos of patients taking medications are recorded using a camera image sensor integrated into a wearable device. The collected data are used as a training dataset based on applying the latest convolutional neural network (CNN) technique. As for an artificial intelligence (AI) algorithm to analyze the medication behavior, we constructed an object detection model (Model 1) using the faster region-based CNN technique and a second model that uses the combined feature values to perform action recognition (Model 2). Moreover, 50,000 image data were collected from 89 participants, and labeling was performed on different data categories to train the algorithm. The experimental combination of the object detection model (Model 1) and action recognition model (Model 2) was newly developed, and the accuracy was 92.7%, which is significantly high for medication behavior recognition. This study is expected to enable rapid intervention for providers seeking to treat patients through rapid reporting of drug errors.<\/jats:p>","DOI":"10.3390\/s21113594","type":"journal-article","created":{"date-parts":[[2021,5,24]],"date-time":"2021-05-24T00:01:20Z","timestamp":1621814480000},"page":"3594","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Development of a Wearable Camera and AI Algorithm for Medication Behavior Recognition"],"prefix":"10.3390","volume":"21","author":[{"given":"Hwiwon","family":"Lee","sequence":"first","affiliation":[{"name":"InHandPlus, Inc., Seoul 06248, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sekyoung","family":"Youm","sequence":"additional","affiliation":[{"name":"Department of Industrial and Systems Engineering and Gerontechnology Research Center, Dongguk University, Seoul 40620, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,5,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1038\/s41437-020-0303-2","article-title":"Big data in digital healthcare: Lessons learnt and recommendations for general practice","volume":"124","author":"Agrawal","year":"2020","journal-title":"Heredity"},{"key":"ref_2","first-page":"384","article-title":"Big data in healthcare\u2014The promises, challenges and opportunities from a research perspective: A case study with a model database","volume":"2017","author":"Adibuzzaman","year":"2017","journal-title":"AMIA Annu. Symp."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1173","DOI":"10.15537\/smj.2017.12.20631","article-title":"The impact of health information technology on patient safety","volume":"38","author":"Alotaibi","year":"2017","journal-title":"Saudi Med. J."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lee, D., and Yoon, S. (2021). Application of artificial intelligence-based technologies in the healthcare industry: Opportunities and challenges. Int. J. Environ. Res. Public Health, 18.","DOI":"10.3390\/ijerph18010271"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1111\/j.1547-5069.2011.01386.x","article-title":"Barriers for nurses to safe medication management in nursing homes","volume":"43","author":"Dilles","year":"2011","journal-title":"J. Nurs. Scholarsh."},{"key":"ref_6","unstructured":"Peter, G. (2021, March 01). Peterson Foundation. Why Are Americans Paying More for Healthcare?. Available online: https:\/\/www.pgpf.org\/blog\/2020\/04\/why-are-americans-paying-more-for-healthcare."},{"key":"ref_7","unstructured":"Alsumidaie, M. (2021, March 01). Non-Adherence: A Direct Influence on Clinical Trial Duration and Cost. Applied Clinical Trials Online. Available online: www.appliedclinicaltrialsonline.com\/non-adherence-direct-influenceclinical-trial-duration-and-cost."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"18","DOI":"10.7812\/TPP\/18-033","article-title":"The unmet challenge of medication nonadherence","volume":"22","author":"Kleinsinger","year":"2018","journal-title":"Perm. J."},{"key":"ref_9","unstructured":"Rhodes, C. (2021, March 01). Three Reasons for Nonadherence, Managed Healthcare Executive. Available online: https:\/\/www.managedhealthcareexecutive.com."},{"key":"ref_10","unstructured":"Muoio, D. (2021, March 01). \u2018Digital pill\u2019 maker Proteus Digital Health Files for Bankruptcy. Insights, HIMSS Publication, Special Edition: E-health Trend Barometer. Available online: https:\/\/www.mobihealthnews.com\/news\/digital-pill-maker-proteus-digital-health-files-bankruptcy."},{"key":"ref_11","unstructured":"Son, J. (2021, March 01). AI-Based Medication Notification Service, AiCure. The Invention Lab. Available online: https:\/\/verticalplatform.kr\/archives\/6991."},{"key":"ref_12","first-page":"1","article-title":"A Review of Medication Adherence Monitoring Technologies","volume":"1","author":"Aldeer","year":"2018","journal-title":"Appl. Syst. Innov."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_14","unstructured":"Mwiti, D. (2021, March 01). A 2019 Guide to Object Detection. Heartbeat. Available online: https:\/\/heartbeat.fritz.ai\/a-2019-guide-to-object-detection-9509987954c3."},{"key":"ref_15","unstructured":"Google Code Archive (2020, December 01). The Sports-1M Dataset. Available online: https:\/\/code.google.com\/archive\/p\/sports-1m-dataset\/."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.patcog.2017.01.030","article-title":"Accurate object detection using memory-based models in surveillance scenes","volume":"67","author":"Li","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Qiao, S., Xie, C., Shen, W., Wang, B., and Yuille, A.L. (2018, January 18\u201323). Single-shot object detection with enriched semantics. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00609"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Nie, B.X., Xiong, C., and Zhu, S.C. (2015, January 7\u201312). Joint action recognition and pose estimation from video. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298734"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Du, W., Wang, Y., and Qiao, Y. (2017, January 22\u201329). RPAN: An end-to-end recurrent pose-attention network for action recognition in videos. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.402"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hong, J., Cho, B., Hong, Y.W., and Byun, H. (2019). Contextual action cues from camera sensor for multi-stream action recognition. Sensors, 19.","DOI":"10.3390\/s19061382"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1007\/s10916-014-0095-0","article-title":"Better physical activity classification using smartphone acceleration sensor","volume":"38","author":"Arif","year":"2014","journal-title":"J. Med. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.neucom.2015.08.096","article-title":"From action to activity: Sensor-based activity recognition","volume":"181","author":"Liu","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_23","unstructured":"Lee, S.M., Yoon, S.M., and Cho, H. (2017, January 13\u201316). Human activity recognition from accelerometer data using convolutional neural network. Proceedings of the 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju, Korea."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ma, J., Ovalle, A., and Woodbridge, D.M.K. (2018, January 18\u201321). Medhere: A smartwatch-based medication adherence monitoring system using machine learning and distributed computing. Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA.","DOI":"10.1109\/EMBC.2018.8513169"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wang, J., Huang, Z., Zhang, W., Patil, A., Patil, K., Zhu, T., and Harris, T.B. (2016, January 5\u20138). Wearable sensor based human posture recognition. Proceedings of the 2016 IEEE International Conference on Big Data (Big Data), Washington, DC, USA.","DOI":"10.1109\/BigData.2016.7841004"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1007\/s10586-017-0977-2","article-title":"Wearable sensor devices for early detection of Alzheimer disease using dynamic time warping algorithm","volume":"21","author":"Varatharajan","year":"2018","journal-title":"Clust. Comput."},{"key":"ref_27","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., and Shlens, J. (July, January 26). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e115","DOI":"10.2196\/mhealth.7521","article-title":"Feature-free activity classification of inertial sensor data with machine vision techniques: Method, development, and evaluation","volume":"5","author":"Veiga","year":"2017","journal-title":"JMIR mHealth uHealth"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1016\/j.future.2017.11.029","article-title":"A robust human activity recognition system using smartphone sensors and deep learning","volume":"81","author":"Hassan","year":"2018","journal-title":"Future Generat. Comp. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., and Fei-Fei, L. (2014, January 23\u201328). Large-scale video classification with convolutional neural networks. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.223"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gkioxari, G., Girshick, R., Doll\u00e1r, P., and He, K. (2018, January 18\u201323). Detecting and recognizing human-object interactions. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00872"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/11\/3594\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:05:41Z","timestamp":1760162741000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/11\/3594"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,21]]},"references-count":31,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["s21113594"],"URL":"https:\/\/doi.org\/10.3390\/s21113594","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,21]]}}}