{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T15:01:06Z","timestamp":1775833266334,"version":"3.50.1"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,3,1]],"date-time":"2019-03-01T00:00:00Z","timestamp":1551398400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,3,1]],"date-time":"2019-03-01T00:00:00Z","timestamp":1551398400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Early and frequent patient mobilization substantially mitigates risk for post-intensive care syndrome and long-term functional impairment. We developed and tested computer vision algorithms to detect patient mobilization activities occurring in an adult ICU. Mobility activities were defined as moving the patient into and out of bed, and moving the patient into and out of a chair. A data set of privacy-safe-depth-video images was collected in the Intermountain LDS Hospital ICU, comprising 563 instances of mobility activities and 98,801 total frames of video data from seven wall-mounted depth sensors. In all, 67% of the mobility activity instances were used to train algorithms to detect mobility activity occurrence and duration, and the number of healthcare personnel involved in each activity. The remaining 33% of the mobility instances were used for algorithm evaluation. The algorithm for detecting mobility activities attained a mean specificity of 89.2% and sensitivity of 87.2% over the four activities; the algorithm for quantifying the number of personnel involved attained a mean accuracy of 68.8%.<\/jats:p>","DOI":"10.1038\/s41746-019-0087-z","type":"journal-article","created":{"date-parts":[[2019,3,1]],"date-time":"2019-03-01T11:06:43Z","timestamp":1551438403000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":100,"title":["A computer vision system for deep learning-based detection of patient mobilization activities in the ICU"],"prefix":"10.1038","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0529-0628","authenticated-orcid":false,"given":"Serena","family":"Yeung","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3296-6630","authenticated-orcid":false,"given":"Francesca","family":"Rinaldo","sequence":"additional","affiliation":[]},{"given":"Jeffrey","family":"Jopling","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6621-0356","authenticated-orcid":false,"given":"Bingbin","family":"Liu","sequence":"additional","affiliation":[]},{"given":"Rishab","family":"Mehra","sequence":"additional","affiliation":[]},{"given":"N. Lance","family":"Downing","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6574-6669","authenticated-orcid":false,"given":"Michelle","family":"Guo","sequence":"additional","affiliation":[]},{"given":"Gabriel M.","family":"Bianconi","sequence":"additional","affiliation":[]},{"given":"Alexandre","family":"Alahi","sequence":"additional","affiliation":[]},{"given":"Julia","family":"Lee","sequence":"additional","affiliation":[]},{"given":"Brandi","family":"Campbell","sequence":"additional","affiliation":[]},{"given":"Kayla","family":"Deru","sequence":"additional","affiliation":[]},{"given":"William","family":"Beninati","sequence":"additional","affiliation":[]},{"given":"Li","family":"Fei-Fei","sequence":"additional","affiliation":[]},{"given":"Arnold","family":"Milstein","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,3,1]]},"reference":[{"key":"87_CR1","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1056\/NEJMoa022450","volume":"348","author":"MS Herridge","year":"2003","unstructured":"Herridge, M. S. et al. One-year outcomes in survivors of the acute respiratory distress syndrome. New Engl. J. Med. 348, 683\u2013693 (2003).","journal-title":"New Engl. J. Med."},{"key":"87_CR2","doi-asserted-by":"publisher","first-page":"1787","DOI":"10.1001\/jama.2010.1553","volume":"304","author":"TJ Iwashyna","year":"2010","unstructured":"Iwashyna, T. J., Ely, E. W., Smith, D. M. & Langa, K. M. Long-term cognitive impairment and functional disability among survivors of severe sepsis. JAMA 304, 1787\u20131794 (2010).","journal-title":"JAMA"},{"key":"87_CR3","doi-asserted-by":"publisher","first-page":"502","DOI":"10.1097\/CCM.0b013e318232da75","volume":"40","author":"DM Needham","year":"2012","unstructured":"Needham, D. M. et al. Improving long-term outcomes after discharge from intensive care unit: report from a stakeholders\u2019 conference. Crit. Care Med. 40, 502\u2013509 (2012).","journal-title":"Crit. Care Med."},{"key":"87_CR4","doi-asserted-by":"publisher","first-page":"1874","DOI":"10.1016\/S0140-6736(09)60658-9","volume":"373","author":"WD Schweickert","year":"2009","unstructured":"Schweickert, W. D. et al. Early physical and occupational therapy in mechanically ventilated, critically ill patients: a randomised controlled trial. Lancet. 373, 1874\u20131882 (2009).","journal-title":"Lancet."},{"key":"87_CR5","doi-asserted-by":"publisher","first-page":"2499","DOI":"10.1097\/CCM.0b013e3181a38937","volume":"37","author":"C Burtin","year":"2009","unstructured":"Burtin, C. et al. Early exercise in critically ill patients enhances short-term functional recovery. Crit. Care Med. 37, 2499\u20132505 (2009).","journal-title":"Crit. Care Med."},{"key":"87_CR6","doi-asserted-by":"publisher","first-page":"931","DOI":"10.1016\/j.apmr.2016.11.007","volume":"98","author":"CC Lai","year":"2017","unstructured":"Lai, C. C. et al. Early mobilization reduces duration of mechanical ventilation and intensive care unit stay in patients with acute respiratory failure. Arch. Phys. Med. Rehabil. 98, 931\u2013939 (2017).","journal-title":"Arch. Phys. Med. Rehabil."},{"key":"87_CR7","doi-asserted-by":"publisher","first-page":"722","DOI":"10.1016\/j.chest.2016.03.003","volume":"150","author":"MD Hashem","year":"2016","unstructured":"Hashem, M. D., Parker, A. M. & Needham, D. M. Early mobilization and rehabilitation of patients who are critically ill. Chest 150, 722\u2013731 (2016).","journal-title":"Chest"},{"key":"87_CR8","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1186\/1477-7525-9-9","volume":"9","author":"MT Vest","year":"2011","unstructured":"Vest, M. T., Murphy, T. E., Araujo, K. L. & Pisani, M. A. Disability in activities of daily living, depression, and quality of life among older medical ICU survivors: a prospective cohort study. Health Qual. Life Outcomes 9, 9 (2011).","journal-title":"Health Qual. Life Outcomes"},{"key":"87_CR9","doi-asserted-by":"publisher","first-page":"R105","DOI":"10.1186\/cc10121","volume":"15","author":"E Sacanella","year":"2011","unstructured":"Sacanella, E. et al. Functional status and quality of life 12 months after discharge from a medical ICU in healthy elderly patients: a prospective observational study. Crit. Care 15, R105 (2011).","journal-title":"Crit. Care"},{"key":"87_CR10","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1186\/s12871-015-0121-x","volume":"15","author":"S Steenbergen","year":"2015","unstructured":"Steenbergen, S. et al. Long-term treated intensive care patients outcomes: the one-year mortality rate, quality of life, health care use and long-term complications as reported by general practitioners. BMC Anesthesiol. 15, 142 (2015).","journal-title":"BMC Anesthesiol."},{"key":"87_CR11","doi-asserted-by":"publisher","first-page":"S69","DOI":"10.1097\/CCM.0b013e3182a240d5","volume":"41","author":"HJ Engel","year":"2013","unstructured":"Engel, H. J., Needham, D. M., Morris, P. E. & Gropper, M. A. ICU early mobilization: from recommendation to implementation at three medical centers. Crit. Care Med. 41, S69\u2013S80 (2013).","journal-title":"Crit. Care Med."},{"key":"87_CR12","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1097\/MAJ.0b013e31820ab4f6","volume":"341","author":"PE Morris","year":"2011","unstructured":"Morris, P. E. et al. Receiving early mobility during an intensive care unit admission is a predictor of improved outcomes in acute respiratory failure. Am. J. Med Sci. 341, 373\u2013377 (2011).","journal-title":"Am. J. Med Sci."},{"key":"87_CR13","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1007\/s00134-016-4612-0","volume":"43","author":"CJ Tipping","year":"2017","unstructured":"Tipping, C. J. et al. The effects of active mobilisation and rehabilitation in ICU on mortality and function: a systematic review. Intensive Care Med. 43, 171\u2013183 (2017).","journal-title":"Intensive Care Med."},{"key":"87_CR14","first-page":"Cd010754","volume":"3","author":"KA Doiron","year":"2018","unstructured":"Doiron, K. A., Hoffmann, T. C. & Beller, E. M. Early intervention (mobilization or active exercise) for critically ill adults in the intensive care unit. Cochrane Database Syst. Rev. 3, Cd010754 (2018).","journal-title":"Cochrane Database Syst. Rev."},{"key":"87_CR15","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1016\/j.chest.2017.03.054","volume":"152","author":"DK Costa","year":"2017","unstructured":"Costa, D. K. et al. Identifying barriers to delivering the awakening and breathing coordination, delirium, and early exercise\/mobility bundle to minimize adverse outcomes for mechanically ventilated patients: a systematic review. Chest 152, 304\u2013311 (2017).","journal-title":"Chest"},{"key":"87_CR16","doi-asserted-by":"publisher","first-page":"658","DOI":"10.1016\/j.jcrc.2015.03.006","volume":"30","author":"SC Berney","year":"2015","unstructured":"Berney, S. C., Rose, J. W., Bernhardt, J. & Denehy, L. Prospective observation of physical activity in critically ill patients who were intubated for more than 48 h. J. Crit. Care. 30, 658\u2013663 (2015).","journal-title":"J. Crit. Care."},{"key":"87_CR17","doi-asserted-by":"publisher","first-page":"379","DOI":"10.2522\/ptj.20130199","volume":"94","author":"DU Jette","year":"2014","unstructured":"Jette, D. U. et al. Validity of the AM-PAC \u201c6-Clicks\u201d inpatient daily activity and basic mobility short forms. Phys. Ther. 94, 379\u2013391 (2014).","journal-title":"Phys. Ther."},{"key":"87_CR18","doi-asserted-by":"publisher","first-page":"1271","DOI":"10.1056\/NEJMp1716891","volume":"378","author":"S Yeung","year":"2018","unstructured":"Yeung, S., Downing, N. L., Fei-Fei, L. & Milstein, A. Bedside computer vision - moving artificial intelligence from driver assistance to patient safety. New Engl. J. Med. 378, 1271\u20131273 (2018).","journal-title":"New Engl. J. Med."},{"key":"87_CR19","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1109\/MIS.2008.18","volume":"23","author":"D S\u00e1nchez","year":"2008","unstructured":"S\u00e1nchez, D., Tentori, M. & Favela, J. Activity recognition for the smart hospital. IEEE Intell. Syst. 23, 50\u201357 (2008).","journal-title":"IEEE Intell. Syst."},{"key":"87_CR20","unstructured":"Haque A., et al. Towards vision-based smart hospitals: a system for tracking and monitoring hand hygiene compliance. arXiv preprint: 1708.00163v3. Preprint at: https:\/\/arxiv.org\/pdf\/1708.00163.pdf (2017)."},{"key":"87_CR21","doi-asserted-by":"publisher","first-page":"958","DOI":"10.1016\/j.jbi.2012.04.003","volume":"45","author":"S Parlak","year":"2012","unstructured":"Parlak, S., Sarcevic, A., Marsic, I. & Burd, R. S. Introducing RFID technology in dynamic and time-critical medical settings: requirements and challenges. J. Biomed. Inform. 45, 958\u2013974 (2012).","journal-title":"J. Biomed. Inform."},{"key":"87_CR22","doi-asserted-by":"crossref","unstructured":"Chakraborty, I., Elgammal, A. & Burd, R. S. Video based activity recognition in trauma resuscitation. In Proceedings of the 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). 1\u20138 (2013).","DOI":"10.1109\/FG.2013.6553758"},{"key":"87_CR23","doi-asserted-by":"publisher","first-page":"737","DOI":"10.1007\/s11548-015-1186-1","volume":"10","author":"AP Twinanda","year":"2015","unstructured":"Twinanda, A. P., Alkan, E. O., Gangi, A., de Mathelin, M. & Padoy, N. Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms. Int J. Comput. Assist Radiol. Surg. 10, 737\u2013747 (2015).","journal-title":"Int J. Comput. Assist Radiol. Surg."},{"key":"87_CR24","doi-asserted-by":"crossref","unstructured":"Jin, A. et al. Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. In Proceedings of the IEEE winter conference on applications of computer vision. 691\u2013699 (2018).","DOI":"10.1109\/WACV.2018.00081"},{"key":"87_CR25","doi-asserted-by":"publisher","first-page":"630","DOI":"10.1097\/CCM.0000000000002265","volume":"45","author":"AJ Ma","year":"2017","unstructured":"Ma, A. J. et al. Measuring patient mobility in the icu using a novel noninvasive sensor. Crit. Care Med. 45, 630\u2013636 (2017).","journal-title":"Crit. Care Med."},{"key":"87_CR26","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1016\/j.ccc.2018.06.005","volume":"34","author":"M Paton","year":"2018","unstructured":"Paton, M., Lane, R. & Hodgson, C. L. Early mobilization in the intensive care unit to improve long-term recovery. Crit. Care Clin. 34, 557\u2013571 (2018).","journal-title":"Crit. Care Clin."},{"key":"87_CR27","first-page":"46","volume":"89","author":"RS Kaplan","year":"2011","unstructured":"Kaplan, R. S. & Porter, M. E. How to solve the cost crisis in health care. Harv. Bus. Rev. 89, 46\u201352 (2011).","journal-title":"Harv. Bus. Rev."},{"key":"87_CR28","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1037\/h0031619","volume":"76","author":"JL Fleiss","year":"1971","unstructured":"Fleiss, J. L. Measuring nominal scale agreement among many raters. Psychol. Bull. 76, 378\u2013382 (1971).","journal-title":"Psychol. Bull."},{"key":"87_CR29","doi-asserted-by":"publisher","first-page":"375","DOI":"10.1007\/s11263-017-1013-y","volume":"126","author":"S Yeung","year":"2018","unstructured":"Yeung, S. et al. Every moment counts: dense detailed labeling of actions in complex videos. Int J. Comput. Vis. 126, 375\u2013389 (2018).","journal-title":"Int J. Comput. Vis."},{"key":"87_CR30","doi-asserted-by":"crossref","unstructured":"He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770\u2013778 (2016).","DOI":"10.1109\/CVPR.2016.90"},{"key":"87_CR31","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky, O. et al. Imagenet large scale visual recognition challenge. Int J. Comput. Vis. 115, 211\u2013252 (2015).","journal-title":"Int J. Comput. Vis."},{"key":"87_CR32","doi-asserted-by":"crossref","unstructured":"Redmon J., Farhadi A. YOLO9000: better, faster, stronger. In Proceedings of IEEE conference on computer vision and pattern recognition. 6517\u20136525 (2017).","DOI":"10.1109\/CVPR.2017.690"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0087-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0087-z","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0087-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,17]],"date-time":"2022-12-17T18:28:17Z","timestamp":1671301697000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0087-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,3,1]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["87"],"URL":"https:\/\/doi.org\/10.1038\/s41746-019-0087-z","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,3,1]]},"assertion":[{"value":"11 October 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 February 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 March 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A.A. serves as the chief executive officer of VisioSafe Inc. and provided the depth sensors purchased for this study at cost. The remaining authors have no potential conflicts of interest, relevant financial interests, activities, relationships, or affiliations to disclose.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"11"}}