{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T23:10:23Z","timestamp":1773097823806,"version":"3.50.1"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2025,3,11]],"date-time":"2025-03-11T00:00:00Z","timestamp":1741651200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,3,11]],"date-time":"2025-03-11T00:00:00Z","timestamp":1741651200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000923","name":"Australian Research Council","doi-asserted-by":"publisher","award":["LP140100152"],"award-info":[{"award-number":["LP140100152"]}],"id":[{"id":"10.13039\/501100000923","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001249","name":"University of Tasmania","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001249","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Ann Oper Res"],"published-print":{"date-parts":[[2025,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>We study the patient assignment scheduling (PAS) problem in a random environment that arises in the management of patient flow in hospital systems, due to the stochastic nature of the arrivals as well as the length of stay (LoS) distribution. At the start of each time period, emergency patients in the waiting area of a hospital system need to be admitted to relevant wards. Decisions may involve allocation to less suitable wards, or transfers of the existing inpatients to accommodate higher priority cases when wards are at full capacity. However, the LoS for patients in non-primary wards may increase, potentially leading to long-term congestion. To assist with decision-making in this PAS problem, we construct a discrete-time Markov decision process over an infinite horizon, with multiple patient types and multiple wards. Since the instances of realistic size of this problem are not easy to solve, we develop numerical methods based on approximate dynamic programming. We demonstrate the application potential of our methodology under practical considerations with numerical examples, using parameters obtained from data at a tertiary referral hospital in Australia. We gain valuable insights, such as the number of patients in non-primary wards, the number of transferred patients, and the number of patients redirected to other facilities, under different policies that enhance the system\u2019s performance. This approach allows for more realistic assumptions and can also help determine the appropriate size of wards for different patient types within the hospital system.<\/jats:p>","DOI":"10.1007\/s10479-025-06553-4","type":"journal-article","created":{"date-parts":[[2025,3,11]],"date-time":"2025-03-11T13:08:05Z","timestamp":1741698485000},"page":"1493-1531","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Markov decision process and approximate dynamic programming for a patient assignment scheduling problem"],"prefix":"10.1007","volume":"347","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3898-3957","authenticated-orcid":false,"given":"Ma\u0142gorzata M.","family":"O\u2019Reilly","sequence":"first","affiliation":[]},{"given":"Sebastian","family":"Krasnicki","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5360-7514","authenticated-orcid":false,"given":"James","family":"Montgomery","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9277-9370","authenticated-orcid":false,"given":"Mojtaba","family":"Heydar","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2276-8999","authenticated-orcid":false,"given":"Richard","family":"Turner","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3341-6793","authenticated-orcid":false,"given":"Pieter","family":"Van Dam","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9709-7691","authenticated-orcid":false,"given":"Peter","family":"Maree","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,3,11]]},"reference":[{"issue":"4","key":"6553_CR1","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1016\/0167-6377(92)90050-D","volume":"12","author":"J Abate","year":"1992","unstructured":"Abate, J., & Whitt, W. (1992). Numerical inversion of probability generating functions. Operations Research Letters, 12(4), 245\u2013251.","journal-title":"Operations Research Letters"},{"issue":"2","key":"6553_CR2","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1080\/15326349.2020.1742161","volume":"36","author":"AK Abera","year":"2020","unstructured":"Abera, A. K., O\u2019Reilly, M. M., Fackrell, M., Holland, B. R., & Heydar, M. (2020). On the decision support model for the patient admission scheduling problem with random arrivals and departures: A solution approach. Stochastic Models, 36(2), 312\u2013336.","journal-title":"Stochastic Models"},{"issue":"2","key":"6553_CR3","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1007\/s43069-020-0009-6","volume":"1","author":"A Andersen","year":"2020","unstructured":"Andersen, A., Stidsen, T., & Reinhardt, L. (2020). Simulation-based rolling horizon scheduling for operating theatres. Operations Research Forum, 1(2), 20.","journal-title":"Operations Research Forum"},{"issue":"7203","key":"6553_CR4","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1136\/bmj.319.7203.155","volume":"318","author":"A Bagust","year":"1999","unstructured":"Bagust, A., Place, M., & Posnett, J. W. (1999). Dynamics of bed use in accommodating emergency admissions: Stochastic simulation model. British Medical Journal, 318(7203), 155\u2013158.","journal-title":"British Medical Journal"},{"issue":"4","key":"6553_CR5","doi-asserted-by":"publisher","first-page":"666","DOI":"10.1007\/s10729-021-09560-6","volume":"24","author":"J Bai","year":"2021","unstructured":"Bai, J., F\u00fcgener, A., G\u00f6nsch, J., Brunner, J., & Blobner, M. (2021). Managing admission and discharge processes in intensive care units. Health Care Management Science, 24(4), 666\u2013685.","journal-title":"Health Care Management Science"},{"issue":"1","key":"6553_CR6","doi-asserted-by":"publisher","first-page":"42","DOI":"10.5694\/j.1326-5377.2010.tb03401.x","volume":"192","author":"CA Bain","year":"2010","unstructured":"Bain, C. A., Taylor, P. G., McDonnell, G., & Georgiou, A. (2010). Myths of ideal hospital occupancy. Medical Journal of Australia, 192(1), 42\u201343.","journal-title":"Medical Journal of Australia"},{"issue":"12","key":"6553_CR7","doi-asserted-by":"publisher","first-page":"1907","DOI":"10.1111\/poms.12395","volume":"24","author":"C Barz","year":"2015","unstructured":"Barz, C., & Rajaram, K. (2015). Elective patient admission and scheduling under multiple resource constraints. Production and Operations Management, 24(12), 1907\u20131930.","journal-title":"Production and Operations Management"},{"issue":"3","key":"6553_CR8","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1007\/s10732-011-9192-0","volume":"18","author":"B Bilgin","year":"2012","unstructured":"Bilgin, B., Demeester, P., Misir, M., Vancroonenburg, W., & Vanden Berghe, G. (2012). One hyperheuristic approach to two timetabling problems in health care. Journal of Heuristics, 18(3), 401\u2013434.","journal-title":"Journal of Heuristics"},{"issue":"2","key":"6553_CR9","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1007\/s11009-016-9533-4","volume":"19","author":"K Butler","year":"2017","unstructured":"Butler, K., & Stephens, M. A. (2017). The distribution of a sum of independent binomial random variables. Methodology and Computing in Applied Probability, 19(2), 557\u2013571.","journal-title":"Methodology and Computing in Applied Probability"},{"issue":"2","key":"6553_CR10","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1111\/jnu.12055","volume":"46","author":"EJ Carter","year":"2014","unstructured":"Carter, E. J., Pouch, S. M., & Larson, E. L. (2014). The relationship between emergency department crowding and patient outcomes: A systematic review. Journal of Nursing Scholarship, 46(2), 106\u2013115.","journal-title":"Journal of Nursing Scholarship"},{"key":"6553_CR11","unstructured":"Cummings, E., Ellis, L., Georgiou, A., K. E., Showell, C., & Turner, P. (2012). An evidence-based review and training resource on smooth patient flow. Ministry of Health, New South Wales Government, Australia (pp. 1\u201334)."},{"issue":"4","key":"6553_CR12","doi-asserted-by":"publisher","first-page":"894","DOI":"10.1287\/msom.2018.0730","volume":"21","author":"JG Dai","year":"2019","unstructured":"Dai, J. G., & Shi, P. (2019). Inpatient overflow: An approximate dynamic programming approach. Manufacturing and Service Operations Management, 21(4), 894\u2013911.","journal-title":"Manufacturing and Service Operations Management"},{"issue":"1","key":"6553_CR13","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1016\/j.artmed.2009.09.001","volume":"48","author":"P Demeester","year":"2012","unstructured":"Demeester, P., Souffriau, W., De Causmaecker, P., & Vanden Berghe, G. (2012). A hybrid tabu search algorithm for automatically assigning patients to beds. Artificial Intelligence in Medicine, 48(1), 61\u201370.","journal-title":"Artificial Intelligence in Medicine"},{"issue":"3","key":"6553_CR14","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1007\/s10729-016-9388-9","volume":"21","author":"Y Gocgun","year":"2018","unstructured":"Gocgun, Y. (2018). Simulation-based approximate policy iteration for dynamic patient scheduling for radiation therapy. Health Care Management Science, 21(3), 317\u2013325.","journal-title":"Health Care Management Science"},{"issue":"4","key":"6553_CR15","doi-asserted-by":"publisher","first-page":"957","DOI":"10.1111\/hsc.13130","volume":"29","author":"R Grant","year":"2021","unstructured":"Grant, R., Smith, A. K. J., Newett, L., Nash, M., Turner, R., & Owen, L. (2021). Tasmanian healthcare professionals\u2019 and students\u2019 capacity for LGBTI + inclusive care: A qualitative inquiry. Health and Social Care in the Community, 29(4), 957\u2013966.","journal-title":"Health and Social Care in the Community"},{"issue":"1","key":"6553_CR16","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1164\/rccm.201106-1084OC","volume":"185","author":"WB Hall","year":"2012","unstructured":"Hall, W. B., Willis, L. E., Medvedev, S., & Carson, S. S. (2012). The implications of long-term acute care hospital transfer practices for measures of in-hospital mortality and length of stay. American Journal of Respiratory and Critical Care Medicine, 185(1), 53\u201357.","journal-title":"American Journal of Respiratory and Critical Care Medicine"},{"key":"6553_CR17","first-page":"1","volume":"8","author":"M Heydar","year":"2021","unstructured":"Heydar, M., O\u2019Reilly, M. M., Trainer, E., Fackrell, M., Taylor, P., & Tirdad, A. (2021). A stochastic model for the patient-bed assignment problem with random arrivals and departures. Annals of Operations Research, 8, 1\u201333.","journal-title":"Annals of Operations Research"},{"key":"6553_CR18","volume-title":"Dynamic programming and Markov processes","author":"R Howard","year":"1960","unstructured":"Howard, R. (1960). Dynamic programming and Markov processes. The MIT Press."},{"issue":"11","key":"6553_CR19","doi-asserted-by":"publisher","first-page":"804","DOI":"10.7326\/0003-4819-149-11-200812020-00006","volume":"149","author":"E Howell","year":"2008","unstructured":"Howell, E., Bessman, E., Kravet, S., Kolodner, K., Marshall, R., & Wright, S. (2008). Active bed management by hospitalists and emergency department throughput. Annals of Internal Medicine, 149(11), 804\u2013810.","journal-title":"Annals of Internal Medicine"},{"issue":"1\u20132","key":"6553_CR20","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1007\/s10696-015-9219-1","volume":"28","author":"PJH Hulshof","year":"2016","unstructured":"Hulshof, P. J. H., Mes, M. R. K., Boucherie, R. J., & Hans, E. W. (2016). Patient admission planning using approximate dynamic programming. Flexible Services and Manufacturing Journal, 28(1\u20132), 30\u201361.","journal-title":"Flexible Services and Manufacturing Journal"},{"key":"6553_CR21","unstructured":"Krasnicki, S. (2021). Modelling of optimal decision making in healthcare systems, Honours Thesis, The University of Tasmania."},{"key":"6553_CR22","unstructured":"Markus, L., & Sandson, B. (2020). Australia\u2019s hospitals at a glance 2018-19. Cat. no. HSE 247. Canberra: AIHW. Australian Institute of Health and Welfare."},{"issue":"8","key":"6553_CR23","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1371\/journal.pone.0203316","volume":"13","author":"C Morley","year":"2018","unstructured":"Morley, C., Unwin, M., Peterson, G. M., Stankovich, J., & Kinsman, L. (2018). Emergency department crowding: A systematic review of causes, consequences and solutions. PLoS ONE, 13(8), 78.","journal-title":"PLoS ONE"},{"issue":"3","key":"6553_CR24","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1002\/nav.20347","volume":"56","author":"W Powell","year":"2009","unstructured":"Powell, W. (2009). What you should know about approximate dynamic programming. Naval Research Logistics, 56(3), 239\u2013249.","journal-title":"Naval Research Logistics"},{"issue":"3","key":"6553_CR25","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1002\/nav.20347","volume":"56","author":"WB Powell","year":"2009","unstructured":"Powell, W. B. (2009). What you should know about approximate dynamic programming. Naval Research Logistics, 56(3), 239\u2013249.","journal-title":"Naval Research Logistics"},{"key":"6553_CR26","doi-asserted-by":"publisher","DOI":"10.1002\/9781118029176","volume-title":"Approximate dynamic programming: Solving the curses of dimensionality","author":"W Powell","year":"2011","unstructured":"Powell, W. (2011). Approximate dynamic programming: Solving the curses of dimensionality (2nd ed.). Wiley.","edition":"2"},{"issue":"1\u20132","key":"6553_CR27","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1007\/s10479-012-1077-6","volume":"241","author":"WB Powell","year":"2016","unstructured":"Powell, W. B. (2016). Perspectives of approximate dynamic programming. Annals of Operations Research, 241(1\u20132), 319\u2013356.","journal-title":"Annals of Operations Research"},{"key":"6553_CR28","doi-asserted-by":"publisher","DOI":"10.1002\/9780470316887","volume-title":"Markov decision processes: Discrete stochastic dynamic programming","author":"ML Puterman","year":"1994","unstructured":"Puterman, M. L. (1994). Markov decision processes: Discrete stochastic dynamic programming. Wiley."},{"issue":"3","key":"6553_CR29","doi-asserted-by":"publisher","first-page":"635","DOI":"10.1287\/opre.2016.1574","volume":"65","author":"S Samiedaluie","year":"2017","unstructured":"Samiedaluie, S., Kucukyazici, B., Verter, V., & Zhang, D. (2017). Managing patient admissions in a neurology ward. Operations Research, 65(3), 635\u2013656.","journal-title":"Operations Research"},{"key":"6553_CR30","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1007\/s10479-013-1478-1","volume":"239","author":"W Vancroonenburg","year":"2016","unstructured":"Vancroonenburg, W., De Causmaecker, P., & Vanden Berghe, G. (2016). A study of decision support models for online patient-to-room assignment planning. Annals of Operations Research, 239, 253\u2013271.","journal-title":"Annals of Operations Research"},{"issue":"2","key":"6553_CR31","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1016\/j.ejor.2019.01.036","volume":"276","author":"J Zhang","year":"2019","unstructured":"Zhang, J., Dridi, M., & El Moudni, A. (2019). A two-level optimization model for elective surgery scheduling with downstream capacity constraints. European Journal of Operational Research, 276(2), 602\u2013613.","journal-title":"European Journal of Operational Research"},{"key":"6553_CR32","first-page":"132","volume":"8","author":"J Zhang","year":"2021","unstructured":"Zhang, J., Dridi, M., & El Moudni, A. (2021). An approximate dynamic programming approach to the admission control of elective patients. Computers and Operations Research, 8, 132.","journal-title":"Computers and Operations Research"}],"container-title":["Annals of Operations Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10479-025-06553-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10479-025-06553-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10479-025-06553-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T16:23:24Z","timestamp":1746721404000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10479-025-06553-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,11]]},"references-count":32,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,4]]}},"alternative-id":["6553"],"URL":"https:\/\/doi.org\/10.1007\/s10479-025-06553-4","relation":{},"ISSN":["0254-5330","1572-9338"],"issn-type":[{"value":"0254-5330","type":"print"},{"value":"1572-9338","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,11]]},"assertion":[{"value":"21 March 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 March 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no Conflict of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}