{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T23:34:47Z","timestamp":1769729687394,"version":"3.49.0"},"reference-count":55,"publisher":"Springer Science and Business Media LLC","issue":"12","license":[{"start":{"date-parts":[[2023,9,9]],"date-time":"2023-09-09T00:00:00Z","timestamp":1694217600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,9]],"date-time":"2023-09-09T00:00:00Z","timestamp":1694217600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"ICAR - PALERMO"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Ambient Intell Human Comput"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Indoor Environmental Quality (IEQ) concerns several aspects of environmental comforts, such as thermal, visual and acoustics comfort. In particular, IEQ plays a relevant role in workers\u2019 satisfaction since it strongly influences health, well-being, and productivity. Specifically, it has been demonstrated that the furniture configuration in working spaces affects the occupant\u2019s comfort perception. Nevertheless, IEQ has been either neglected or partially addressed in the context of interior design. The contribution of this paper is to introduce a novel method for furniture layout optimisation in terms of IEQ requirements in multi-occupant offices. In particular, we explore the furniture arrangement task as a Multi-Objective Markov Decision Process (MOMDP), which is solved by a reinforcement learning (RL) agent. The goal is to determine optimal workstation positions that maximise workers\u2019 IEQ satisfaction and functional aspects of working spaces under analysis. Firstly, we formulated the furniture layout task as a MOMDP problem by defining reward functions in terms of thermal, acoustics and visual comfort. Then, we train the RL agent to produce optimal\/suboptimal layout patterns through a Q-learning-based algorithm. We conducted experiments in two different offices. The experimental results demonstrated that the proposed multi-objective RL approach is able to determine optimal furniture arrangements that provide a balance among office occupants in terms of IEQ satisfaction. Moreover, numerical results show that the proposed approach can be a valuable tool for evaluating the conformity to the environmental comfort standard of working environments during the furniture layout design phase instead of applying corrections during the post-occupancy evaluation.<\/jats:p>","DOI":"10.1007\/s12652-023-04685-8","type":"journal-article","created":{"date-parts":[[2023,9,9]],"date-time":"2023-09-09T19:01:10Z","timestamp":1694286070000},"page":"16749-16770","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["A multi-objective reinforcement learning approach for furniture arrangement with optimal IEQ in multi-occupant offices"],"prefix":"10.1007","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3266-9617","authenticated-orcid":false,"given":"Patrizia","family":"Ribino","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marina","family":"Bonomolo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,9,9]]},"reference":[{"key":"4685_CR1","unstructured":"Abadi M, Barham P, Chen J, et\u00a0al (2016) $$\\{$$TensorFlow$$\\}$$: a system for $$\\{$$Large-Scale$$\\}$$ machine learning. In: 12th USENIX symposium on operating systems design and implementation (OSDI 16), pp 265\u2013283"},{"key":"4685_CR2","unstructured":"Akazawa Y, Okada Y, Niijima K (2006) Interactive learning interface for automatic 3d scene generation. In: 7th International Conference on Intelligent Games and Simulation, GAME-ON 2006, pp 30\u201335"},{"issue":"6","key":"4685_CR3","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","volume":"34","author":"K Arulkumaran","year":"2017","unstructured":"Arulkumaran K, Deisenroth MP, Brundage M et al (2017) Deep reinforcement learning: a brief survey. IEEE Signal Process Mag 34(6):26\u201338","journal-title":"IEEE Signal Process Mag"},{"key":"4685_CR4","doi-asserted-by":"crossref","unstructured":"Bechikh S, Kessentini M, Said LB et al (2015) Preference incorporation in evolutionary multiobjective optimization: a survey of the state-of-the-art. In: Advances in computers, vol\u00a098. Elsevier, pp 141\u2013207","DOI":"10.1016\/bs.adcom.2015.03.001"},{"issue":"5A","key":"4685_CR5","doi-asserted-by":"publisher","first-page":"1223","DOI":"10.1121\/1.1912760","volume":"50","author":"LL Beranek","year":"1971","unstructured":"Beranek LL, Blazier WE, Figwer JJ (1971) Preferred noise criterion (pnc) curves and their application to rooms. J Acoust Soc Am 50(5A):1223\u20131228","journal-title":"J Acoust Soc Am"},{"key":"4685_CR6","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.enbuild.2018.08.020","volume":"178","author":"VL Castaldo","year":"2018","unstructured":"Castaldo VL, Pigliautile I, Rosso F et al (2018) How subjective and non-physical parameters affect occupants\u2019 environmental comfort perception. Energy Build 178:107\u2013129","journal-title":"Energy Build"},{"key":"4685_CR7","doi-asserted-by":"publisher","first-page":"591","DOI":"10.1016\/j.buildenv.2018.07.049","volume":"143","author":"JH Choi","year":"2018","unstructured":"Choi JH, Lee K (2018) Investigation of the feasibility of poe methodology for a modern commercial office building. Build Environ 143:591\u2013604","journal-title":"Build Environ"},{"key":"4685_CR8","doi-asserted-by":"crossref","unstructured":"Colenberg S, Jylh\u00e4 T, Arkesteijn M (2020) The relationship between interior office space and employee health and well-being\u2013a literature","DOI":"10.1080\/09613218.2019.1710098"},{"issue":"2","key":"4685_CR9","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1109\/4235.996017","volume":"6","author":"K Deb","year":"2002","unstructured":"Deb K, Pratap A, Agarwal S et al (2002) A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans Evol Comput 6(2):182\u2013197","journal-title":"IEEE Trans Evol Comput"},{"key":"4685_CR10","unstructured":"Di X, Yu P (2021) Deep reinforcement learning for producing furniture layout in indoor scenes. arXiv preprint arXiv:2101.07462"},{"key":"4685_CR11","unstructured":"DIAL (2018) Dialux evo 8.2, professional lighting design software. Available at https:\/\/www.dial.de\/en\/dialux-desktop\/"},{"key":"4685_CR12","unstructured":"Falkner S, Klein A, Hutter F (2018) Bohb: Robust and efficient hyperparameter optimization at scale. In: International Conference on Machine Learning, PMLR, pp 1437\u20131446"},{"key":"4685_CR13","unstructured":"Fanger PO, et\u00a0al (1970) Thermal comfort. analysis and applications in environmental engineering. Thermal comfort Analysis and applications in environmental engineering"},{"issue":"6","key":"4685_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2366145.2366154","volume":"31","author":"M Fisher","year":"2012","unstructured":"Fisher M, Ritchie D, Savva M et al (2012) Example-based synthesis of 3d object arrangements. ACM Trans Graph (TOG) 31(6):1\u201311","journal-title":"ACM Trans Graph (TOG)"},{"issue":"4","key":"4685_CR15","doi-asserted-by":"publisher","first-page":"922","DOI":"10.1016\/j.buildenv.2010.10.021","volume":"46","author":"M Frontczak","year":"2011","unstructured":"Frontczak M, Wargocki P (2011) Literature survey on how different factors influence human comfort in indoor environments. Build Environ 46(4):922\u2013937","journal-title":"Build Environ"},{"key":"4685_CR16","doi-asserted-by":"crossref","unstructured":"Germer T, Schwarz M (2009) Procedural arrangement of furniture for real-time walkthroughs. In: Computer Graphics Forum, Wiley Online Library, pp 2068\u20132078","DOI":"10.1111\/j.1467-8659.2009.01351.x"},{"key":"4685_CR17","doi-asserted-by":"crossref","unstructured":"Grondman I, Busoniu L, Lopes GA, et\u00a0al (2012) A survey of actor-critic reinforcement learning: Standard and natural policy gradients. IEEE Trans Syst Man Cybern Part C (Applications and Reviews) 42(6):1291\u20131307","DOI":"10.1109\/TSMCC.2012.2218595"},{"issue":"109","key":"4685_CR18","first-page":"207","volume":"117","author":"H Guo","year":"2020","unstructured":"Guo H, Aviv D, Loyola M et al (2020) On the understanding of the mean radiant temperature within both the indoor and outdoor environment, a critical review. Renew Sustain Energy Rev 117(109):207","journal-title":"Renew Sustain Energy Rev"},{"key":"4685_CR19","unstructured":"Henderson P, Subr K, Ferrari V (2017) Automatic generation of constrained furniture layouts. arXiv e-prints pp arXiv\u20131711"},{"key":"4685_CR20","doi-asserted-by":"crossref","unstructured":"K\u00e1n P, Kaufmann H (2017) Automated interior design using a genetic algorithm. In: Proceedings of the 23rd ACM symposium on virtual reality software and technology, pp 1\u201310","DOI":"10.1145\/3139131.3139135"},{"key":"4685_CR21","doi-asserted-by":"crossref","unstructured":"K\u00e1n P, Kaufmann H (2018) Automatic furniture arrangement using greedy cost minimization. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), IEEE, pp 491\u2013498","DOI":"10.1109\/VR.2018.8448291"},{"key":"4685_CR22","unstructured":"Kuhnle A, Schaarschmidt M, Fricke K (2017) Tensorforce: a tensorflow library for applied reinforcement learning. Web page, https:\/\/github.com\/tensorforce\/tensorforce"},{"key":"4685_CR23","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1016\/j.buildenv.2019.05.002","volume":"157","author":"M Kwon","year":"2019","unstructured":"Kwon M, Rem\u00f8y H, Van den Bogaard M (2019) Influential design factors on occupant satisfaction with indoor environment in workplaces. Build Environ 157:356\u2013365","journal-title":"Build Environ"},{"key":"4685_CR24","doi-asserted-by":"crossref","unstructured":"Leaman A, Bordass B (1999) Productivity in buildings: the\u2018killer\u2019 variables. Build Res Inf 27(1):4\u201319","DOI":"10.1080\/096132199369615"},{"key":"4685_CR25","unstructured":"Lewis HB, Bell L (1994) Industrial noise control, fundamentals and applications, New York: M"},{"issue":"2","key":"4685_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3303766","volume":"38","author":"M Li","year":"2019","unstructured":"Li M, Patil AG, Xu K et al (2019) Grains: Generative recursive autoencoders for indoor scenes. ACM Trans Graph (TOG) 38(2):1\u201316","journal-title":"ACM Trans Graph (TOG)"},{"key":"4685_CR27","doi-asserted-by":"crossref","unstructured":"Li P, Froese TM, Brager G (2018) Post-occupancy evaluation: state-of-the-art analysis and state-of-the-practice review. Build Environ","DOI":"10.1016\/j.buildenv.2018.02.024"},{"issue":"3","key":"4685_CR28","first-page":"385","volume":"45","author":"C Liu","year":"2014","unstructured":"Liu C, Xu X, Hu D (2014) Multiobjective reinforcement learning: a comprehensive overview. IEEE Trans Syst Man Cybern Syst 45(3):385\u2013398","journal-title":"IEEE Trans Syst Man Cybern Syst"},{"issue":"6","key":"4685_CR29","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1145\/2980179.2980223","volume":"35","author":"R Ma","year":"2016","unstructured":"Ma R, Li H, Zou C et al (2016) Action-driven 3d indoor scene evolution. ACM Trans Graph 35(6):173\u20131","journal-title":"ACM Trans Graph"},{"issue":"6","key":"4685_CR30","doi-asserted-by":"publisher","first-page":"853","DOI":"10.1007\/s00158-009-0460-7","volume":"41","author":"RT Marler","year":"2010","unstructured":"Marler RT, Arora JS (2010) The weighted sum method for multi-objective optimization: new insights. Struct Multidiscip Optim 41(6):853\u2013862","journal-title":"Struct Multidiscip Optim"},{"issue":"4","key":"4685_CR31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2010324.1964982","volume":"30","author":"P Merrell","year":"2011","unstructured":"Merrell P, Schkufza E, Li Z et al (2011) Interactive furniture layout using interior design guidelines. ACM Trans Graph (TOG) 30(4):1\u201310","journal-title":"ACM Trans Graph (TOG)"},{"key":"4685_CR32","doi-asserted-by":"crossref","unstructured":"Naeem M, Rizvi STH, Coronato A (2020) A gentle introduction to reinforcement learning and its application in different fields. IEEE Access (2020)","DOI":"10.1109\/ACCESS.2020.3038605"},{"issue":"6","key":"4685_CR33","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1016\/S0378-7788(02)00004-X","volume":"34","author":"BW Olesen","year":"2002","unstructured":"Olesen BW, Parsons K (2002) Introduction to thermal comfort standards and to the proposed new version of en iso 7730. Energy Build 34(6):537\u2013548","journal-title":"Energy Build"},{"key":"4685_CR34","unstructured":"Pham Ngo\u00a0Gia B, Tram\u00a0Loi Q, Quan\u00a0Thanh T, et\u00a0al (2018) NSGA-II Python Library. https:\/\/github.com\/baopng\/NSGA-II"},{"issue":"3","key":"4685_CR35","doi-asserted-by":"publisher","first-page":"264","DOI":"10.1177\/1744259117702882","volume":"41","author":"M Piasecki","year":"2017","unstructured":"Piasecki M, Kostyrko K, Pykacz S (2017) Indoor environmental quality assessment: Part 1: Choice of the indoor environmental quality sub-component models. J Build Phys 41(3):264\u2013289","journal-title":"J Build Phys"},{"key":"4685_CR36","doi-asserted-by":"crossref","unstructured":"Ribino P, Bonomolo M (2021) An rl-based approach for ieq optimization in reorganizing interior spaces for home-working. In: intelligent environments 2021: workshop proceedings of the 17th international conference on intelligent environments, IOS Press, p 179","DOI":"10.3233\/AISE210095"},{"issue":"4","key":"4685_CR37","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1080\/09613218.2020.1775065","volume":"49","author":"MJ Roskams","year":"2021","unstructured":"Roskams MJ, Haynes BP (2021) Testing the relationship between objective indoor environment quality and subjective experiences of comfort. Build Res Inf 49(4):387\u2013398","journal-title":"Build Res Inf"},{"key":"4685_CR38","doi-asserted-by":"crossref","unstructured":"Sabine WC, Egan MD (1994) Collected papers on acoustics","DOI":"10.1121\/1.409944"},{"key":"4685_CR39","unstructured":"Sanchez S, Roux O, Luga H, et\u00a0al (2003) Constraint-based 3d-object layout using a genetic algorithm"},{"key":"4685_CR40","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1016\/j.scs.2018.08.027","volume":"43","author":"D Sant\u2019Anna","year":"2018","unstructured":"Sant\u2019Anna D, Dos Santos P, Vianna N et al (2018) Indoor environmental quality perception and users\u2019 satisfaction of conventional and green buildings in brazil. Sustain Cities Soc 43:95\u2013110","journal-title":"Sustain Cities Soc"},{"key":"4685_CR41","unstructured":"Schulman J, Wolski F, Dhariwal P, et\u00a0al (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347"},{"key":"4685_CR42","unstructured":"Sepp\u00e4nen O, Fisk WJ (2003) A conseptual model to estimate cost effectiveness of the indoor environment improvements. In: Healthy Buildings 2003. Singapore 7.-11.12. 2003. p 368\u2013374"},{"key":"4685_CR43","doi-asserted-by":"publisher","first-page":"1005","DOI":"10.1007\/s11269-020-02485-9","volume":"34","author":"R Tang","year":"2020","unstructured":"Tang R, Li K, Ding W et al (2020) Reference point based multi-objective optimization of reservoir operation: a comparison of three algorithms. Water Resour Manage 34:1005\u20131020","journal-title":"Water Resour Manage"},{"key":"4685_CR44","unstructured":"Tutenel T, Bidarra R, Smelik RM, et\u00a0al (2009) Rule-based layout solving and its application to procedural interior generation. In: CASA workshop on 3D advanced media in gaming and simulation"},{"key":"4685_CR45","unstructured":"UNI E (2011) 12464-1: 2011. Light and lighting Lighting of work places Part 1"},{"key":"4685_CR46","doi-asserted-by":"crossref","unstructured":"Van\u00a0Otterlo M, Wiering M (2012) Reinforcement learning and markov decision processes. In: Reinforcement learning. Springer, p 3\u201342","DOI":"10.1007\/978-3-642-27645-3_1"},{"issue":"1","key":"4685_CR47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40201-014-0113-7","volume":"12","author":"K Vimalanathan","year":"2014","unstructured":"Vimalanathan K, Babu TR (2014) The effect of indoor office environment on the work performance, health and well-being of office workers. J Environ Health Sci Eng 12(1):1\u20138","journal-title":"J Environ Health Sci Eng"},{"key":"4685_CR48","doi-asserted-by":"crossref","unstructured":"Vitsas N, Papaioannou G, Gkaravelis A, et\u00a0al (2020) Illumination-guided furniture layout optimization. In: Computer Graphics Forum, Wiley Online Library, pp 291\u2013301","DOI":"10.1111\/cgf.13930"},{"issue":"6","key":"4685_CR49","first-page":"1","volume":"39","author":"H Wang","year":"2020","unstructured":"Wang H, Liang W, Yu LF (2020) Scene mover: Automatic move planning for scene arrangement by deep reinforcement learning. ACM Trans Graph (TOG) 39(6):1\u201315","journal-title":"ACM Trans Graph (TOG)"},{"issue":"4","key":"4685_CR50","first-page":"1","volume":"37","author":"K Wang","year":"2018","unstructured":"Wang K, Savva M, Chang AX et al (2018) Deep convolutional priors for indoor scene synthesis. ACM Trans Graph (TOG) 37(4):1\u201314","journal-title":"ACM Trans Graph (TOG)"},{"issue":"3","key":"4685_CR51","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1109\/TEVC.2017.2778560","volume":"22","author":"S Wang","year":"2017","unstructured":"Wang S, Ali S, Yue T et al (2017) Integrating weight assignment strategies with nsga-ii for supporting user preference multiobjective optimization. IEEE Trans Evol Comput 22(3):378\u2013393","journal-title":"IEEE Trans Evol Comput"},{"issue":"3\u20134","key":"4685_CR52","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","volume":"8","author":"CJ Watkins","year":"1992","unstructured":"Watkins CJ, Dayan P (1992) Q-learning. Machine learning 8(3\u20134):279\u2013292","journal-title":"Machine learning"},{"key":"4685_CR53","doi-asserted-by":"crossref","unstructured":"Yamakawa T, Dobashi Y, Okabe M, et\u00a0al (2017) Computer simulation of furniture layout when moving from one house to another. In: Proceedings of the 33rd Spring Conference on Computer Graphics, pp 1\u20138","DOI":"10.1145\/3154353.3154356"},{"key":"4685_CR54","doi-asserted-by":"crossref","unstructured":"Yu LF, Yeung SK, Tang CK, et\u00a0al (2011) Make it home: automatic optimization of furniture arrangement. In: ACM Transactions on Graphics (TOG)-Proceedings of ACM SIGGRAPH 2011, v 30,(4), July 2011, article no 86 30(4)","DOI":"10.1145\/2010324.1964981"},{"issue":"6","key":"4685_CR55","first-page":"1","volume":"35","author":"X Zhao","year":"2016","unstructured":"Zhao X, Hu R, Guerrero P et al (2016) Relationship templates for creating scene variations. ACM Trans Graph (TOG) 35(6):1\u201313","journal-title":"ACM Trans Graph (TOG)"}],"container-title":["Journal of Ambient Intelligence and Humanized Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12652-023-04685-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12652-023-04685-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12652-023-04685-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,27]],"date-time":"2023-12-27T11:38:22Z","timestamp":1703677102000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12652-023-04685-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,9]]},"references-count":55,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["4685"],"URL":"https:\/\/doi.org\/10.1007\/s12652-023-04685-8","relation":{},"ISSN":["1868-5137","1868-5145"],"issn-type":[{"value":"1868-5137","type":"print"},{"value":"1868-5145","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,9]]},"assertion":[{"value":"29 December 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 August 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 September 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}