{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T18:56:52Z","timestamp":1780081012961,"version":"3.54.0"},"reference-count":26,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,6,20]],"date-time":"2019-06-20T00:00:00Z","timestamp":1560988800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["723521"],"award-info":[{"award-number":["723521"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>Urban areas can be considered the ground for the challenges related to the UN\u2019s Sustainable Development Goals (SDGs). The objective of shaping cities as human settlement that will see a more inclusive, safe, resilient, and sustainable future is often argued in literature as an issue dependent on behavioral change of inhabitants in urban areas. In this paper, the authors question if experimental applications based on gamification can co-produce more sustainable neighborhoods through an impact evaluation method that departs from individual choices within the complex of urban mobility. This investigation is carried out within MUV (Mobility Urban Values), an EU research and innovation project, which aims to trigger more sustainable urban mobility in six pilot cities. This article describes the critical method of validation, an impact assessment of the MUV experimental gamification in the pilot cities, in order to represent a proof for future urban strategies. This methodological approach is based on an evaluation structured on indicators of both impact and process suitable for urban contexts. As based on six pilot cities, with possibilities for transferability to other contexts and scalability to other cities, the method represents a reference work for the evaluation of similar experimental applications.<\/jats:p>","DOI":"10.3390\/systems7020030","type":"journal-article","created":{"date-parts":[[2019,6,20]],"date-time":"2019-06-20T10:49:59Z","timestamp":1561027799000},"page":"30","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Measuring the Change Towards More Sustainable Mobility: MUV Impact Evaluation Approach"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2747-9093","authenticated-orcid":false,"given":"Brunella","family":"Caroleo","sequence":"first","affiliation":[{"name":"LINKS Foundation (Leading Innovation &amp; Knowledge for Society), via Pier Carlo Boggio, 61, 10138 Turin, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicola","family":"Morelli","sequence":"additional","affiliation":[{"name":"Department of Architecture, Design and Media Technology, Aalborg University, 9000 Aalborg, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0683-7816","authenticated-orcid":false,"given":"Enza","family":"Lissandrello","sequence":"additional","affiliation":[{"name":"Department of Planning, Aalborg University, 9000 Aalborg, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7431-6655","authenticated-orcid":false,"given":"Andrea","family":"Vesco","sequence":"additional","affiliation":[{"name":"LINKS Foundation (Leading Innovation &amp; Knowledge for Society), via Pier Carlo Boggio, 61, 10138 Turin, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Salvatore","family":"Di Dio","sequence":"additional","affiliation":[{"name":"PUSH Design Lab, 90133 Palermo, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefania","family":"Mauro","sequence":"additional","affiliation":[{"name":"LINKS Foundation (Leading Innovation &amp; Knowledge for Society), via Pier Carlo Boggio, 61, 10138 Turin, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,6,20]]},"reference":[{"key":"ref_1","unstructured":"United Nations General Assembly (2019, April 09). Transforming Our World: The 2030 Agenda for Sustainable Development (A\/RES\/70\/1), 21 October 2015. Available online: https:\/\/www.refworld.org\/docid\/57b6e3e44.html."},{"key":"ref_2","unstructured":"Wendel, S. (2013). Designing for Behavior Change: Applying Psychology and Behavioral Economics, O\u2019Reilly Media, Inc.. [1st ed.]."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Strengers, Y., and Maller, C. (2015). Social Practices, Intervention and Sustainability\u2014Beyond Behaviour Change, Routledge. [1st ed.].","DOI":"10.4324\/9781315816494"},{"key":"ref_4","unstructured":"Caillois, R. (2001). Man, Play and Games, University of Illinois."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Deterding, S., Sicart, M., Nacke, L., O\u2019Hara, K., and Dixon, D. (2011, January 7\u201312). Gamification: Using game-design elements in non-gaming contexts. Proceedings of the International Conference on Human Factors in Computing Systems (CHI 2011), Extended Abstracts Volume, Vancouver, BC, Canada.","DOI":"10.1145\/1979742.1979575"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hamari, J., Koivisto, J., and Sarsa, H. (2014, January 6\u20139). Does Gamification Work?\u2014A Literature Review of Empirical Studies on Gamification. Proceedings of the 47th Hawaii International Conference on System Sciences, Waikoloa, HI, USA.","DOI":"10.1109\/HICSS.2014.377"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"15","DOI":"10.4018\/IJGBL.2018070102","article-title":"A systematic review of gamification research: In pursuit of homo ludens","volume":"8","author":"Bozkurt","year":"2018","journal-title":"Int. J. Game-Based Learn."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Di Dio, S., Peri, G., Rizzo, G., and Vinci, I. (2019). Design, technology and social innovation: the serious game of Traffic O2. Participatory Design Theory: Using Technology and Social Media to Foster Civic Engagement, Routledge.","DOI":"10.4324\/9781315110332-11"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Groos, K. (1901). The Play of Man, D. Appleton and Company.","DOI":"10.1037\/13084-000"},{"key":"ref_10","unstructured":"Suits, B.H. (2005). The Grasshopper: Games, Life and Utopia, Broadview Press Ltd.. [3rd ed.]."},{"key":"ref_11","unstructured":"Csikszentmihalyi, M. (2008). Flow: The Psychology of Optimal Experience, HarperCollins Publishers Inc.. [1st ed.]."},{"key":"ref_12","unstructured":"(2019, April 09). MUV Mobility Urban Values. Available online: https:\/\/www.muv2020.eu\/."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lissandrello, E., Morelli, N., Schillaci, D., and Di Dio, S. (2018). Urban innovation through co\u2013design scenarios. Conference on Smart Learning Ecosystems and Regional Development, Springer.","DOI":"10.1007\/978-3-319-92022-1_10"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1080\/1464935042000293170","article-title":"Reframing public participation: strategies for the 21st century","volume":"5","author":"Innes","year":"2004","journal-title":"Plan. Theory Pract."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1162\/DESI_a_00248","article-title":"Making things happen: Social innovation and design","volume":"30","author":"Manzini","year":"2014","journal-title":"Des. Issues"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1177\/1350507611426239","article-title":"The origins and conceptualizations of \u2018triple\u2013loop\u2019 learning: A critical review","volume":"43","author":"Tosey","year":"2012","journal-title":"Manag. Learn."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Nabatchi, T., and Leighninger, M. (2015). Public Participation for 21st Century Democracy, Wiley.","DOI":"10.1002\/9781119154815"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Gentile, M., Allegra, M., and S\u00f6bke, H. (2019). MUV: A game to encourage sustainable mobility habits. Lecture Notes in Computer Science Vol. 11385, Proceedings of the Games and Learning Alliance Conference (GALA 2018), Palermo, Italy, 5\u20137 December 2018, Springer.","DOI":"10.1007\/978-3-030-11548-7"},{"key":"ref_19","unstructured":"(2019, April 09). CIVITAS SATELLITE D2.3\u2014Refined CIVITAS Process and Impact Evaluation Framework (31.08.2017). Available online: http:\/\/exchange-hub.civitas.eu\/system\/files\/satellite_d2.3_refined_evaluation_framework-final_version_2017-08-31.pdf."},{"key":"ref_20","unstructured":"European Commission (2019, April 09). Impact Assessment Guidelines (2009). Available online: http:\/\/ec.europa.eu\/smart-regulation\/impact\/commission_guidelines\/docs\/iag_2009_en.pdf."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"55","DOI":"10.3141\/2242-07","article-title":"Framework for Assessing Indicators of Environmental Impacts in the Transport Sector","volume":"2242","author":"Joumard","year":"2011","journal-title":"Trans. Res. Rec."},{"key":"ref_22","unstructured":"Bosch, P., Jongeneel, S., Rovers, V., Neumann, H.M., Airaksinen, M., and Huovila, A. (2019, April 10). CITYkeys Indicators for Smart City Projects and Smart Cities (2017). Available online: http:\/\/nws.eurocities.eu\/MediaShell\/media\/CITYkeystheindicators.pdf."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/j.cities.2016.09.009","article-title":"What are the differences between sustainable and smart cities?","volume":"60","author":"Ahvenniemi","year":"2017","journal-title":"Cities"},{"key":"ref_24","unstructured":"(2019, April 10). TrafficO2. Available online: http:\/\/www.traffico2.com\/data\/en\/index.html."},{"key":"ref_25","unstructured":"Evaluation Unit DEVCO (2019, April 10). Evaluation Methodological Approach (2015). Available online: https:\/\/europa.eu\/capacity4dev\/evaluation_guidelines\/minisite\/en-methodological-bases-and-approach\/evaluation-tools\/context-indicators."},{"key":"ref_26","unstructured":"European Environment Agency (2016). EMEP\/EEA Air Pollutant Emission Inventory Guidebook 2016: Technical Guidance to Prepare National Emission Inventories (EEA Report No 21\/2016), Publications Office of the European Union."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/7\/2\/30\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:00:01Z","timestamp":1760187601000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/7\/2\/30"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,20]]},"references-count":26,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,6]]}},"alternative-id":["systems7020030"],"URL":"https:\/\/doi.org\/10.3390\/systems7020030","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,6,20]]}}}