{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:21:11Z","timestamp":1760059271607,"version":"build-2065373602"},"reference-count":22,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,6,5]],"date-time":"2025-06-05T00:00:00Z","timestamp":1749081600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Jungseok Logistics Foundation"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>Crowdshipping establishes a short-term connection between shippers and individual carriers, bridging the service requirements in last-mile logistics. From the perspective of a carrier operating multiple vehicles, this study considers the challenge of maximizing profits by optimizing bid strategies for delivery prices and transportation conditions in the context of bid-based crowdshipping services. We considered two types of bid strategies: a price bid that adjusts the RFQ freight charge and a multi-attribute bid that scores both price and service quality. We formulated the problem as a Markov decision process (MDP) to represent uncertain and sequential decision-making procedures. Furthermore, given the complexity of the newly proposed problem, which involves multiple vehicles, route optimizations, and multiple attributes of bids, we employed a reinforcement learning (RL) approach that learns an optimal bid strategy. Finally, numerical experiments are conducted to illustrate the superiority of the bid strategy learned by RL and to analyze the behavior of the bid strategy. A numerical analysis shows that the bid strategies learned by RL provide more rewards and lower costs than other benchmark strategies. In addition, a comparison of price-based and multi-attribute strategies reveals that the choice of appropriate strategies is situation-dependent.<\/jats:p>","DOI":"10.3390\/systems13060440","type":"journal-article","created":{"date-parts":[[2025,6,5]],"date-time":"2025-06-05T11:04:06Z","timestamp":1749121446000},"page":"440","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Reinforcement Learning Model for Optimizing Bid Price and Service Quality in Crowdshipping"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3583-5879","authenticated-orcid":false,"given":"Daiki","family":"Min","sequence":"first","affiliation":[{"name":"School of Business, Ewha Womans University, Seoul 03760, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seokgi","family":"Lee","sequence":"additional","affiliation":[{"name":"Rayen School of Engineering, Youngstown State University, Youngstown, OH 44555, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3614-9379","authenticated-orcid":false,"given":"Yuncheol","family":"Kang","sequence":"additional","affiliation":[{"name":"School of Business, Ewha Womans University, Seoul 03760, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1111\/jbl.12173","article-title":"Crowdsourcing last mile delivery: Strategic implications and future research directions","volume":"39","author":"Castillo","year":"2018","journal-title":"J. Bus. Logist."},{"key":"ref_2","unstructured":"Gartner (2023, December 01). Adopt Crowdsourcing Last-Mile Platforms as a Key Component of Your Logistics Strategy. Available online: https:\/\/www.gartner.com\/en\/documents\/4008043."},{"key":"ref_3","first-page":"100614","article-title":"Crowd-shipping delivery performance from bidding to delivering","volume":"41","author":"Ermagun","year":"2021","journal-title":"Res. Transp. Bus. Manag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"101063","DOI":"10.1016\/j.seps.2021.101063","article-title":"Crowdshipping in last mile deliveries: Operational challenges and research opportunities","volume":"78","author":"Pourrahmani","year":"2021","journal-title":"Socio-Econ. Plan. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"348","DOI":"10.1016\/j.tre.2019.03.021","article-title":"Freight transportation service procurement: A literature review and future research opportunities in omnichannel E-commerce","volume":"125","author":"Lafkihi","year":"2019","journal-title":"Transp. Res. Part E Logist. Transp. Rev."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"105110","DOI":"10.1016\/j.trc.2025.105110","article-title":"A market-based efficient matching mechanism for crowdsourced delivery systems with demand\/supply elasticities","volume":"174","author":"Oyama","year":"2025","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"103451","DOI":"10.1016\/j.tre.2024.103451","article-title":"Outsourcing service price for crowd-shipping based on on-demand mobility services","volume":"183","author":"Peng","year":"2024","journal-title":"Transp. Res. Part E Logist. Transp. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"108914","DOI":"10.1016\/j.ijpe.2023.108914","article-title":"A game-theoretic model for crowd-shipping operations with profit improvement strategies","volume":"262","author":"Xiao","year":"2023","journal-title":"Int. J. Prod. Econ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"102802","DOI":"10.1016\/j.trb.2023.102802","article-title":"Crowd-shipping as a Service: Game-based operating strategy design and analysis","volume":"176","author":"Xiao","year":"2023","journal-title":"Transp. Res. Part B Methodol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1016\/j.trb.2021.04.010","article-title":"Exact and hybrid heuristic methods to solve the combinatorial bid construction problem with stochastic prices in truckload transportation services procurement auctions","volume":"149","author":"Hammami","year":"2021","journal-title":"Transp. Res. Part B Methodol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.trb.2014.11.012","article-title":"Bid price optimization for truckload carriers in simultaneous transportation procurement auctions","volume":"73","author":"Kuyzu","year":"2015","journal-title":"Transp. Res. Part B Methodol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"9157","DOI":"10.1109\/TITS.2021.3091692","article-title":"A bid generation problem in truckload transportation service procurement considering multiple periods and uncertainty: Model and benders decomposition approach","volume":"23","author":"Lyu","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_13","unstructured":"Kang, Y. An order control policy in crowdsourced parcel pickup and delivery service. Proceedings of the Advances in Production\nManagement Systems. Smart Manufacturing for Industry 4.0: IFIP WG 5.7 International Conference, APMS 2018, Seoul, Republic of\nKorea, 26\u201330 August 2018, Springer. Proceedings, Part II."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1016\/j.cie.2019.04.044","article-title":"Learning-based logistics planning and scheduling for crowdsourced parcel delivery","volume":"132","author":"Kang","year":"2019","journal-title":"Comput. Ind. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"107596","DOI":"10.1016\/j.cie.2021.107596","article-title":"Reinforcement learning enabled dynamic bidding strategy for instant delivery trading","volume":"160","author":"Guo","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"107042","DOI":"10.1016\/j.cie.2020.107042","article-title":"A learning-based approach for dynamic freight brokerages with transfer and territory-based assignment","volume":"153","author":"Min","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"108233","DOI":"10.1016\/j.ijpe.2021.108233","article-title":"Efficient and truthful multi-attribute auctions for crowdsourced delivery","volume":"240","author":"Xiao","year":"2021","journal-title":"Int. J. Prod. Econ."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1109\/TMC.2022.3219881","article-title":"Auction-based crowdsourced first and last mile logistics","volume":"23","author":"Li","year":"2022","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"40","DOI":"10.3141\/1833-06","article-title":"Combinatorial auctions for transportation service procurement: The carrier perspective","volume":"1833","author":"Song","year":"2003","journal-title":"Transp. Res. Rec."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/j.omega.2018.06.010","article-title":"An optimal efficient multi-attribute auction for transportation procurement with carriers having multi-unit supplies","volume":"83","author":"Zhang","year":"2019","journal-title":"Omega"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"628","DOI":"10.1109\/TASE.2014.2339281","article-title":"A dynamic algorithm for distributed feedback control for manufacturing production, capacity, and maintenance","volume":"12","author":"Lee","year":"2014","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/0278-6125(94)90025-6","article-title":"Real-time distributed scheduling of heterarchical manufacturing systems","volume":"13","author":"Duffie","year":"1994","journal-title":"J. Manuf. Syst."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/13\/6\/440\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:47:16Z","timestamp":1760032036000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/13\/6\/440"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,5]]},"references-count":22,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["systems13060440"],"URL":"https:\/\/doi.org\/10.3390\/systems13060440","relation":{},"ISSN":["2079-8954"],"issn-type":[{"type":"electronic","value":"2079-8954"}],"subject":[],"published":{"date-parts":[[2025,6,5]]}}}