{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:02:11Z","timestamp":1760144531828,"version":"build-2065373602"},"reference-count":48,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2024,4,28]],"date-time":"2024-04-28T00:00:00Z","timestamp":1714262400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62272198","2021B1111600001","2023CXB022"],"award-info":[{"award-number":["62272198","2021B1111600001","2023CXB022"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Guangdong Provincial Science and Technology Plan Project","award":["62272198","2021B1111600001","2023CXB022"],"award-info":[{"award-number":["62272198","2021B1111600001","2023CXB022"]}]},{"name":"Outstanding Innovative Talents Cultivation Funded Programs for Doctoral Students of Jinan University","award":["62272198","2021B1111600001","2023CXB022"],"award-info":[{"award-number":["62272198","2021B1111600001","2023CXB022"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>Real-time bidding has become a major means for online advertisement exchange. The goal of a real-time bidding strategy is to maximize the benefits for stakeholders, e.g., click-through rates or conversion rates. However, in practise, the optimal bidding strategy for real-time bidding is constrained by at least three aspects: cost-effectiveness, the dynamic nature of market prices, and the issue of missing bidding values. To address these challenges, we propose Imagine and Imitate Bidding (IIBidder), which includes Strategy Imitation and Imagination modules, to generate cost-effective bidding strategies under partially observable price landscapes. Experimental results on the iPinYou and YOYI datasets demonstrate that IIBidder reduces investment costs, optimizes bidding strategies, and improves future market price predictions.<\/jats:p>","DOI":"10.3390\/bdcc8050046","type":"journal-article","created":{"date-parts":[[2024,4,30]],"date-time":"2024-04-30T09:50:07Z","timestamp":1714470607000},"page":"46","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Imagine and Imitate: Cost-Effective Bidding under Partially Observable Price Landscapes"],"prefix":"10.3390","volume":"8","author":[{"given":"Xiaotong","family":"Luo","sequence":"first","affiliation":[{"name":"School of Management, Jinan University, Guangzhou 510632, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yongjian","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Management, Jinan University, Guangzhou 510632, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shengda","family":"Zhuo","sequence":"additional","affiliation":[{"name":"College of Cyber Security, Jinan University, Guangzhou 510632, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Lu","sequence":"additional","affiliation":[{"name":"School of Management, Jinan University, Guangzhou 510632, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ziyang","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Management, Jinan University, Guangzhou 510632, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lichun","family":"Li","sequence":"additional","affiliation":[{"name":"School of Management, Jinan University, Guangzhou 510632, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingyan","family":"Tian","sequence":"additional","affiliation":[{"name":"School of Management, Jinan University, Guangzhou 510632, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaotong","family":"Ye","sequence":"additional","affiliation":[{"name":"School of Management, Jinan University, Guangzhou 510632, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7693-7543","authenticated-orcid":false,"given":"Yin","family":"Tang","sequence":"additional","affiliation":[{"name":"School of Management, Jinan University, Guangzhou 510632, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Yuan, S., Wang, J., and Zhao, X. (2013, January 11). Real-time bidding for online advertising: Measurement and analysis. Proceedings of the Seventh International Workshop on Data Mining for Online Advertising, Chicago, IL, USA.","DOI":"10.1145\/2501040.2501980"},{"key":"ref_2","unstructured":"Muthukrishnan, S. (2009). International Workshop on Internet and Network Economics, Springer."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wu, W.C.-H., Yeh, M.-Y., and Chen, M.-S. (2015, January 10\u201313). Predicting winning price in real time bidding with censored data. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Australia.","DOI":"10.1145\/2783258.2783276"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Cui, Y., Zhang, R., Li, W., and Mao, J. (2011, January 21\u201324). Bid landscape forecasting in online ad exchange marketplace. Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA.","DOI":"10.1145\/2020408.2020454"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhang, W., Yuan, S., and Wang, J. (2014, January 24\u201327). Optimal real-time bidding for display advertising. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.","DOI":"10.1145\/2623330.2623633"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhu, W.-Y., Shih, W.-Y., Lee, Y.-H., Peng, W.-C., and Huang, J.-L. (2017, January 11\u201314). A gamma-based regression for winning price estimation in real-time bidding advertising. Proceedings of the 2017 IEEE International Conference on Big Data (Big Data), Boston, MA, USA.","DOI":"10.1109\/BigData.2017.8258095"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Wu, W., Yeh, M.-Y., and Chen, M.-S. (2018, January 19\u201323). Deep censored learning of the winning price in the real time bidding. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.","DOI":"10.1145\/3219819.3220066"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"68692","DOI":"10.1109\/ACCESS.2019.2919450","article-title":"A revenue-maximizing bidding strategy for demand-side platforms","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"109248","DOI":"10.1016\/j.knosys.2022.109248","article-title":"Kaplan\u2013meier markov network: Learning the distribution of market price by censored data in online advertising","volume":"251","author":"Wang","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1109\/TKDE.2017.2775228","article-title":"Bidding machine: Learning to bid for directly optimizing profits in display advertising","volume":"30","author":"Ren","year":"2017","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_11","first-page":"2672","article-title":"Generative adversarial nets","volume":"27","author":"Goodfellow","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1109\/JAS.2017.7510583","article-title":"Generative adversarial networks: Introduction and outlook","volume":"4","author":"Wang","year":"2017","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_13","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, W., Zhou, T., Wang, J., and Xu, J. (2016, January 13\u201317). Bid-aware gradient descent for unbiased learning with censored data in display advertising. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939713"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wang, Y., Ren, K., Zhang, W., Wang, J., and Yu, Y. (2016, January 19\u201323). Functional bid landscape forecasting for display advertising. Proceedings of the Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2016, Riva del Garda, Italy. Proceedings, Part I 16.","DOI":"10.1007\/978-3-319-46128-1_8"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"110392","DOI":"10.1016\/j.knosys.2023.110392","article-title":"A multimodal approach for improving market price estimation in online advertising","volume":"266","author":"Wang","year":"2023","journal-title":"Knowl.-Based Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cai, H., Ren, K., Zhang, W., Malialis, K., Wang, J., Yu, Y., and Guo, D. (2017, January 6\u201310). Real-time bidding by reinforcement learning in display advertising. Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, Cambridge, UK.","DOI":"10.1145\/3018661.3018702"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Perlich, C., Dalessandro, B., Hook, R., Stitelman, O., Raeder, T., and Provost, F. (2012, January 12\u201316). Bid optimizing and inventory scoring in targeted online advertising. Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Beijing, China.","DOI":"10.1145\/2339530.2339655"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Lin, C.-C., Chuang, K.-T., Wu, W.C.-H., and Chen, M.-S. (2016, January 24\u201328). Combining powers of two predictors in optimizing real-time bidding strategy under constrained budget. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, Indianapolis, IN, USA.","DOI":"10.1145\/2983323.2983656"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chen, Y., Berkhin, P., Anderson, B., and Devanur, N.R. (2011, January 21\u201324). Real-time bidding algorithms for performance-based display ad allocation. Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA.","DOI":"10.1145\/2020408.2020604"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Chapelle, O. (2014, January 24\u201327). Modeling delayed feedback in display advertising. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.","DOI":"10.1145\/2623330.2623634"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Xu, J., Lee, K.-c., Li, W., Qi, H., and Lu, Q. (2015, January 10\u201313). Smart pacing for effective online ad campaign optimization. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Australia.","DOI":"10.1145\/2783258.2788615"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Lee, K.-C., Jalali, A., and Dasdan, A. (2013, January 11). Real time bid optimization with smooth budget delivery in online advertising. Proceedings of the Seventh International Workshop on Data Mining for Online Advertising, Chicago, IL, USA.","DOI":"10.1145\/2501040.2501979"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1007\/s44196-022-00077-6","article-title":"Dynamic ad network ordering method using reinforcement learning","volume":"15","author":"Afshar","year":"2022","journal-title":"Int. J. Comput. Intell. Syst."},{"key":"ref_25","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_26","unstructured":"Wang, Y., Liu, J., Liu, Y., Hao, J., He, Y., Hu, J., Yan, W.P., and Li, M. (2017). Ladder: A human-level bidding agent for large-scale real-time online auctions. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1016\/j.neucom.2022.05.108","article-title":"Bid optimization using maximum entropy reinforcement learning","volume":"501","author":"Liu","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, H., Du, C., Fang, P., Yuan, S., He, X., Wang, L., and Zheng, B. (2022). Roi constrained bidding via curriculum-guided bayesian reinforcement learning. arXiv.","DOI":"10.1145\/3534678.3539211"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lu, J., Yang, C., Gao, X., Wang, L., Li, C., and Chen, G. (2019, January 3\u20137). Reinforcement learning with sequential information clustering in real-time bidding. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China.","DOI":"10.1145\/3357384.3358027"},{"key":"ref_30","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016). International Conference on Machine Learning, PMLR."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Jin, J., Song, C., Li, H., Gai, K., Wang, J., and Zhang, W. (2018, January 22\u201326). Real-time bidding with multi-agent reinforcement learning in display advertising. Proceedings of the 27th ACM International Conference on Information and knowledge Management, Torino, Italy.","DOI":"10.1145\/3269206.3272021"},{"key":"ref_32","unstructured":"Lu, Y., Lu, C., Bandyopadhyay, N., Kumar, M., and Gupta, G. (2022). Functional optimization reinforcement learning for real-time bidding. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Abbeel, P., and Ng, A.Y. (2004, January 4\u20138). Apprenticeship learning via inverse reinforcement learning. Proceedings of the Twenty-First International Conference on Machine Learning, Banff, AB, Canada.","DOI":"10.1145\/1015330.1015430"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3054912","article-title":"Imitation learning: A survey of learning methods","volume":"50","author":"Hussein","year":"2017","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1162\/neco.1991.3.1.88","article-title":"Efficient training of artificial neural networks for autonomous navigation","volume":"3","author":"Pomerleau","year":"1991","journal-title":"Neural Comput."},{"key":"ref_36","unstructured":"Ross, S., Gordon, G., and Bagnell, D. (2011, January 11\u201313). A reduction of imitation learning and structured prediction to no-regret online learning. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, Ft. Lauderdale, FL, USA."},{"key":"ref_37","unstructured":"Ho, J., Gupta, J., and Ermon, S. (2016). International Conference on Machine Learning, PMLR."},{"key":"ref_38","unstructured":"Pathak, D., Agrawal, P., Efros, A.A., and Darrell, T. (2017). International Conference on Machine Learning, PMLR."},{"key":"ref_39","unstructured":"Hafner, D., Lillicrap, T., Ba, J., and Norouzi, M. (2019). Dream to control: Learning behaviors by latent imagination. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Ren, K., Qin, J., Zheng, L., Yang, Z., Zhang, W., and Yu, Y. (2019, January 4\u20138). Deep landscape forecasting for real-time bidding advertising. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330870"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Wu, D., Chen, X., Yang, X., Wang, H., Tan, Q., Zhang, X., Xu, J., and Gai, K. (2018, January 22\u201326). Budget constrained bidding by model-free reinforcement learning in display advertising. Proceedings of the 27th ACM International Conference on Information and Knowledge Management, Torino, Italy.","DOI":"10.1145\/3269206.3271748"},{"key":"ref_42","first-page":"493","article-title":"A generalized vickrey auction","volume":"23","author":"Ausubel","year":"1999","journal-title":"Econo0 Metr."},{"key":"ref_43","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_44","unstructured":"Wang, S., Jia, D., and Weng, X. (2018). Deep reinforcement learning for autonomous driving. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Zhou, F., Luo, B., Hu, T., Chen, Z., and Wen, Y. (2021, January 15\u201318). A combinatorial recommendation system framework based on deep reinforcement learning. Proceedings of the 2021 IEEE International Conference on Big Data (Big Data), Orlando, FL, USA.","DOI":"10.1109\/BigData52589.2021.9671593"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Cui, B., Sun, R., and Su, J. (2022, January 4\u20136). A novel deep reinforcement learning strategy in financial portfolio management. Proceedings of the 2022 7th International Conference on Big Data Analytics (ICBDA), Guangzhou, China.","DOI":"10.1109\/ICBDA55095.2022.9760351"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_48","unstructured":"Ross, S., and Bagnell, D. (2010, January 13\u201315). Efficient reductions for imitation learning. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, Sardinia, Italy."}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/8\/5\/46\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:35:11Z","timestamp":1760106911000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/8\/5\/46"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,28]]},"references-count":48,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2024,5]]}},"alternative-id":["bdcc8050046"],"URL":"https:\/\/doi.org\/10.3390\/bdcc8050046","relation":{},"ISSN":["2504-2289"],"issn-type":[{"type":"electronic","value":"2504-2289"}],"subject":[],"published":{"date-parts":[[2024,4,28]]}}}