{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T15:51:23Z","timestamp":1774453883178,"version":"3.50.1"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,6,22]],"date-time":"2023-06-22T00:00:00Z","timestamp":1687392000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGKDD Explor. Newsl."],"published-print":{"date-parts":[[2023,6,22]]},"abstract":"<jats:p>Real-time bidding (RTB) has become a major paradigm of display advertising. Each ad impression generated from a user visit is auctioned in real time, where demand-side plat- form (DSP) automatically provides bid price usually relying on the ad impression value estimation and the optimal bid price determination. However, the current bid strategy over- looks the randomness of the user behaviors (e.g., click) and the cost uncertainty caused by the auction competition. In this work, we propose a novel adaptive risk-aware bidding algorithm with budget constraint via reinforcement learn- ing, which is the rst to simultaneously consider estimation uncertainty and the dynamic risk tendency of a DSP. Specif- ically, we explicitly factor in the uncertainty of estimated ad impression values and model the risk preference of a DSP under a speci c state and market environment via a sequen- tial decision process. Additionally, we theoretically unveil the intrinsic relation between the uncertainty and the risk tendency based on value at risk (VaR). Consequently, we propose two instantiations to model risk tendency, includ- ing an expert knowledge-based formulation embracing three essential properties and an adaptive learning method based on self-supervised reinforcement learning. We conduct ex- periments on public datasets and show that the proposed framework achieves better performance in terms of the num- ber of clicks under di erent budget constraints 1.<\/jats:p>","DOI":"10.1145\/3606274.3606281","type":"journal-article","created":{"date-parts":[[2023,7,5]],"date-time":"2023-07-05T17:33:05Z","timestamp":1688578385000},"page":"73-82","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Adaptive RiskAware Bidding with Budget Constraint in Display Advertising"],"prefix":"10.1145","volume":"25","author":[{"given":"Zhimeng","family":"Jiang","sequence":"first","affiliation":[{"name":"Texas A&amp;M University, TX, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kaixiong","family":"Zhou","sequence":"additional","affiliation":[{"name":"Rice University, TX, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mi","family":"Zhang","sequence":"additional","affiliation":[{"name":"Samsung Electronics, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rui","family":"Chen","sequence":"additional","affiliation":[{"name":"Samsung Electronics, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xia","family":"Hu","sequence":"additional","affiliation":[{"name":"Rice University, TX, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"SooHyun","family":"Choi","sequence":"additional","affiliation":[{"name":"Samsung Electronics, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,7,5]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"A meta-analysis of randomness in human behavioral research","author":"Armstrong S. A.","year":"2004","unstructured":"S. A. Armstrong . A meta-analysis of randomness in human behavioral research . Louisiana State University and Agricultural & Mechanical College , 2004 . S. A. Armstrong. A meta-analysis of randomness in human behavioral research. Louisiana State University and Agricultural & Mechanical College, 2004."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3132868"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3018661.3018702"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623634"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2988450.2988454"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/319"},{"key":"e_1_2_1_7_1","first-page":"306","volume-title":"Proceedings of the 2020 SIAM Inter- national Conference on Data Mining","author":"Ghosh A.","unstructured":"A. Ghosh , S. Mitra , S. Sarkhel , and V. Swaminathan . Optimal bidding strategy without exploration in real- time bidding . In Proceedings of the 2020 SIAM Inter- national Conference on Data Mining , pages 298{ 306 . SIAM, 2020. A. Ghosh, S. Mitra, S. Sarkhel, and V. Swaminathan. Optimal bidding strategy without exploration in real- time bidding. In Proceedings of the 2020 SIAM Inter- national Conference on Data Mining, pages 298{306. SIAM, 2020."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526744"},{"key":"e_1_2_1_9_1","volume-title":"Google White Paper","year":"2012","unstructured":"Google. The arrival of real-time bidding and what it means for media buyers . In Google White Paper , 2012 . Google. The arrival of real-time bidding and what it means for media buyers. In Google White Paper, 2012."},{"key":"e_1_2_1_10_1","volume-title":"Her- brich. Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft's bing search engine","author":"Graepel T.","year":"2010","unstructured":"T. Graepel , J. Q. Candela , T. Borchert , and R. Her- brich. Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft's bing search engine . Omnipress , 2010 . T. Graepel, J. Q. Candela, T. Borchert, and R. Her- brich. Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft's bing search engine. Omnipress, 2010."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330749"},{"key":"e_1_2_1_12_1","first-page":"1731","volume-title":"Proceedings of the 26th International Joint Conference on Arti cial Intelligence","author":"Guo H.","year":"2017","unstructured":"H. Guo , R. Tang , Y. Ye , Z. Li , and X. He . Deepfm: a factorization-machine based neural network for ctr pre- diction . In Proceedings of the 26th International Joint Conference on Arti cial Intelligence , pages 1725{ 1731 , 2017 . H. Guo, R. Tang, Y. Ye, Z. Li, and X. He. Deepfm: a factorization-machine based neural network for ctr pre- diction. In Proceedings of the 26th International Joint Conference on Arti cial Intelligence, pages 1725{1731, 2017."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080777"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2648584.2648589"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467199"},{"key":"e_1_2_1_16_1","volume-title":"John Wiley & Sons","author":"Hull J.","year":"2012","unstructured":"J. Hull . Risk management and nancial institutions,, volume 733 . John Wiley & Sons , 2012 . J. Hull. Risk management and nancial institutions,, volume 733. John Wiley & Sons, 2012."},{"key":"e_1_2_1_17_1","volume-title":"Academic press","author":"Krishna V.","year":"2009","unstructured":"V. Krishna . Auction theory. Academic press , 2009 . V. Krishna. Auction theory. Academic press, 2009."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339651"},{"issue":"2","key":"e_1_2_1_19_1","first-page":"67","volume":"56","author":"Linsmeier T. J.","year":"2000","unstructured":"T. J. Linsmeier and N. D. Pearson . Value at risk. Fi- nancial Analysts Journal , 56 ( 2 ):47{ 67 , 2000 . T. J. Linsmeier and N. D. Pearson. Value at risk. Fi- nancial Analysts Journal, 56(2):47{67, 2000.","journal-title":"Value at risk. Fi- nancial Analysts Journal"},{"issue":"1","key":"e_1_2_1_20_1","first-page":"91","volume":"7","author":"Markowitz H.","year":"1952","unstructured":"H. Markowitz . Harry m. markowitz. Portfolio selection , Journal of Finance , 7 ( 1 ):77{ 91 , 1952 . H. Markowitz. Harry m. markowitz. Portfolio selection, Journal of Finance, 7(1):77{91, 1952.","journal-title":"Journal of Finance"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487575.2488200"},{"key":"e_1_2_1_22_1","volume-title":"Modeling risk: Applying Monte Carlo simula- tion, real options analysis, forecasting, and optimiza- tion techniques","author":"Mun J.","year":"2006","unstructured":"J. Mun . Modeling risk: Applying Monte Carlo simula- tion, real options analysis, forecasting, and optimiza- tion techniques , volume 347 . John Wiley & Sons , 2006 . J. Mun. Modeling risk: Applying Monte Carlo simula- tion, real options analysis, forecasting, and optimiza- tion techniques, volume 347. John Wiley & Sons, 2006."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556195.2556240"},{"key":"e_1_2_1_24_1","first-page":"812","volume-title":"Proceed- ings of the 18th ACM SIGKDD International Confer- ence on Knowledge Discovery and Data Mining","author":"Perlich C.","year":"2012","unstructured":"C. Perlich , B. Dalessandro , R. Hook , O. Stitelman , T. Raeder , and F. Provost . Bid optimizing and inven- tory scoring in targeted online advertising . In Proceed- ings of the 18th ACM SIGKDD International Confer- ence on Knowledge Discovery and Data Mining , pages 804{ 812 , 2012 . C. Perlich, B. Dalessandro, R. Hook, O. Stitelman, T. Raeder, and F. Provost. Bid optimizing and inven- tory scoring in targeted online advertising. In Proceed- ings of the 18th ACM SIGKDD International Confer- ence on Knowledge Discovery and Data Mining, pages 804{812, 2012."},{"key":"e_1_2_1_25_1","volume-title":"ACM Transactions on Intelligent Systems and Technology (TIST), 3(3):1{22","author":"Rendle S.","year":"2012","unstructured":"S. Rendle . Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology (TIST), 3(3):1{22 , 2012 . S. Rendle. Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology (TIST), 3(3):1{22, 2012."},{"key":"e_1_2_1_26_1","volume-title":"Optimization of conditional value-at-risk. Journal of risk, 2:21{42","author":"Rockafellar R. T.","year":"2000","unstructured":"R. T. Rockafellar , S. Uryasev , Optimization of conditional value-at-risk. Journal of risk, 2:21{42 , 2000 . R. T. Rockafellar, S. Uryasev, et al. Optimization of conditional value-at-risk. Journal of risk, 2:21{42, 2000."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939704"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.3905\/jpm.1994.409501"},{"key":"e_1_2_1_29_1","first-page":"6","volume-title":"Proceedings of the Sixth International Workshop on Data Mining for Online Advertising and Internet Economy","author":"I. Tro","year":"2012","unstructured":"I. Tro mov, A. Kornetova , and V. Topinskiy . Using boosted trees for click-through rate prediction for spon- sored search . In Proceedings of the Sixth International Workshop on Data Mining for Online Advertising and Internet Economy , pages 1{ 6 , 2012 . I. Tro mov, A. Kornetova, and V. Topinskiy. Using boosted trees for click-through rate prediction for spon- sored search. In Proceedings of the Sixth International Workshop on Data Mining for Online Advertising and Internet Economy, pages 1{6, 2012."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2684822.2697041"},{"key":"e_1_2_1_31_1","first-page":"1451","volume-title":"Proceedings of the 27th ACM International Con- ference on Information and Knowledge Management","author":"Wu D.","year":"2018","unstructured":"D. Wu , X. Chen , X. Yang , H. Wang , Q. Tan , X. Zhang , J. Xu , and K. Gai . Budget constrained bidding by model-free reinforcement learning in display advertis- ing . In Proceedings of the 27th ACM International Con- ference on Information and Knowledge Management , pages 1443{ 1451 , 2018 . D. Wu, X. Chen, X. Yang, H. Wang, Q. Tan, X. Zhang, J. Xu, and K. Gai. Budget constrained bidding by model-free reinforcement learning in display advertis- ing. In Proceedings of the 27th ACM International Con- ference on Information and Knowledge Management, pages 1443{1451, 2018."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783276"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2017\/435"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2501040.2501980"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/589"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939759"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3018661.3018701"},{"key":"e_1_2_1_38_1","first-page":"57","volume-title":"European Conference on Information Retrieval","author":"Zhang W.","year":"2016","unstructured":"W. Zhang , T. Du , and J. Wang . Deep learning over multi- eld categorical data . In European Conference on Information Retrieval , pages 45{ 57 , 2016 . W. Zhang, T. Du, and J. Wang. Deep learning over multi- eld categorical data. In European Conference on Information Retrieval, pages 45{57, 2016."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783269"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623633"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219823"}],"container-title":["ACM SIGKDD Explorations Newsletter"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3606274.3606281","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3606274.3606281","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:47:08Z","timestamp":1750178828000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3606274.3606281"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,22]]},"references-count":41,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,6,22]]}},"alternative-id":["10.1145\/3606274.3606281"],"URL":"https:\/\/doi.org\/10.1145\/3606274.3606281","relation":{},"ISSN":["1931-0145","1931-0153"],"issn-type":[{"value":"1931-0145","type":"print"},{"value":"1931-0153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,22]]},"assertion":[{"value":"2023-07-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}