{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T15:02:25Z","timestamp":1775228545744,"version":"3.50.1"},"reference-count":73,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Priv. Secur."],"published-print":{"date-parts":[[2023,2,28]]},"abstract":"<jats:p>Risk-based authentication (RBA) aims to protect users against attacks involving stolen passwords. RBA monitors features during login, and requests re-authentication when feature values widely differ from those previously observed. It is recommended by various national security organizations, and users perceive it more usable than and equally secure to equivalent two-factor authentication. Despite that, RBA is still used by very few online services. Reasons for this include a lack of validated open resources on RBA properties, implementation, and configuration. This effectively hinders the RBA research, development, and adoption progress.<\/jats:p>\n          <jats:p>To close this gap, we provide the first long-term RBA analysis on a real-world large-scale online service. We collected feature data of 3.3 million users and 31.3 million login attempts over more than 1 year. Based on the data, we provide (i) studies on RBA\u2019s real-world characteristics plus its configurations and enhancements to balance usability, security, and privacy; (ii) a machine learning\u2013based RBA parameter optimization method to support administrators finding an optimal configuration for their own use case scenario; (iii) an evaluation of the round-trip time feature\u2019s potential to replace the IP address for enhanced user privacy; and (iv) a synthesized RBA dataset to reproduce this research and to foster future RBA research. Our results provide insights on selecting an optimized RBA configuration so that users profit from RBA after just a few logins. The open dataset enables researchers to study, test, and improve RBA for widespread deployment in the wild.<\/jats:p>","DOI":"10.1145\/3546069","type":"journal-article","created":{"date-parts":[[2022,6,30]],"date-time":"2022-06-30T10:21:57Z","timestamp":1656584517000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":29,"title":["Pump Up Password Security! Evaluating and Enhancing Risk-Based Authentication on a Real-World Large-Scale Online Service"],"prefix":"10.1145","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7917-6065","authenticated-orcid":false,"given":"Stephan","family":"Wiefling","sequence":"first","affiliation":[{"name":"H-BRS University of Applied Sciences, Germany and Ruhr University Bochum, Bochum, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3806-714X","authenticated-orcid":false,"given":"Paul Ren\u00e9","family":"J\u00f8rgensen","sequence":"additional","affiliation":[{"name":"Telenor Digital, Fornebu, Norway"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7569-8501","authenticated-orcid":false,"given":"Sigurd","family":"Thunem","sequence":"additional","affiliation":[{"name":"Telenor Digital, Fornebu, Norway"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7863-0622","authenticated-orcid":false,"given":"Luigi Lo","family":"Iacono","sequence":"additional","affiliation":[{"name":"H-BRS University of Applied Sciences, Sankt Augustin, Germany"}]}],"member":"320","published-online":{"date-parts":[[2022,11,7]]},"reference":[{"issue":"1","key":"e_1_3_9_2_2","first-page":"7","article-title":"Secure client and server geolocation over the internet","volume":"43","author":"Abdou Abdelrahman","year":"2018","unstructured":"Abdelrahman Abdou and Paul C. Van Oorschot. 2018. Secure client and server geolocation over the internet. ;login: Spring 2018 43, 1 (2018), 7.","journal-title":";login: Spring 2018"},{"key":"e_1_3_9_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCB48548.2020.9304908"},{"key":"e_1_3_9_4_2","article-title":"Credential stuffing: Attacks and economies","volume":"5","year":"2019","unstructured":"Akamai. 2019. Credential stuffing: Attacks and economies. [state of the internet] \/ security 5, Special Media Edition (April 2019). Retrieved July 7, 2022 from https:\/\/web.archive.org\/web\/20210824114851\/https:\/\/www.akamai.com\/us\/en\/multimedia\/documents\/state-of-the-internet\/soti-security-credential-stuffing-attacks-and-economies-report-2019.pdf.","journal-title":"[state of the internet] \/ security"},{"issue":"3","key":"e_1_3_9_5_2","article-title":"Loyalty for sale \u2013 retail and hospitality fraud","volume":"6","year":"2020","unstructured":"Akamai. 2020. Loyalty for sale \u2013 retail and hospitality fraud. [state of the internet] \/ security 6, 3 (Oct. 2020). Retrieved July 7, 2022 from https:\/\/web.archive.org\/web\/20201101013317\/https:\/\/www.akamai.com\/us\/en\/multimedia\/documents\/state-of-the-internet\/soti-security-loyalty-for-sale-retail-and-hospitality-fraud-report-2020.pdf.","journal-title":"[state of the internet] \/ security"},{"key":"e_1_3_9_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/2991079.2991091"},{"key":"e_1_3_9_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-50399-4_16"},{"key":"e_1_3_9_8_2","volume-title":"Australian Government Information Security Manual","author":"Centre Australian Cyber Security","year":"2021","unstructured":"Australian Cyber Security Centre. 2021. Australian Government Information Security Manual. Technical Report. Retrieved July 7, 2022 from https:\/\/web.archive.org\/web\/20210830131917\/https:\/\/www.cyber.gov.au\/sites\/default\/files\/2021-06\/01.%20ISM%20-%20Using%20the%20Australian%20Government%20Information%20Security%20Manual%20(June%202021).pdf."},{"key":"e_1_3_9_9_2","article-title":"Executive order on improving the nation\u2019s cybersecurity","author":"Jr. Joseph R. Biden","year":"2021","unstructured":"Joseph R. Biden Jr.2021. Executive order on improving the nation\u2019s cybersecurity. The White House (May 2021). Retrieved July 7, 2022 from https:\/\/www.whitehouse.gov\/briefing-room\/presidential-actions\/2021\/05\/12\/executive-order-on-improving-the-nations-cybersecurity\/.","journal-title":"The White House"},{"key":"e_1_3_9_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2012.44"},{"key":"e_1_3_9_11_2","volume-title":"Machine Learning Engineering","author":"Burkov Andriy","year":"2020","unstructured":"Andriy Burkov. 2020. Machine Learning Engineering. True Positive Inc."},{"key":"e_1_3_9_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-37282-7_4"},{"key":"e_1_3_9_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372297.3417892"},{"key":"e_1_3_9_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ergon.2018.12.003"},{"key":"e_1_3_9_15_2","volume-title":"Statistical Power Analysis for the Behavioral Sciences (2nd ed.)","author":"Cohen Jacob","year":"1988","unstructured":"Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences (2nd ed.). L. Erlbaum Associates."},{"key":"e_1_3_9_16_2","doi-asserted-by":"publisher","DOI":"10.1186\/2196-064X-1-7"},{"key":"e_1_3_9_17_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2014.23357"},{"key":"e_1_3_9_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICB45273.2019.8987433"},{"key":"e_1_3_9_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CNS48642.2020.9162317"},{"key":"e_1_3_9_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313481"},{"key":"e_1_3_9_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSPW.2019.00020"},{"key":"e_1_3_9_22_2","doi-asserted-by":"publisher","DOI":"10.14764\/10.ASEAS-2016.2-9"},{"key":"e_1_3_9_23_2","unstructured":"European Union. 2016. General data protection regulation. Regulation (EU) 2016\/679. https:\/\/eur-lex.europa.eu\/eli\/reg\/2016\/679\/oj."},{"key":"e_1_3_9_24_2","unstructured":"FireHOL. 2021. All Cybercrime IP Feeds. (May 2021). Retrieved July 7 2022 from http:\/\/iplists.firehol.org\/?ipset=firehol_level4."},{"key":"e_1_3_9_25_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2016.23240"},{"key":"e_1_3_9_26_2","unstructured":"Google. 2018. 1.5 Billion Users and Counting. Thank You. (Oct. 2018). Retrieved July 7 2022 from https:\/\/web.archive.org\/web\/20211030190958\/https:\/\/twitter.com\/gmail\/status\/1055806807174725633."},{"key":"e_1_3_9_27_2","unstructured":"Google. 2021. Making Sign-In Safer and More Convenient. (Oct. 2021). Retrieved July 7 2022 from https:\/\/web.archive.org\/web\/20211006012012\/https:\/\/blog.google\/technology\/safety-security\/making-sign-safer-and-more-convenient\/."},{"key":"e_1_3_9_28_2","doi-asserted-by":"publisher","DOI":"10.6028\/NIST.SP.800-63b"},{"key":"e_1_3_9_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4842-5914-6_4"},{"key":"e_1_3_9_30_2","unstructured":"Have I. Been Pwned. 2022. Pwned websites. (March 2022). Retrieved July 7 2022 from https:\/\/haveibeenpwned.com\/PwnedWebsites\/."},{"key":"e_1_3_9_31_2","volume-title":"Third International Conference on Informatics Engineering and Information Science (Lodz, Poland) (ICIEIS\u201914)","author":"Hurka\u0142a Adam","year":"2014","unstructured":"Adam Hurka\u0142a and Jaros\u0142aw Hurka\u0142a. 2014. Architecture of context-risk-aware authentication system for web environments. In Third International Conference on Informatics Engineering and Information Science (Lodz, Poland) (ICIEIS\u201914)."},{"key":"e_1_3_9_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-7138-7"},{"key":"e_1_3_9_33_2","first-page":"225","volume-title":"Eleventh Symposium On Usable Privacy and Security (Ottawa, Canada) (SOUPS\u201915)","author":"Khan Hassan","year":"2015","unstructured":"Hassan Khan, Urs Hengartner, and Daniel Vogel. 2015. Usability and security perceptions of implicit authentication: Convenient, secure, sometimes annoying. In Eleventh Symposium On Usable Privacy and Security (Ottawa, Canada) (SOUPS\u201915). USENIX Association, 225\u2013239."},{"key":"e_1_3_9_34_2","volume-title":"31st USENIX Security Symposium (Boston, MA, USA) (USENIX Security\u201922)","author":"Kohls Katharina","year":"2022","unstructured":"Katharina Kohls and Claudia Diaz. 2022. VerLoc: Verifiable localization in decentralized systems. In 31st USENIX Security Symposium (Boston, MA, USA) (USENIX Security\u201922). USENIX Association. Retrieved July 7, 2022 from https:\/\/www.usenix.org\/conference\/usenixsecurity22\/presentation\/kohls."},{"key":"e_1_3_9_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIS.2014.6912180"},{"key":"e_1_3_9_36_2","volume-title":"30th USENIX Security Symposium (USENIX Security\u201921)","author":"Mayer Peter","year":"2021","unstructured":"Peter Mayer, Yixin Zou, Florian Schaub, and Adam J. Aviv. 2021. \u201cNow I\u2019m a bit angry:\u201d Individuals\u2019 awareness, perception, and responses to data breaches that affected them. In 30th USENIX Security Symposium (USENIX Security\u201921). USENIX Association."},{"key":"e_1_3_9_37_2","doi-asserted-by":"publisher","DOI":"10.17487\/RFC6455"},{"key":"e_1_3_9_38_2","volume-title":"Enigma 2018 (Santa Clara, CA, USA)","author":"Milka Grzergor","year":"2018","unstructured":"Grzergor Milka. 2018. Anatomy of account takeover. In Enigma 2018 (Santa Clara, CA, USA). USENIX Association. https:\/\/www.usenix.org\/node\/208154."},{"key":"e_1_3_9_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/UIC-ATC.2017.8397628"},{"key":"e_1_3_9_40_2","unstructured":"Mozilla. 2021. Mozilla VPN. (July 2021). https:\/\/www.mozilla.org\/en-US\/products\/vpn\/."},{"key":"e_1_3_9_41_2","volume-title":"Cloud security guidance: 10, Identity and authentication","author":"Centre National Cyber Security","year":"2018","unstructured":"National Cyber Security Centre. 2018. Cloud security guidance: 10, Identity and authentication. Technical Report. Retrieved July 7, 2022 from https:\/\/www.ncsc.gov.uk\/collection\/cloud-security\/implementing-the-cloud-security-principles\/identity-and-authentication."},{"key":"e_1_3_9_42_2","unstructured":"Lily Hay Newman. 2021. Facebook Will Force More At-Risk Accounts to Use Two-Factor. (Dec. 2021). Retrieved July 7 2022 from https:\/\/web.archive.org\/web\/20211212185008\/https:\/\/www.wired.com\/story\/facebook-protect-two-factor-authentication-requirement\/."},{"key":"e_1_3_9_43_2","unstructured":"Open Identity Platform. 2016. OpenAM: Adaptive Authentication Module. (Aug. 2016). Retrieved July 7 2022 from https:\/\/github.com\/OpenIdentityPlatform\/OpenAM\/blob\/master\/openam-authentication\/openam-auth-adaptive\/src\/main\/java\/org\/forgerock\/openam\/authentication\/modules\/adaptive\/Adaptive.java."},{"key":"e_1_3_9_44_2","unstructured":"Jarrod Overson. 2020. The State of Credential Stuffing and the Future of Account Takeovers. (Feb. 2020). Retrieved July 7 2022 from https:\/\/youtu.be\/XgPtLZQKQzo."},{"key":"e_1_3_9_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2019.00056"},{"key":"e_1_3_9_46_2","unstructured":"Tommy Pauly and Delziel Fernandes. 2021. Get ready for iCloud Private Relay. (June 2021). Retrieved July 7 2022 from https:\/\/developer.apple.com\/videos\/play\/wwdc2021\/10096\/."},{"key":"e_1_3_9_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/2751323.2751327"},{"key":"e_1_3_9_48_2","volume-title":"Who are you?! Adventures in Authentication Workshop 2018 (Baltimore, MD, USA) (WAY\u201918)","author":"Quermann Nils","year":"2018","unstructured":"Nils Quermann, Marian Harbach, and Markus D\u00fcrmuth. 2018. The state of user authentication in the wild. In Who are you?! Adventures in Authentication Workshop 2018 (Baltimore, MD, USA) (WAY\u201918). https:\/\/wayworkshop.org\/2018\/papers\/way2018-quermann.pdf."},{"key":"e_1_3_9_49_2","volume-title":"Who Are You?! Adventures in Authentication Workshop 2017 (Santa Clara, CA, USA) (WAY\u201917)","author":"Redmiles Elissa M.","year":"2017","unstructured":"Elissa M. Redmiles, Everest Liu, and Michelle L. Mazurek. 2017. You want me to do what? A design study of two-factor authentication messages. In Who Are You?! Adventures in Authentication Workshop 2017 (Santa Clara, CA, USA) (WAY\u201917). USENIX Association. Retrieved July 7, 2022 from https:\/\/www.usenix.org\/conference\/soups2017\/workshop-program\/way2017\/redmiles."},{"key":"e_1_3_9_50_2","first-page":"357","volume-title":"Fifteenth Symposium on Usable Privacy and Security (Santa Clara, CA, USA) (SOUPS\u201919)","author":"Reese Ken","year":"2019","unstructured":"Ken Reese, Trevor Smith, Jonathan Dutson, Jonathan Armknecht, Jacob Cameron, and Kent Seamons. 2019. A usability study of five two-factor authentication methods. In Fifteenth Symposium on Usable Privacy and Security (Santa Clara, CA, USA) (SOUPS\u201919). USENIX Association, 357\u2013370. https:\/\/www.usenix.org\/conference\/soups2019\/presentation\/reese."},{"key":"e_1_3_9_51_2","doi-asserted-by":"publisher","DOI":"10.5220\/0004533904270434"},{"key":"e_1_3_9_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411508.3421377"},{"key":"e_1_3_9_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-47072-6_9"},{"key":"e_1_3_9_54_2","unstructured":"scikit learn. 2020. OneHotEncoder. (2020). Retrieved July 7 2022 from https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.preprocessing.OneHotEncoder.html."},{"key":"e_1_3_9_55_2","unstructured":"scikit learn. 2021. Novelty and Outlier Detection. (2021). Retrieved July 7 2022 from https:\/\/web.archive.org\/web\/20210812084024\/https:\/\/scikit-learn.org\/stable\/modules\/outlier_detection.html."},{"key":"e_1_3_9_56_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23808-6_10"},{"key":"e_1_3_9_57_2","unstructured":"Theresa Stadler and Bristena Oprisanu. 2021. Privacy evaluation framework for synthetic data publishing. (July 2021). https:\/\/github.com\/spring-epfl\/synthetic_data_release."},{"key":"e_1_3_9_58_2","volume-title":"31st USENIX Security Symposium (Boston, MA, USA)","author":"Stadler Theresa","year":"2022","unstructured":"Theresa Stadler, Bristena Oprisanu, and Carmela Troncoso. 2022. Synthetic data \u2013 anonymisation groundhog day. In 31st USENIX Security Symposium (Boston, MA, USA). USENIX Association. https:\/\/www.usenix.org\/conference\/usenixsecurity22\/presentation\/stadler."},{"key":"e_1_3_9_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-67220-5_31"},{"key":"e_1_3_9_60_2","unstructured":"statcounter. 2021. Browser Market Share Norway. (July 2021). Retrieved July 7 2022 from https:\/\/gs.statcounter.com\/browser-market-share\/all\/norway\/."},{"key":"e_1_3_9_61_2","unstructured":"State of California. 2018. California Consumer Privacy Act. (June 2018). Assembly Bill No. 375.https:\/\/leginfo.legislature.ca.gov\/faces\/billTextClient.xhtml?bill_id=201720180AB375."},{"key":"e_1_3_9_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3011784.3011800"},{"key":"e_1_3_9_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCCNT51525.2021.9580020"},{"key":"e_1_3_9_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134067"},{"key":"e_1_3_9_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSEC.2020.2970639"},{"key":"e_1_3_9_66_2","unstructured":"Twitter. 2022. Account Security - Twitter Transparency Center. (Jan. 2022). Retrieved July 7 2022 from https:\/\/web.archive.org\/web\/20220211182429\/https:\/\/transparency.twitter.com\/en\/reports\/account-security.html#2021-jan-jun."},{"key":"e_1_3_9_67_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-39200-9_18"},{"key":"e_1_3_9_68_2","unstructured":"W3Counter. 2021. Web Browser Usage Trends. (July 2021). https:\/\/www.w3counter.com\/trends."},{"key":"e_1_3_9_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978339"},{"key":"e_1_3_9_70_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-64331-0_19"},{"key":"e_1_3_9_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3427228.3427243"},{"key":"e_1_3_9_72_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-22312-0_10"},{"key":"e_1_3_9_73_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58201-2_19"},{"key":"e_1_3_9_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSPW54576.2021.00040"}],"container-title":["ACM Transactions on Privacy and Security"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3546069","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3546069","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:19Z","timestamp":1750188619000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3546069"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,7]]},"references-count":73,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,2,28]]}},"alternative-id":["10.1145\/3546069"],"URL":"https:\/\/doi.org\/10.1145\/3546069","relation":{},"ISSN":["2471-2566","2471-2574"],"issn-type":[{"value":"2471-2566","type":"print"},{"value":"2471-2574","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,7]]},"assertion":[{"value":"2021-09-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-06-24","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}