{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T19:33:31Z","timestamp":1781120011587,"version":"3.54.1"},"reference-count":74,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T00:00:00Z","timestamp":1762387200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>The exponential growth of open web data provides unprecedented opportunities for business-to-business (B2B) lead generation. However, automating the discovery and qualification of new leads from unstructured web content is a complex challenge requiring the integration of web crawling, information extraction, and data-driven analytics. This article presents a comprehensive review of artificial intelligence (AI) methods for automated lead generation and introduces Scrapus, an AI-driven web prospecting platform that unifies these methods into an end-to-end system. Scrapus autonomously crawls the open web for company information, extracts and enriches relevant data (using natural language processing and knowledge graphs), matches findings to user-defined ideal customer profiles, and generates concise natural-language lead summaries using large language models. We survey relevant literature in web mining, focused crawling, entity resolution, and text summarization \u2013 highlighting how Scrapus builds upon and extends prior work. The system\u2019s modular architecture and AI components are described in detail, reflecting accurate implementation details. We also report an experimental evaluation on real-world data: Scrapus significantly outperforms baseline approaches in lead discovery rate, extraction accuracy, lead qualification (achieving ~90% precision and recall), and summary usefulness. The results show a\u202f~3\u202f\u00d7\u202fhigher relevant lead yield from web crawling due to reinforcement learning, a substantial increase in extraction F1 (from ~0.77 to ~0.92) through transformer-based NLP, and greatly improved lead scoring over traditional methods. This review and case study demonstrate that combining reinforcement learning, transformer-based NLP, and knowledge-enhanced analysis can effectively automate B2B lead generation. The advances surveyed here point toward a new generation of intelligent sales prospecting tools, in which AI techniques augment human expertise to identify and engage leads at scale.<\/jats:p>","DOI":"10.3389\/frai.2025.1606431","type":"journal-article","created":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T11:46:21Z","timestamp":1762429581000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["A review of AI-based business lead generation: Scrapus as a case study"],"prefix":"10.3389","volume":"8","author":[{"given":"Ahmet","family":"Kaplan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sadi Evren","family":"Seker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rabia","family":"Yoruk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,11,6]]},"reference":[{"key":"ref1","author":"Agichtein","year":"2000"},{"key":"ref9001","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1145\/3442200","article-title":"Neural Networks for Entity Matching: A Survey","volume":"15","author":"Barlaug","year":"2021","journal-title":"ACM Trans. Knowl. Discov. Data"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3591366.3591378","article-title":"Linked data\u2013the story so far","volume":"5","author":"Bizer","year":"2009","journal-title":"Int. J. Semant. Web Inf. Syst."},{"key":"ref4","doi-asserted-by":"publisher","first-page":"993","DOI":"10.5555\/944919.944937","article-title":"Latent Dirichlet allocation","volume":"3","author":"Blei","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref5","first-page":"3","article-title":"Integration of machine learning insights into organizational learning: a case of B2B sales forecasting","volume-title":"Blurring the boundaries through digital innovation (lecture notes in information systems)","author":"Bohanec","year":"2016"},{"key":"ref6","first-page":"1247","article-title":"Freebase: a collaboratively created graph database for structuring human knowledge,","author":"Bollacker","year":"2008"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.48550\/arXiv.2108.07258","article-title":"On the opportunities and risks of foundation models","volume":"72","author":"Bommasani","year":"2021","journal-title":"J. Artif. Intell. Res."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"1877","DOI":"10.48550\/arXiv.2005.14165","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Advances in neural information processing systems"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"1306","DOI":"10.5555\/2898607.2898816","article-title":"Toward an architecture for never-ending language learning","volume":"24","author":"Carlson","year":"2010","journal-title":"Proc. AAAI"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"1623","DOI":"10.1016\/S1389-1286(99)00052-3","article-title":"Focused crawling: a new approach to topic-specific web resource discovery","volume":"31","author":"Chakrabarti","year":"1999","journal-title":"Comput. Netw."},{"key":"ref11","first-page":"785","author":"Chen","year":"2016"},{"key":"ref12","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-31164-2","volume-title":"Data matching: concepts and techniques for record linkage, entity resolution, and duplicate detection","author":"Christen","year":"2012"},{"key":"ref9003","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1911.02116","article-title":"Unsupervised Cross-lingual Representation Learning at Scale","author":"Conneau","year":"2019","journal-title":"arXiv preprint"},{"key":"ref13","first-page":"4171","author":"Devlin","year":"2019"},{"key":"ref14","author":"Diligenti","year":"2000"},{"key":"ref15","author":"Dong","year":"2015"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"102642","DOI":"10.1016\/j.ijinfomgt.2023.102642","article-title":"\u201cSo what if ChatGPT wrote it?\u201d multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy","volume":"71","author":"Dwivedi","year":"2023","journal-title":"Int. J. Inf. Manag."},{"key":"ref17","volume-title":"Consumer behavior","author":"Engel","year":"1978"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"107122","DOI":"10.1016\/j.cie.2021.107122","article-title":"Lead management optimization using data mining: a case in the telecommunications sector","volume":"154","author":"Espadinha-Cruz","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"ref19","first-page":"3","article-title":"Open information extraction: the second generation","author":"Etzioni","year":"2011"},{"key":"ref9002","doi-asserted-by":"publisher","first-page":"2670","DOI":"10.1145\/1409360.1409378","article-title":"Open Information Extraction from the Web","author":"Etzioni","year":"2007","journal-title":"Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/1051712X.2025.2481374","article-title":"Artificial intelligence applications in the B2B sales funnel","volume":"28","author":"Fehrenbach","year":"2025","journal-title":"J. Bus. Bus. Mark."},{"key":"ref21","doi-asserted-by":"publisher","first-page":"1183","DOI":"10.1080\/01621459.1969.10501049","article-title":"A theory for record linkage","volume":"64","author":"Fellegi","year":"1969","journal-title":"J. Am. Stat. Assoc."},{"key":"ref22","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1007\/s12599-023-00834-7","article-title":"Generative AI","volume":"66","author":"Feuerriegel","year":"2024","journal-title":"Bus. Inf. Syst. Eng."},{"key":"ref23","doi-asserted-by":"publisher","first-page":"74","DOI":"10.3390\/informatics8040074","article-title":"Information technology adoption in digital marketing: a literature review","volume":"8","author":"Figueiredo","year":"2021","journal-title":"Informatics"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"1554325","DOI":"10.3389\/frai.2025.1554325","article-title":"The relevance of lead prioritization: a B2B lead scoring model based on machine learning","volume":"8","author":"Gonz\u00e1lez-Flores","year":"2025","journal-title":"Front. Artif. Int."},{"key":"ref26","volume-title":"Gemini 1.5 technical report","year":"2024"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1016\/j.procs.2022.08.007","article-title":"Industry 4.0: predicting Lead conversion opportunities with machine learning in small and medium sized enterprises","volume":"204","author":"Gouveia","year":"2022","journal-title":"Proc. Comput. Sci."},{"key":"ref28","first-page":"142","article-title":"Focused crawling using temporal difference learning,","author":"Grigoriadis","year":"2004"},{"key":"ref29","volume-title":"BERTopic: Neural Topic Modeling with a Class-Based TF-IDF Procedure","author":"Grootendorst","year":"2022"},{"key":"ref30","first-page":"261","author":"Han","year":"2018"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"1161","DOI":"10.1108\/EJM-04-2020-0247","article-title":"Digital marketing and business-to-business relationships: a close look at the Interface and a roadmap for the future","volume":"54","author":"Hofacker","year":"2020","journal-title":"Eur. J. Mark."},{"key":"ref32","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1038\/s41746-024-01079-8","article-title":"A critical assessment of using ChatGPT for extracting structured data from clinical notes","volume":"7","author":"Huang","year":"2024","journal-title":"NPJ Digital Med."},{"key":"ref33","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1016\/j.indmarman.2015.07.002","article-title":"Harnessing marketing automation for B2B content marketing","volume":"54","author":"J\u00e4rvinen","year":"2016","journal-title":"Ind. Mark. Manag."},{"key":"ref34","doi-asserted-by":"publisher","first-page":"187","DOI":"10.34028\/iajit\/20\/5\/14","article-title":"Enhancing focused web crawling using multi-armed bandits and adaptive epsilon decay","volume":"21","author":"Kaleel","year":"2023","journal-title":"Web Int."},{"key":"ref35","article-title":"A Survey on Recent Advances in Named Entity Recognition","volume-title":"arXiv preprint","author":"Keraghel","year":"2024"},{"key":"ref36","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1145\/1718487.1718542","article-title":"Boilerplate detection using shallow text features,","volume-title":"Proceedings of the Third ACM International Conference on Web Search and Data Mining","author":"Kohlsch\u00fctter","year":"2010"},{"key":"ref37","article-title":"TRES: Tree-based Focused Web Crawling with Reinforcement Learning","volume-title":"arXiv preprint","author":"Kontogiannis","year":"2021"},{"key":"ref38","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-658-49241-0_9","article-title":"Bandit on the Hunt: Dynamic Crawling for Cyber Threat Intelligence","volume-title":"arXiv preprint","author":"Kuehn","year":"2025"},{"key":"ref39","article-title":"ThreatCrawl: A BERT-based Focused Crawler for the Cybersecurity Domain","volume-title":"arXiv preprint","author":"Kuehn","year":"2023"},{"key":"ref40","author":"Lample","year":"2016"},{"key":"ref41","doi-asserted-by":"publisher","first-page":"167","DOI":"10.3233\/SW-140134","article-title":"DBpedia\u2013A large-scale, multilingual knowledge base extracted from Wikipedia","volume":"6","author":"Lehmann","year":"2015","journal-title":"Semantic Web"},{"key":"ref42","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","volume":"33","author":"Lewis","year":"2020","journal-title":"Adv. Neural Inf. Proces. Syst."},{"key":"ref45","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1016\/j.intmar.2020.04.002","article-title":"Brave New World? On AI and the Management of Customer Relationships","volume":"51","author":"Libai","year":"2020","journal-title":"J. Interact. Mark."},{"key":"ref46","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-19460-3","volume-title":"Web data mining: Exploring hyperlinks, contents, and usage data","author":"Liu","year":"2011"},{"key":"ref43","first-page":"50","article-title":"Deep entity matching with pre-trained language models","volume-title":"Proc. VLDB Endow.","author":"Li","year":"2020"},{"key":"ref47","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1145\/1031114.1031117","article-title":"Topical web crawlers: evaluating adaptive algorithms","volume":"4","author":"Menczer","year":"2004","journal-title":"ACM Trans. Internet Technol."},{"key":"ref48","first-page":"1003","article-title":"Distant supervision for relation extraction without Labeled data","author":"Mintz","year":"2009"},{"key":"ref49","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref50","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1145\/3183713.3196926","article-title":"Deep learning for entity matching: a design space exploration,","volume-title":"Proceedings of the 2018 ACM SIGMOD International Conference on Management of Data","author":"Mudgal","year":"2018"},{"key":"ref51","doi-asserted-by":"publisher","first-page":"2592","DOI":"10.1016\/j.eswa.2008.02.021","article-title":"Application of data mining techniques in customer relationship management: a literature review and classification","volume":"36","author":"Ngai","year":"2009","journal-title":"Expert Syst. Appl."},{"key":"ref52","first-page":"1108","article-title":"Automating Lead scoring with machine learning: an experimental study,","author":"Nyg\u00e5rd","year":"2020"},{"key":"ref53","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1561\/1500000017","article-title":"Web crawling","volume":"4","author":"Olston","year":"2010","journal-title":"Found. Trends Inf. Retr."},{"key":"ref54","first-page":"2303.08774","volume-title":"GPT-4 Technical Report","year":"2023"},{"key":"ref55","doi-asserted-by":"publisher","first-page":"27730","DOI":"10.48550\/arXiv.2203.02155","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang","year":"2022","journal-title":"Adv. Neural Inf. Proc. Syst."},{"key":"ref56","article-title":"A Deep Reinforced Model for Abstractive Summarization","volume-title":"arXiv preprint","author":"Paulus","year":"2017"},{"key":"ref9004","first-page":"714","article-title":"Named Entity Recognition using Negative Sampling and Reinforcement Learning","volume-title":"2021 IEEE International Conference on Bioinformatics and Biomedicine","author":"Peng","year":"2021"},{"key":"ref57","first-page":"2463","article-title":"Language models as knowledge bases?","author":"Petroni","year":"2019"},{"key":"ref58","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1080\/08853134.2016.1275973","article-title":"Generating leads with sequential persuasion: should sales influence tactics be consistent or complementary?","volume":"37","author":"P\u00f6yry","year":"2017","journal-title":"J. Pers. Sell. Sales Manag."},{"key":"ref59","first-page":"91","article-title":"Business Lead generation for online real estate services: a case study","author":"Rahman","year":"2012"},{"key":"ref60","first-page":"101","article-title":"Automatic sales Lead generation from web data","author":"Ramakrishnan","year":"2006"},{"key":"ref61","first-page":"3982","author":"Reimers","year":"2019"},{"key":"ref62","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1561\/1900000003","article-title":"Information extraction","volume":"1","author":"Sarawagi","year":"2008","journal-title":"Foundations Trends Databases"},{"key":"ref63","first-page":"1073","author":"See","year":"2017"},{"key":"ref64","volume-title":"Active learning literature survey","author":"Settles","year":"2010"},{"key":"ref65","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1080\/10496491.2021.1987973","article-title":"Comparing AI-based and traditional prospect generating methods","volume":"28","author":"Stadlmann","year":"2021","journal-title":"J. Promot. Manag."},{"key":"ref66","first-page":"3008","article-title":"Learning to summarize with human feedback","volume":"33","author":"Stiennon","year":"2020","journal-title":"Adv. Neural Inf. Proc. Syst."},{"key":"ref67","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1145\/1242572.1242667","article-title":"YAGO: a Core of semantic knowledge","volume-title":"Proceedings of the 16th International Conference on World Wide Web (WWW)","author":"Suchanek","year":"2007"},{"key":"ref68","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1016\/j.indmarman.2017.12.019","article-title":"Waiting for a sales renaissance in the fourth industrial revolution: machine learning and AI in sales research and practice","volume":"69","author":"Syam","year":"2018","journal-title":"Ind. Mark. Manag."},{"key":"ref69","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1016\/j.ejor.2004.04.022","article-title":"Predicting online-purchasing behaviour","volume":"166","author":"Van den Poel","year":"2005","journal-title":"Eur. J. Oper. Res."},{"key":"ref70","doi-asserted-by":"publisher","first-page":"5998","DOI":"10.48550\/arXiv.1706.03762","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neur. Inf. Proc. Syst."},{"key":"ref71","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1145\/2629489","article-title":"Wikidata: a free collaborative Knowledge Base","volume":"57","author":"Vrande\u010di\u0107","year":"2014","journal-title":"Commun. ACM"},{"key":"ref72","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1007\/s10799-023-00388-w","article-title":"The state of Lead scoring models and their impact on sales performance","volume":"25","author":"Wu","year":"2024","journal-title":"Inf. Technol. Manag."},{"key":"ref73","first-page":"844","article-title":"Customer behavior pattern discovering with web mining,","volume-title":"Proceedings of the Asia-Pacific Web Conference (APWeb), LNCS 3007","author":"Zhang","year":"2004"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1606431\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T09:40:08Z","timestamp":1762940408000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1606431\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,6]]},"references-count":74,"alternative-id":["10.3389\/frai.2025.1606431"],"URL":"https:\/\/doi.org\/10.3389\/frai.2025.1606431","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,6]]},"article-number":"1606431"}}