{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T14:53:41Z","timestamp":1768402421163,"version":"3.49.0"},"reference-count":70,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2018,7,26]],"date-time":"2018-07-26T00:00:00Z","timestamp":1532563200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>The success of the Silk Road has prompted the growth of many Dark Web marketplaces. This exponential growth has provided criminal enterprises with new outlets to sell illicit items. Thus, the Dark Web has generated great interest from academics and governments who have sought to unveil the identities of participants in these highly lucrative, yet illegal, marketplaces. Traditional Web scraping methodologies and investigative techniques have proven to be inept at unmasking these marketplace participants. This research provides an analytical framework for automating Dark Web scraping and analysis with free tools found on the World Wide Web. Using a case study marketplace, we successfully tested a Web crawler, developed using AppleScript, to retrieve the account information for thousands of vendors and their respective marketplace listings. This paper clearly details why AppleScript was the most viable and efficient method for scraping Dark Web marketplaces. The results from our case study validate the efficacy of our proposed analytical framework, which has relevance for academics studying this growing phenomenon and for investigators examining criminal activity on the Dark Web.<\/jats:p>","DOI":"10.3390\/info9080186","type":"journal-article","created":{"date-parts":[[2018,7,26]],"date-time":"2018-07-26T11:32:07Z","timestamp":1532604727000},"page":"186","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":38,"title":["A Framework for More Effective Dark Web Marketplace Investigations"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5166-3240","authenticated-orcid":false,"given":"Darren","family":"Hayes","sequence":"first","affiliation":[{"name":"Seidenberg School of Computer Science &amp; Information Systems, Pace University, One Pace Plaza, New York, NY 10038, USA"},{"name":"Dipartimento di Ingegneria Meccanica e Aerospaziale, Sapienza Universit\u00e0 di Roma, Via Eudossiana 18, 00184 Roma, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5628-731X","authenticated-orcid":false,"given":"Francesco","family":"Cappa","sequence":"additional","affiliation":[{"name":"Department of Business and Management, LUISS Guido Carli University, Viale Pola 12, 00198 Roma, Italy"}]},{"given":"James","family":"Cardon","sequence":"additional","affiliation":[{"name":"Seidenberg School of Computer Science &amp; Information Systems, Pace University, One Pace Plaza, New York, NY 10038, USA"}]}],"member":"1968","published-online":{"date-parts":[[2018,7,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1109\/MC.2017.110","article-title":"Shining Light on the Dark Web","volume":"50","author":"Hurlburt","year":"2017","journal-title":"Computer"},{"key":"ref_2","unstructured":"Chertoff, M., and Simon, T. (2015). The Impact of the Dark Web on Internet Governance and Cyber Security, Centre for International Governance Innovation and Chatham House."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1080\/1057610X.2015.1119546","article-title":"Going dark: Terrorism on the dark web","volume":"39","author":"Weimann","year":"2016","journal-title":"Stud. Conf. Terr."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/S1361-3723(09)70150-2","article-title":"Darknets","volume":"2009","year":"2009","journal-title":"Comput. Fraud Secur."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Song, J., Lee, Y., Choi, J.W., Gil, J.M., Han, J., and Choi, S.S. (2017). Practical in-depth analysis of ids alerts for tracing and identifying potential attackers on darknet. Sustainability, 9.","DOI":"10.3390\/su9020262"},{"key":"ref_6","first-page":"95","article-title":"Darknet Mining and Game Theory for Enhanced Cyber Threat Intelligence","volume":"1","author":"Robertson","year":"2016","journal-title":"Cyber D\u00e9f. Rev."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Robertson, J., Diab, A., Marin, E., Nunes, E., Paliath, V., Shakarian, J., and Shakarian, P. (2017). Darkweb Cyber Threat Intelligence Mining, Cambridge University Press.","DOI":"10.1017\/9781316888513"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/S1353-4858(14)70042-X","article-title":"Unveiling the dark web","volume":"2014","author":"Bradbury","year":"2014","journal-title":"Netw. Secur."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.forsciint.2016.02.045","article-title":"Studying illicit drug trafficking on Darknet markets: Structure and organisation from a Canadian perspective","volume":"264","author":"Rhumorbarbe","year":"2016","journal-title":"Forensic Sci. Int."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Jardine, E. (2017). Privacy, censorship, data breaches and Internet freedom: The drivers of support and opposition to Dark Web technologies. New Media Soc.","DOI":"10.1177\/1461444817733134"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/S0262-4079(15)31855-8","article-title":"Shopping on the dark net","volume":"228","author":"George","year":"2015","journal-title":"New Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1016\/j.comnet.2015.03.008","article-title":"Internet Censorship detection: A survey","volume":"83","author":"Aceto","year":"2015","journal-title":"Comput. Netw."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Pescape, A., Montieri, A., Aceto, G., and Ciuonzo, D. (2017, January 4\u20138). Anonymity Services Tor, I2P, JonDonym: Classifying in the Dark (Web). Proceedings of the 2017 29th International Teletraffic Congress (ITC 29), Genoa, Italy.","DOI":"10.23919\/ITC.2017.8064342"},{"key":"ref_14","unstructured":"(2017, September 09). Tor Project. Available online: https:\/\/www.torproject.org\/."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Scholz, R.W. (2016). Sustainable digital environments: What major challenges is humankind facing?. Sustainability, 8.","DOI":"10.3390\/su8080726"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.forsciint.2016.08.032","article-title":"Buying drugs on a Darknet market: A better deal? Studying the online illicit drug market through the analysis of digital, physical and chemical data","volume":"267","author":"Rhumorbarbe","year":"2016","journal-title":"Forensic Sci. Int."},{"key":"ref_17","unstructured":"European Monitoring Center for Drugs and Drug Addiction (2017). European Drug Report\u2014Trends and Developments, European Monitoring Center for Drugs and Drug Addiction."},{"key":"ref_18","first-page":"40","article-title":"The 21st Century DarkNet Market: Lessons from the Fall of Silk Road","volume":"10","author":"Lacson","year":"2016","journal-title":"Int. J. Cyber Criminol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1016\/j.forsciint.2017.08.026","article-title":"Forensic drug intelligence and the rise of cryptomarkets. Part I: Studying the Australian virtual market","volume":"279","author":"Morelato","year":"2017","journal-title":"Forensic Sci. Int."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1016\/j.drugpo.2013.01.005","article-title":"\u201cSilk Road\u201d, the virtual drug marketplace: A single case study of user experiences","volume":"24","author":"Bingham","year":"2013","journal-title":"Int. J. Drug Policy"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ahmed, M., and Litchfield, A.T. (2016). Taxonomy for Identification of Security Issues in Cloud Computing Environments. J. Comput. Inf. Syst., 1\u201310.","DOI":"10.1080\/08874417.2016.1192520"},{"key":"ref_22","first-page":"12","article-title":"US guns sold in Europe via dark web","volume":"235","author":"Revell","year":"2017","journal-title":"New Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1111\/jcpt.12628","article-title":"The \u201cDarknet\u201d: The new street for street drugs","volume":"42","author":"Pergolizzi","year":"2017","journal-title":"J. Clin. Pharm. Ther."},{"key":"ref_24","first-page":"21","article-title":"Financing the dark web","volume":"60","author":"Kirkpatrick","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"969","DOI":"10.3233\/THC-161244","article-title":"Darknet and bitcoin, the obscure and anonymous side of the internet in healthcare","volume":"24","author":"Masoni","year":"2016","journal-title":"Technol. Health Care"},{"key":"ref_26","unstructured":"Soska, K., and Christin, N. (2015, January 12\u201314). Measuring the Longitudinal Evolution of the Online Anonymous Marketplace Ecosystem. Proceedings of the 24th USENIX Security Symposium, Washington, DC, USA."},{"key":"ref_27","unstructured":"(2018, March 02). Tor Metrics. Available online: https:\/\/metrics.torproject.org\/hidserv-dir-onions-seen.html."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1109\/MNET.2012.6135854","article-title":"Issues and future directions in traffic classification","volume":"26","author":"Dainotti","year":"2012","journal-title":"IEEE Netw."},{"key":"ref_29","first-page":"122","article-title":"Early classification of network traffic through multi-classification","volume":"Volume 6613","author":"Dainotti","year":"2011","journal-title":"Traffic Monitoring and Analysis TMA 2011. Lecture Notes in Computer Science"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Park, J.H., and Chao, H.C. (2017). Advanced IT-based Future sustainable computing. Sustainability, 9.","DOI":"10.3390\/su9050757"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1347","DOI":"10.1002\/asi.20838","article-title":"Uncovering the dark Web: A case study of Jihad on the Web","volume":"59","author":"Chen","year":"2008","journal-title":"J. Am. Soc. Inf. Sci. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zulkarnine, A.T., Frank, R., Monk, B., Mitchell, J., and Davies, G. (2016, January 27\u201330). Surfacing collaborated networks in dark web to find illicit and criminal content. Proceedings of the 2016 IEEE Conference on Intelligence and Security Informatics (ISI), Tucson, AZ, USA.","DOI":"10.1109\/ISI.2016.7745452"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Spitters, M., Klaver, F., Koot, G., and van Staalduinen, M. (2015, January 7\u20139). Authorship Analysis on Dark Marketplace Forums. Proceedings of the 2015 European Intelligence and Security Informatics Conference, Manchester, UK.","DOI":"10.1109\/EISIC.2015.47"},{"key":"ref_34","unstructured":"(2018, June 28). DarkOwl. Available online: https:\/\/www.darkowl.com\/."},{"key":"ref_35","unstructured":"(2018, June 28). Intelliagg. Available online: https:\/\/www.intelliagg.com\/."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1177\/000312240507000103","article-title":"The discourse of globalization: Framing and sensemaking of an emerging concept","volume":"70","author":"Fiss","year":"2005","journal-title":"Am. Sociol. Rev."},{"key":"ref_37","first-page":"3","article-title":"Research Framework for AIS Grand Vision of the Bright ICT Initiative","volume":"39","author":"Lee","year":"2015","journal-title":"MIS Q."},{"key":"ref_38","unstructured":"Moloney, P. (2016). Dark Net Drug Marketplaces Begin to Emulate Organised Street Crime, National Drug and Alcohol Research Centre."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1113","DOI":"10.1016\/j.drugpo.2015.01.008","article-title":"Evaluating drug trafficking on the Tor Network: Silk Road 2, the sequel","volume":"26","author":"Dolliver","year":"2015","journal-title":"Int. J. Drug Policy"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1016\/j.is.2010.11.003","article-title":"The dark side of the Internet: Attacks, costs and responses","volume":"36","author":"Kim","year":"2011","journal-title":"Inf. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Koch, R. (2018). The Darkweb\u2014A Growing Risk for Military Operations?. Information, in press.","DOI":"10.23919\/CYCON.2019.8756708"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"900","DOI":"10.1111\/cobi.12707","article-title":"Assessing the extent and nature of wildlife trade on the dark web","volume":"30","author":"Harrison","year":"2016","journal-title":"Conserv. Boil."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.diin.2017.12.003","article-title":"Criminal motivation on the dark web: A categorisation model for law enforcement","volume":"24","author":"Dalins","year":"2018","journal-title":"Digit. Investig."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.drugpo.2013.10.009","article-title":"Responsible vendors, intelligent consumers: Silk Road, the online revolution in drug trading","volume":"25","author":"Bingham","year":"2014","journal-title":"Int. J. Drug Policy"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Christin, N. (2013, January 13\u201317). Nicolas Traveling the silk road. Proceedings of the 22nd International Conference on World Wide Web\u2014WWW \u201913, Rio de Janeiro, Brazil.","DOI":"10.1145\/2488388.2488408"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Edwards, M.J., Rashid, A., and Rayson, P. (2014, January 24\u201326). A Service-Indepenent Model for Linking Online User Profile Information. Proceedings of the 2014 IEEE Joint Intelligence and Security Informatics Conference, The Hague, The Netherlands.","DOI":"10.1109\/JISIC.2014.55"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1016\/j.diin.2014.08.001","article-title":"I shop online\u2014Recreationally! Internet anonymity and Silk Road enabling drug use in Australia","volume":"11","author":"Phelps","year":"2014","journal-title":"Digit. Investig."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1007\/s10796-010-9277-6","article-title":"A multi-region empirical study on the internet presence of global extremist organizations","volume":"13","author":"Qin","year":"2011","journal-title":"Inf. Syst. Front."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Lacey, D., and Salmon, P.M. (2015, January 2\u20137). It\u2019s Dark in There: Using Systems Analysis to Investigate Trust and Engagement in Dark Web Forums. Proceedings of the International Conference on Engineering Psychology and Cognitive Ergonomics, Los Angeles, CA, USA.","DOI":"10.1007\/978-3-319-20373-7_12"},{"key":"ref_50","unstructured":"UNODC (2016). World Drug Report, UNODC."},{"key":"ref_51","unstructured":"(2017, September 09). Reddit. Available online: https:\/\/www.reddit.com\/r\/darknetmarkets."},{"key":"ref_52","unstructured":"Zetter, K. (2017, October 26). New \u201cGoogle\u201d for the Dark Web Makes Buying Dope and Guns Easy. Available online: https:\/\/www.wired.com\/2014\/04\/grams-search-engine-dark-web\/."},{"key":"ref_53","unstructured":"Zillman, M.P. (2017). Deep Web Research and Discovery Resources 2017, Deep Web."},{"key":"ref_54","unstructured":"(2017, October 05). Defense Advanced Research Project Agency Memex. Available online: https:\/\/www.darpa.mil\/program\/memex."},{"key":"ref_55","unstructured":"Zetter, K. (2017, October 26). Darpa Is Developing a Search Engine for the Dark Web|WIRED. Available online: https:\/\/www.wired.com\/2015\/02\/darpa-memex-dark-web\/."},{"key":"ref_56","unstructured":"Mahto, D.K., and Singh, L. (2016, January 16\u201318). A Dive into Web Scraper World. Proceedings of the 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Baravalle, A., Lopez, M.S., and Lee, S.W. (2017, January 11\u201320). Mining the Dark Web: Drugs and Fake Ids. Proceedings of the IEEE International Conference on Data Mining Workshops, ICDMW, New Orleans, LA, USA.","DOI":"10.1109\/ICDMW.2016.0056"},{"key":"ref_58","first-page":"975","article-title":"Social Media Forensics for Hate Speech Opinion Mining","volume":"155","author":"Wanjala","year":"2016","journal-title":"Int. J. Comput. Appl."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"518","DOI":"10.3390\/fi6030518","article-title":"ARCOMEM Crawling Architecture","volume":"6","author":"Plachouras","year":"2014","journal-title":"Future Internet"},{"key":"ref_60","unstructured":"(2018, May 30). Unix & Linux Stack Exchange Anonymous Url Navigation in Command Line?. Available online: https:\/\/unix.stackexchange.com\/questions\/87156\/anonymous-url-navigation-in-command-line."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1007\/s10515-016-0202-1","article-title":"NLCI: A natural language command interpreter","volume":"24","author":"Landhauber","year":"2016","journal-title":"Autom. Softw. Eng."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Hayes, D., and Cappa, F. (2018). Open Source Intelligence for Risk Assessment. Bus. Horiz., in press.","DOI":"10.1016\/j.bushor.2018.02.001"},{"key":"ref_63","unstructured":"(2017, September 11). Paterva Home Maltego. Available online: https:\/\/www.paterva.com\/web7\/."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1111\/isj.12070","article-title":"Special issue on \u2018Dark side of information technology use\u2019: An introduction and a framework for research","volume":"25","author":"Tarafdar","year":"2015","journal-title":"Inf. Syst. J."},{"key":"ref_65","unstructured":"George, J.F., Derrick, D., Marett, K., Harrison, A., and Thatcher, J.B. (2016, January 11\u201314). The dark internet: Without darkness there is no light. Proceedings of the AMCIS 2016: Surfing the IT Innovation Wave\u201422nd Americas Conference on Information Systems, San Diego, CA, USA."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s10796-010-9271-z","article-title":"Tracking and disrupting dark networks: Challenges of data collection and analysis","volume":"13","author":"Roberts","year":"2011","journal-title":"Inf. Syst. Front."},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.ijcip.2008.08.006","article-title":"Security and the US rail infrastructure","volume":"1","author":"Hartong","year":"2008","journal-title":"Int. J. Crit. Infrastruct. Prot."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.ijcip.2011.02.001","article-title":"May the US government monitor private critical infrastructure assets to combat foreign cyberspace threats?","volume":"4","author":"Rice","year":"2011","journal-title":"Int. J. Crit. Infrastruct. Prot."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1016\/j.bushor.2016.03.015","article-title":"Business and cyber peace: We need you!","volume":"59","author":"Shackelford","year":"2016","journal-title":"Bus. Horiz."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1016\/j.bushor.2016.08.005","article-title":"Cybersecurity in 2016: People, technology, and processes","volume":"59","author":"Parent","year":"2016","journal-title":"Bus. Horiz."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/9\/8\/186\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:14:25Z","timestamp":1760195665000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/9\/8\/186"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,26]]},"references-count":70,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2018,8]]}},"alternative-id":["info9080186"],"URL":"https:\/\/doi.org\/10.3390\/info9080186","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,26]]}}}