{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:18:43Z","timestamp":1760231923137,"version":"build-2065373602"},"reference-count":26,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,4,27]],"date-time":"2022-04-27T00:00:00Z","timestamp":1651017600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Malaysian Ministry of Higher Education under Fundamental Research Grant Scheme (FRGS)","award":["FRGS\/1\/2018\/ICT03\/UTAR\/03\/1"],"award-info":[{"award-number":["FRGS\/1\/2018\/ICT03\/UTAR\/03\/1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Data"],"abstract":"<jats:p>Generally, a courier company needs to employ a fleet of vehicles to travel through a number of locations in order to provide efficient parcel delivery services. The route planning of these vehicles can be formulated as a vehicle routing problem (VRP). Most existing VRP algorithms assume that the traveling durations between locations are time invariant; thus, they normally use only a set of estimated travel times (ETTs) to plan the vehicles\u2019 routes; however, this is not realistic because the traffic pattern in a city varies over time. One solution to tackle the problem is to use different sets of ETTs for route planning in different time periods, and these data are collectively called the time-dependent estimated travel times (TD-ETTs). This paper focuses on a low-cost and robust solution to effectively scrape, process, clean, and analyze the TD-ETT data from free web-mapping services in order to gain the knowledge of the traffic pattern in a city in different time periods. To achieve the abovementioned goal, our proposed framework contains four phases, namely, (i) Full Data Scraping, (ii) Data Pre-Processing and Analysis, (iii) Fast Data Scraping, and (iv) Data Patching and Maintenance. In our experiment, we used the above framework to obtain the TD-ETT data across 68 locations in Penang, Malaysia, for six months. We then fed the data to a VRP algorithm for evaluation. We found that the performance of our low-cost approach is comparable with that of using the expensive paid data.<\/jats:p>","DOI":"10.3390\/data7050054","type":"journal-article","created":{"date-parts":[[2022,4,27]],"date-time":"2022-04-27T13:40:57Z","timestamp":1651066857000},"page":"54","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["An Estimated-Travel-Time Data Scraping and Analysis Framework for Time-Dependent Route Planning"],"prefix":"10.3390","volume":"7","author":[{"given":"Hong-Le","family":"Tee","sequence":"first","affiliation":[{"name":"Department of Computer Science, Faculty of Information and Communication Technology, Universiti Tunku Abdul Rahman, Kampar Campus, Perak 31900, Malaysia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8853-7755","authenticated-orcid":false,"given":"Soung-Yue","family":"Liew","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Faculty of Information and Communication Technology, Universiti Tunku Abdul Rahman, Kampar Campus, Perak 31900, Malaysia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1288-4141","authenticated-orcid":false,"given":"Chee-Siang","family":"Wong","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Faculty of Information and Communication Technology, Universiti Tunku Abdul Rahman, Kampar Campus, Perak 31900, Malaysia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5230-671X","authenticated-orcid":false,"given":"Boon-Yaik","family":"Ooi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Faculty of Information and Communication Technology, Universiti Tunku Abdul Rahman, Kampar Campus, Perak 31900, Malaysia"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,27]]},"reference":[{"key":"ref_1","unstructured":"Watts, J. (2021, October 28). How Much Does GPS Fleet Tracking Cost? The Ultimate Guide. Available online: https:\/\/www.expertmarket.com\/fleet-management\/costs."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Tee, H.L., Liew, S.Y., Wong, C.S., and Ooi, B.Y. (2021, January 13\u201315). Cost-effective scraping and processing of real-time traffic data for route planning. Proceedings of the 2021 International Conference on Computer & Information Sciences (ICCOINS), Kuching, Malaysia.","DOI":"10.1109\/ICCOINS49721.2021.9497145"},{"key":"ref_3","first-page":"310","article-title":"Capacitated vehicle routing problem","volume":"7","author":"Ibrahim","year":"2019","journal-title":"Int. J. Res."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Qi, Y., and Cai, Y. (2021). Hybrid chaotic discrete bat algorithm with variable neighborhood search for vehicle routing problem in complex supply chain. Appl. Sci., 11.","DOI":"10.3390\/app112110101"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Peng, P. (2018, January 8\u20139). Hybrid tabu search algorithm for fleet size and mixed vehicle routing problem with three-dimensional loading constraints. Proceedings of the 11th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China.","DOI":"10.1109\/ISCID.2018.00073"},{"key":"ref_6","unstructured":"Okhrin, I., and Richter, K. (2007, January 5\u20137). The real-time vehicle routing problem. Proceedings of the International Conference of the German Operations Research Society (GOR), Saarbr\u00fccken, Germany."},{"key":"ref_7","unstructured":"(2021, September 16). iFleet Plans & Pricing. Available online: https:\/\/ifleet.my\/ifleet-gps-tracker-pricing."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1109\/TITS.2005.848362","article-title":"Optimal vehicle routing with real-time traffic information","volume":"6","author":"Kim","year":"2005","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Rathore, N., Jain, P.K., and Parida, M. (August, January 31). A routing model for emergency vehicles using the real time traffic data. Proceedings of the 2018 IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI), Singapore.","DOI":"10.1109\/SOLI.2018.8476771"},{"key":"ref_10","unstructured":"(2021, September 16). Distance Matrix API Usage and Billing. Available online: https:\/\/developers.google.com\/maps\/documentation\/distance-matrix\/usage-and-billing?hl=en."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Thomas, D.M., and Mathur, S. (2019, January 12\u201314). Data analysis by web scraping using Python. Proceedings of the 3rd International Conference on Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, India.","DOI":"10.1109\/ICECA.2019.8822022"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Upadhyay, S., Pant, V., Bhasin, S., and Pattanshetti, M.K. (2017, January 22\u201324). Articulating the construction of a web scraper for massive data extraction. Proceedings of the Second International Conference on Electrical, Computer and Communication Technologies (ICECCT), Coimbatore, India.","DOI":"10.1109\/ICECCT.2017.8117827"},{"key":"ref_13","unstructured":"(2021, September 16). What is Rate Limiting? | Rate Limiting and Bots. Available online: https:\/\/www.cloudflare.com\/learning\/bots\/what-is-rate-limiting."},{"key":"ref_14","unstructured":"Ghimiray, D. (2021, September 16). What is a Proxy Server and How Does It Work. Available online: https:\/\/www.avg.com\/en\/signal\/proxy-server-definition."},{"key":"ref_15","unstructured":"(2021, September 16). Residential Proxies Pricing. Available online: https:\/\/oxylabs.io\/pricing\/residential-proxy-pool."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Narayanan, A., Mitrovic, N., Asif, M.T., Dauwels, J., and Jaillet, P. (2015, January 15\u201318). Travel time estimation using speed predictions. Proceedings of the 2015 IEEE 18th International Conference on Intelligent Transportation Systems, Gran Canaria, Spain.","DOI":"10.1109\/ITSC.2015.364"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yang, Q., Gao, Z., Kong, X., Rahim, A., Wang, J., and Xia, F. (2015, January 10\u201314). Taxi operation optimization based on big traffic data. Proceedings of the IEEE 12th International Conference on Ubiquitous Intelligence and Computing and IEEE 12th International Conference on Autonomic and Trusted Computing and IEEE 15th International Conference on Scalable Computing and Communications and Its Associated Workshops (UIC-ATC-ScalCom), Beijing, China.","DOI":"10.1109\/UIC-ATC-ScalCom-CBDCom-IoP.2015.42"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2815","DOI":"10.1007\/s11227-016-1624-z","article-title":"Highway traffic accident prediction using VDS big data analysis","volume":"72","author":"Park","year":"2016","journal-title":"J. Supercomput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1007\/s41019-020-00151-z","article-title":"A survey of traffic prediction: From spatio-temporal data to intelligent transportation","volume":"6","author":"Yuan","year":"2021","journal-title":"Data Sci. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Noussan, M., Carioni, G., Sanvito, F.D., and Colombo, E. (2019). Urban mobility demand profiles: Time series for cars and bike-sharing use as a resource for transport and energy modeling. Data, 4.","DOI":"10.3390\/data4030108"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1287\/mnsc.6.1.80","article-title":"The truck dispatching problem","volume":"6","author":"Dantzig","year":"1959","journal-title":"Manag. Sci."},{"key":"ref_22","first-page":"1945","article-title":"Research on vehicle routing problem with soft time windows based on hybrid tabu search and scatter search algorithm","volume":"64","author":"Ge","year":"2020","journal-title":"Comput. Mater. Contin."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1016\/j.ejor.2008.03.023","article-title":"An approximate dynamic programming approach for the vehicle routing problem with stochastic demands","volume":"196","author":"Novoa","year":"2009","journal-title":"Eur. J. Oper. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1016\/S0305-0548(02)00051-5","article-title":"A genetic algorithm for the vehicle routing problem","volume":"30","author":"Baker","year":"2003","journal-title":"Comput. Oper. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1016\/j.proeng.2017.06.077","article-title":"Implementation of distribution model of an international company with use of simulation method","volume":"192","author":"Kostrzewski","year":"2017","journal-title":"Procedia Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"104820","DOI":"10.1016\/j.knosys.2019.06.028","article-title":"Generalizing identity-based string comparison metrics: Framework and techniques","volume":"187","author":"Cauteruccio","year":"2020","journal-title":"Knowl. Based Syst."}],"container-title":["Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5729\/7\/5\/54\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:01:44Z","timestamp":1760137304000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5729\/7\/5\/54"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,27]]},"references-count":26,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["data7050054"],"URL":"https:\/\/doi.org\/10.3390\/data7050054","relation":{},"ISSN":["2306-5729"],"issn-type":[{"type":"electronic","value":"2306-5729"}],"subject":[],"published":{"date-parts":[[2022,4,27]]}}}