{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T13:07:31Z","timestamp":1765544851553,"version":"build-2065373602"},"reference-count":55,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2022,8,19]],"date-time":"2022-08-19T00:00:00Z","timestamp":1660867200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Artificial Intelligence and Computer Science Laboratory\u2014LIACC","award":["UIDB\/00027\/2020"],"award-info":[{"award-number":["UIDB\/00027\/2020"]}]},{"name":"FCT\/MCTES (PIDDAC)","award":["UIDB\/00027\/2020"],"award-info":[{"award-number":["UIDB\/00027\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Applied Sciences"],"abstract":"<jats:p>In the digital world, the demand for better interactions between subscribers and companies is growing, creating the need for personalized and individualized experiences. With the exponential growth of email usage over the years, broad flows of campaigns are sent and received by subscribers, which reveals itself to be a problem for both companies and subscribers. In this work, subscribers are segmented by their behaviors and profiles, such as (i) open rates, (ii) click-through rates, (iii) frequency, and (iv) period of interactions with the companies. Different regressions are used: (i) Random Forest Regressor, (ii) Multiple Linear Regression, (iii) K-Neighbors Regressor, and (iv) Support Vector Regressor. All these regressions\u2019 results were aggregated into a final prediction achieved by an ensemble approach, which uses averaging and stacking methods. The use of Long Short-Term Memory is also considered in the presented case. The stacking model obtained the best performance, with an R2 score of 0.91 and a Mean Absolute Error of 0.204. This allows us to estimate the week\u2019s days with a half-day error difference. This work presents promising results for subscriber segmentation based on profile information for predicting the best period for email marketing. In the future, subscribers can be segmented using the Recency, Frequency and Monetary value, the Lifetime Value, or Stream Clustering approaches that allow more personalized and tailored experiences for subscribers. The latter tracks segments over time without costly recalculations and handles continuous streams of new observations without the necessity to recompile the entire model.<\/jats:p>","DOI":"10.3390\/app12168310","type":"journal-article","created":{"date-parts":[[2022,8,21]],"date-time":"2022-08-21T21:05:51Z","timestamp":1661115951000},"page":"8310","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["A Novel Approach for Send Time Prediction on Email Marketing"],"prefix":"10.3390","volume":"12","author":[{"given":"Carolina","family":"Ara\u00fajo","sequence":"first","affiliation":[{"name":"Faculty of Science and Technology, Fernando Pessoa University, 4249-004 Porto, Portugal"},{"name":"E-goi, 4450-190 Matosinhos, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0382-879X","authenticated-orcid":false,"given":"Christophe","family":"Soares","sequence":"additional","affiliation":[{"name":"Faculty of Science and Technology, Fernando Pessoa University, 4249-004 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5440-3225","authenticated-orcid":false,"given":"Ivo","family":"Pereira","sequence":"additional","affiliation":[{"name":"Faculty of Science and Technology, Fernando Pessoa University, 4249-004 Porto, Portugal"},{"name":"E-goi, 4450-190 Matosinhos, Portugal"},{"name":"ISRC\u2014Interdisciplinary Studies Research Center, 4200-072 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2665-8057","authenticated-orcid":false,"given":"Duarte","family":"Coelho","sequence":"additional","affiliation":[{"name":"E-goi, 4450-190 Matosinhos, Portugal"},{"name":"ISRC\u2014Interdisciplinary Studies Research Center, 4200-072 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0786-3362","authenticated-orcid":false,"given":"Miguel \u00c2ngelo","family":"Rebelo","sequence":"additional","affiliation":[{"name":"E-goi, 4450-190 Matosinhos, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0264-4710","authenticated-orcid":false,"given":"Ana","family":"Madureira","sequence":"additional","affiliation":[{"name":"ISRC\u2014Interdisciplinary Studies Research Center, 4200-072 Porto, Portugal"},{"name":"Institute of Engineering, Polytechnic of Porto, 4200-072 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,19]]},"reference":[{"key":"ref_1","first-page":"112","article-title":"Study On Artificial Intelligence in Marketing","volume":"6","author":"Deshmukh","year":"2019","journal-title":"Int. J. Adv. Innov. Res."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Abakouy, R., En-naimi, E.M., Haddadi, A.E., and Lotfi, E. (2019, January 2\u20134). Data-Driven Marketing: How Machine Learning Will Improve Decision-Making for Marketers. Proceedings of the 4th International Conference on Smart City Applications, Casablanca, Morocco.","DOI":"10.1145\/3368756.3369024"},{"key":"ref_3","unstructured":"Jonru, G., and Manrci, O. (2020). Highly Effective Techniques That Work Well In Email Marketing. Pardox J., 1, Available online: https:\/\/www.lantenay.com\/index.php\/pardox\/article\/view\/31."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1177\/10949968221095552","article-title":"Email Marketing as a Tool for Strategic Persuasion","volume":"57","author":"Thomas","year":"2022","journal-title":"J. Interact. Mark."},{"key":"ref_5","first-page":"232","article-title":"E-mail Marketing: Advantages, Disadvantages and Improving Techniques","volume":"2","author":"Fariborzi","year":"2012","journal-title":"Int. J. e-Educ. e-Bus. e-Manag. e-Learn."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1080\/01930826.2021.1883372","article-title":"Library marketing: Sending text messages and emails to online library users during COVID-19","volume":"61","author":"Rysavy","year":"2021","journal-title":"J. Libr. Adm."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1016\/j.intmar.2021.02.002","article-title":"The effectiveness of triggered email marketing in addressing browse abandonments","volume":"55","author":"Goic","year":"2021","journal-title":"J. Interact. Mark."},{"key":"ref_8","unstructured":"van Rijn, J. (2020, December 16). DMA National Client Email Report 2015. Available online: www.emailmonday.com\/dma-national-client-email-report-2015\/."},{"key":"ref_9","unstructured":"Salesforce (2020, December 17). Fourth Annual State of Marketing: Insights and Trends from 3.500 Global Marketing Leaders. Available online: www.salesforce.com\/content\/dam\/web\/en_us\/www\/assets\/pdf\/datasheets\/salesforce-research-fourth-annual-state-of-marketing.pdf."},{"key":"ref_10","unstructured":"Forsey, C. (2020, December 17). The Ultimate List of Email Marketing Stats for 2020. Available online: blog.hubspot.com\/marketing\/email-marketing-stats."},{"key":"ref_11","unstructured":"Bump, P. (2020, December 17). Why Consumers Subscribe and Unsubscribe from Email [New Data]. Available online: blog.hubspot.com\/marketing\/why-consumers-subscribe-to-email."},{"key":"ref_12","unstructured":"Monitor, C. (2020, December 17). The New Rules of Email Marketing. Available online: www.campaignmonitor.com\/resources\/guides\/email-marketing-new-rules."},{"key":"ref_13","first-page":"81","article-title":"Digital Marketing and Outdoor Advertising in Smart Cities","volume":"6","author":"Vangelov","year":"2022","journal-title":"Smart Cities Reg. Dev. (SCRD) J."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1016\/j.jretai.2021.03.001","article-title":"A framework for collaborative artificial intelligence in marketing","volume":"98","author":"Huang","year":"2022","journal-title":"J. Retail."},{"key":"ref_15","first-page":"189","article-title":"Software as a Service: Design and Build Lower Usage Cost Email Marketing for Hospitality Industry","volume":"7","author":"Sukarsa","year":"2020","journal-title":"Sci. J. Inform."},{"key":"ref_16","unstructured":"Monitor, C. (2020, December 23). What are KPIs in Email Marketing?. Available online: www.campaignmonitor.com\/resources\/knowledge-base\/what-are-kpis-in-email-marketing."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"117974","DOI":"10.1016\/j.eswa.2022.117974","article-title":"Leveraging email marketing: Using the subject line to anticipate the open rate","volume":"207","author":"Paulo","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_18","unstructured":"iPULLRANK (2020, December 18). Machine Learning for Marketers\u2014A Comprehensive Guide to Machine Learning. Available online: assets.ctfassets.net\/j5zy0n17n2ql\/2D4mX8PjV6iC6i8cIuSCwk\/23a4ebb99a6e9d5a82b2f03e1262f39d\/ml-whitepaper.pdf."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6669","DOI":"10.1109\/JSEN.2018.2851300","article-title":"Collaborative mobile sink sojourn time optimization scheme for cluster-based wireless sensor networks","volume":"18","author":"Gharaei","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"101058","DOI":"10.1016\/j.elerap.2021.101058","article-title":"Campaign participation prediction with deep learning","volume":"48","author":"Ayvaz","year":"2021","journal-title":"Electron. Commer. Res. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"101918","DOI":"10.1016\/j.jretconser.2019.101918","article-title":"Forecasting client retention\u2014A machine-learning approach","volume":"52","author":"Schaeffer","year":"2020","journal-title":"J. Retail. Consum. Serv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.intmar.2020.07.002","article-title":"LSTM response models for direct marketing analytics: Replacing feature engineering with deep learning","volume":"53","author":"Sarkar","year":"2021","journal-title":"J. Interact. Mark."},{"key":"ref_23","unstructured":"AT INTERNET (2021, April 08). Lead. Available online: www.atinternet.com\/en\/glossary\/lead\/."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"26","DOI":"10.7763\/IJMO.2020.V10.742","article-title":"Building a Cloud-based Regression Model to Predict Click-through Rate in Business Messaging Campaigns","volume":"10","author":"Deligiannis","year":"2020","journal-title":"Int. J. Model. Optim."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1613\/jair.4992","article-title":"A Primer on Neural Network Models for Natural Language Processing","volume":"57","author":"Goldberg","year":"2016","journal-title":"J. Artif. Intell. Res."},{"key":"ref_26","first-page":"90","article-title":"Predicting the Optimal Date and Time to Send Personalized Marketing Messages to Repeat Buyers","volume":"11","author":"Deligiannis","year":"2020","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Naik, N. (2017, January 11\u201313). Choice of effective messaging protocols for IoT systems: MQTT, CoAP, AMQP and HTTP. Proceedings of the 2017 IEEE International Systems Engineering Symposium (ISSE), Vienna, Austria.","DOI":"10.1109\/SysEng.2017.8088251"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Paralic, J., Kaszoni, T., and Ma\u010dina, J. (2020). Predicting suitable time for sending marketing emails. Information Systems Architecture and Technology, Proceedings of the 40th Anniversary International Conference on Information Systems Architecture and Technology\u2014ISAT 2019, Wroc\u0142aw, Poland, 15\u201317 September 2019, Springer International Publishing.","DOI":"10.1007\/978-3-030-30604-5_17"},{"key":"ref_30","unstructured":"Zhang, J. (2021). Dive into Decision Trees and Forests: A Theoretical Demonstration. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"16","DOI":"10.3847\/1538-3881\/aaf101","article-title":"Probabilistic Random Forest: A Machine Learning Algorithm for Noisy Data Sets","volume":"157","author":"Reis","year":"2018","journal-title":"Astron. J."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"012011","DOI":"10.1088\/1742-6596\/1142\/1\/012011","article-title":"Comparative study between decision tree and knn of data mining classification technique","volume":"1142","author":"Mohanapriya","year":"2018","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Concei\u00e7\u00e3o, A., and Gama, J. (2019, January 28\u201330). Main Factors Driving the Open Rate of Email Marketing Campaigns. Proceedings of the International Conference on Discovery Science, Split, Coratia.","DOI":"10.1007\/978-3-030-33778-0_12"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: A gradient boostingmachine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Stat."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Japkowicz, N., and Matwin, S. (2015, January 4\u20136). Predictive Analysis on Tracking Emails for Targeted Marketing. Proceedings of the International Conference on Discovery Science, Banff, AB, Canada.","DOI":"10.1007\/978-3-319-24282-8"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"N\u00e9dellec, C., and Rouveirol, C. (1998, January 21\u201323). Text categorization with Support Vector Machines: Learning with many relevant features. Proceedings of the Machine Learning: ECML-98, Chemnitz, Germany.","DOI":"10.1007\/BFb0026664"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Sinha, M., Vinay, V., and Singh, H. (2018, January 5\u20139). Modeling Time to Open of Emails with a Latent State for User Engagement Level. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, Los Angeles, CA, USA.","DOI":"10.1145\/3159652.3159683"},{"key":"ref_38","unstructured":"Fox, J. (2002). Cox proportional-hazards regression for survival data. An R and S-PLUS Companion to Applied Regression, Sage Publications, Inc."},{"key":"ref_39","unstructured":"Glazier, S.W. (2019). Sequential Survival Analysis with Deep Learning. [Master\u2019s Thesis, Brigham Young University]."},{"key":"ref_40","unstructured":"Singh, H., Sinha, M., Sinha, A.R., Garg, S., and Banerjee, N. (2020). An RNN-Survival Model to Decide Email Send Times. arXiv."},{"key":"ref_41","unstructured":"Singh, H., Sinha, M., Sinha, A., Garg, S., and Banerjee, N. (2019). Determining Strategic Digital Content Transmission Time Utilizing Recurrent Neural Networks and Survival Analysis, Adobe Inc."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/S0377-2217(03)00349-7","article-title":"Determining the optimal direct mailing frequency","volume":"158","author":"Piersma","year":"2004","journal-title":"Eur. J. Oper. Res."},{"key":"ref_43","unstructured":"Scikit Learn (2022, January 27). Scikit-Learn Machine Learning in Python. Available online: https:\/\/scikit-learn.org\/stable\/."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Wu, H., Cai, Y., Wu, Y., Zhong, R., Li, Q., Zheng, J., Lin, D., and Li, Y. (2017). Time series analysis of weekly influenza-like illness rate using a one-year period of factors in random forest regression. BioSci. Trends, 11.","DOI":"10.5582\/bst.2017.01035"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1007\/BF03325972","article-title":"Single hidden layer artificial neural network models versus multiple linear regression model in forecasting the time series of total ozone","volume":"4","author":"Bandyopadhyay","year":"2007","journal-title":"Int. J. Environ. Sci. Technol."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Al-Qahtani, F., and Crone, S. (2013, January 4\u20139). Multivariate k-nearest neighbour regression for time series data\u2014A novel algorithm for forecasting UK electricity demand. Proceedings of the International Joint Conference on Neural Networks, Dallas, TX, USA.","DOI":"10.1109\/IJCNN.2013.6706742"},{"key":"ref_47","unstructured":"Kunapul, G. (2022, January 03). Ensemble Methods for Machine Learning. Available online: https:\/\/www.manning.com\/books\/ensemble-methods-for-machine-learning."},{"key":"ref_48","unstructured":"Brownlee, J. (2022, January 03). Regression Metrics for Machine Learning. Available online: https:\/\/machinelearningmastery.com\/regression-metrics-for-machine-learning\/."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.indmarman.2018.01.033","article-title":"Digital, Social Media, and Mobile Marketing in industrial buying: Still in need of customer segmentation? Empirical evidence from Poland and Germany","volume":"73","author":"Pommeranz","year":"2018","journal-title":"Ind. Mark. Manag."},{"key":"ref_50","first-page":"1251","article-title":"RFM ranking\u2013An effective approach to customer segmentation","volume":"33","author":"Christy","year":"2021","journal-title":"J. King Saud Univ. Comput. Inf. Sci."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.eswa.2004.01.010","article-title":"Joint Optimization of Customer Segmentation and Marketing Policy to Maximize Long-Term Profitability","volume":"27","author":"Jonker","year":"2004","journal-title":"Expert Syst. Appl."},{"key":"ref_52","unstructured":"Optideia (2021, June 11). Lifetime Value. Available online: https:\/\/www.optimizely.com\/optimization-glossary\/lifetime-value\/."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"107677","DOI":"10.1016\/j.asoc.2021.107677","article-title":"GPHC: A heuristic clustering method to customer segmentation","volume":"111","author":"Sun","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"107924","DOI":"10.1016\/j.asoc.2021.107924","article-title":"Customer segmentation using K-means clustering and the adaptive particle swarm optimization algorithm","volume":"113","author":"Li","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1108\/JES-08-2015-0155","article-title":"Stock market development and income inequality","volume":"44","author":"Tsagkanos","year":"2017","journal-title":"J. Econ. Stud."}],"container-title":["Applied Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3417\/12\/16\/8310\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:12:32Z","timestamp":1760141552000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3417\/12\/16\/8310"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,19]]},"references-count":55,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["app12168310"],"URL":"https:\/\/doi.org\/10.3390\/app12168310","relation":{},"ISSN":["2076-3417"],"issn-type":[{"type":"electronic","value":"2076-3417"}],"subject":[],"published":{"date-parts":[[2022,8,19]]}}}