{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T12:06:47Z","timestamp":1773403607278,"version":"3.50.1"},"reference-count":30,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2023,3,8]],"date-time":"2023-03-08T00:00:00Z","timestamp":1678233600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"national funds through FCT\/MCTES (PIDDAC)","award":["LA\/P\/0045\/2020 (ALiCE)"],"award-info":[{"award-number":["LA\/P\/0045\/2020 (ALiCE)"]}]},{"name":"national funds through FCT\/MCTES (PIDDAC)","award":["UIDB\/50020\/2020"],"award-info":[{"award-number":["UIDB\/50020\/2020"]}]},{"name":"national funds through FCT\/MCTES (PIDDAC)","award":["UIDP\/50020\/2020 (LSRE-LCM)"],"award-info":[{"award-number":["UIDP\/50020\/2020 (LSRE-LCM)"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Foods"],"abstract":"<jats:p>Flavor is the focal point in the flavor industry, which follows social tendencies and behaviors. The research and development of new flavoring agents and molecules are essential in this field. However, the development of natural flavors plays a critical role in modern society. Considering this, the present work proposes a novel framework based on scientific machine learning to undertake an emerging problem in flavor engineering and industry. It proposes a combining system composed of generative and reinforcement learning models. Therefore, this work brings an innovative methodology to design new flavor molecules. The molecules were evaluated regarding synthetic accessibility, the number of atoms, and the likeness to a natural or pseudo-natural product. This work brings as contributions the implementation of a web scraper code to sample a flavors database and the integration of two scientific machine learning techniques in a complex system as a framework. The implementation of the complex system instead of the generative model by itself obtained 10% more molecules within the optimal results. The designed molecules obtained as an output of the reinforcement learning model\u2019s generation were assessed regarding their existence or not in the market and whether they are already used in the flavor industry or not. Thus, we corroborated the potentiality of the framework presented for the search of molecules to be used in the development of flavor-based products.<\/jats:p>","DOI":"10.3390\/foods12061147","type":"journal-article","created":{"date-parts":[[2023,3,9]],"date-time":"2023-03-09T01:35:37Z","timestamp":1678325737000},"page":"1147","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["A Reinforcement Learning Framework to Discover Natural Flavor Molecules"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3092-8136","authenticated-orcid":false,"given":"Luana P.","family":"Queiroz","sequence":"first","affiliation":[{"name":"LSRE-LCM\u2014Laboratory of Separation and Reaction Engineering-Laboratory of Catalysis and Materials, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"},{"name":"ALiCE\u2014Associate Laboratory in Chemical Engineering, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0796-8116","authenticated-orcid":false,"given":"Carine M.","family":"Rebello","sequence":"additional","affiliation":[{"name":"Chemical Engineering Department, Polytechnic School Federal University of Bahia, Salvador 40210-630, Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1397-9628","authenticated-orcid":false,"given":"Erbet A.","family":"Costa","sequence":"additional","affiliation":[{"name":"Chemical Engineering Department, Polytechnic School Federal University of Bahia, Salvador 40210-630, Brazil"}]},{"given":"Vin\u00edcius V.","family":"Santana","sequence":"additional","affiliation":[{"name":"LSRE-LCM\u2014Laboratory of Separation and Reaction Engineering-Laboratory of Catalysis and Materials, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"},{"name":"ALiCE\u2014Associate Laboratory in Chemical Engineering, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"}]},{"given":"Bruno C. L.","family":"Rodrigues","sequence":"additional","affiliation":[{"name":"LSRE-LCM\u2014Laboratory of Separation and Reaction Engineering-Laboratory of Catalysis and Materials, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"},{"name":"ALiCE\u2014Associate Laboratory in Chemical Engineering, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0715-4761","authenticated-orcid":false,"given":"Al\u00edrio E.","family":"Rodrigues","sequence":"additional","affiliation":[{"name":"LSRE-LCM\u2014Laboratory of Separation and Reaction Engineering-Laboratory of Catalysis and Materials, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"},{"name":"ALiCE\u2014Associate Laboratory in Chemical Engineering, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4269-1420","authenticated-orcid":false,"given":"Ana M.","family":"Ribeiro","sequence":"additional","affiliation":[{"name":"LSRE-LCM\u2014Laboratory of Separation and Reaction Engineering-Laboratory of Catalysis and Materials, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"},{"name":"ALiCE\u2014Associate Laboratory in Chemical Engineering, Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0963-6449","authenticated-orcid":false,"given":"Idelfonso B. R.","family":"Nogueira","sequence":"additional","affiliation":[{"name":"Chemical Engineering Department, Norwegian University of Science and Technology, Sem S\u00e6landsvei 4, Kjemiblokk 5, N-7491 Trondheim, Norway"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,8]]},"reference":[{"key":"ref_1","unstructured":"The Editors of Encyclopaedia Britannica (2022, June 13). Flavour. Available online: https:\/\/www.britannica.com\/topic\/flavor."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Reineccius, G. (2006). Flavor Chemistry and Technology, CRC Press. [2nd ed.].","DOI":"10.1201\/9780203485347"},{"key":"ref_3","unstructured":"Fortune Business Insights (2022). Food Flavors Market Size, Share & COVID-19 Impact Analysis, By Type (Natural and Synthetic), by Application (Bakery, Beverages, Confectionery, Dairy, Convenience Food, Snacks, and Others), and Regional Forecast, 2021\u20132028, Available online: https:\/\/www.fortunebusinessinsights.com\/food-flavors-market-102745."},{"key":"ref_4","unstructured":"Sumesh Kumar, R.D. (2021). Food Flavors Market by Type (Natural, and Artificial), and End-User (Beverages, Dairy & Frozen Products, Bakery & Confectionery, Savory & Snacks, Animal & Pet Food): Global Opportunity Analysis and Industry Forecast, 2021\u20132030, Allied Market Research."},{"key":"ref_5","unstructured":"(2021). Flavors & Fragrances Market by Ingredients (Natural, Synthetic), End use (Beverage, Savory & Snacks, Bakery, Dairy Products, Confectionery, Consumer Products, Fine Fragrances), and Region (Asia Pacific, North America, Europe)\u2014Global Forecast to 2026, Markets and Markets. Available online: https:\/\/www.marketsandmarkets.com\/Market-Reports\/flavors-fragrance-market-175163912.html."},{"key":"ref_6","unstructured":"(2022, May 23). Regulation (EC) No 1334\/2008 of the European Parliament and of the Council of 16 December 2008 on Flavourings and Certain Food Ingredients with Flavouring Properties for Use in and on Foods and Amending Council Regulation (EEC) No 1601\/91, Regulat. Available online: http:\/\/data.europa.eu\/eli\/reg\/2008\/1334\/oj."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Bi, K., Zhang, D., Qiu, T., and Huang, Y. (2020). GC-MS fingerprints profiling using machine learning models for food flavor prediction. Processes, 8.","DOI":"10.3390\/pr8010023"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Fran\u00e7ois-lavet, V., Henderson, P., Islam, R., Bellemare, M.G., Fran\u00e7ois-lavet, V., Pineau, J., and Bellemare, M.G. (2018). An Introduction to Deep Reinforcement Learning. arXiv.","DOI":"10.1561\/9781680835397"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yeh, J.F., Su, P.H., Huang, S.H., and Chiang, T.C. (2017, January 25\u201327). Snake game AI: Movement rating functions and evolutionary algorithm-based optimization. Proceedings of the 2016 Conference on Technologies and Applications of Artificial Intelligence (TAAI), Hsinchu, Taiwan.","DOI":"10.1109\/TAAI.2016.7880166"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2642","DOI":"10.1021\/acs.nanolett.1c00416","article-title":"Surface-Enhanced Raman Scattering (SERS) Taster: A Machine-Learning-Driven Multireceptor Platform for Multiplex Profiling of Wine Flavors","volume":"21","author":"Leong","year":"2021","journal-title":"Nano Lett."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"025023","DOI":"10.1088\/2632-2153\/abcf91","article-title":"Graph networks for molecular design","volume":"2","author":"Mercado","year":"2021","journal-title":"Mach. Learn. Sci. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/j.aiopen.2021.01.001","article-title":"Graph Neural Networks: A Review of Methods and Applications","volume":"1","author":"Zhou","year":"2020","journal-title":"AI Open"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1007\/978-3-319-56991-8_32","article-title":"Deep Reinforcement Learning: An Overview","volume":"16","author":"Mousavi","year":"2018","journal-title":"Lect. Notes Networks Syst."},{"key":"ref_15","unstructured":"Shao, K., Tang, Z., Zhu, Y., Li, N., and Zhao, D. (2019). A Survey of Deep Reinforcement Learning in Video Games. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1186\/s13321-017-0235-x","article-title":"Molecular de-novo design through deep reinforcement learning","volume":"9","author":"Olivecrona","year":"2017","journal-title":"J. Cheminform."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"102269","DOI":"10.1016\/j.isci.2021.102269","article-title":"PaccMannRL: De novo generation of hit-like anticancer molecules from transcriptomic data via reinforcement learning","volume":"24","author":"Born","year":"2021","journal-title":"iScience"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Queiroz, L.P., Rebello, C.M., Costa, E.A., Santana, V.V., Rodrigues, B.C.L., Rodrigues, A.E., Ribeiro, A.M., and Nogueira, I.B.R. (2022). Generating Flavors Using Scientific Machine Learning, preprint.","DOI":"10.21203\/rs.3.rs-1998750\/v1"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"10752","DOI":"10.1038\/s41598-019-47148-x","article-title":"Optimization of Molecules via Deep Reinforcement Learning","volume":"9","author":"Zhou","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_20","unstructured":"Bagler, G. (2022, April 05). FlavorDB. Available online: https:\/\/cosylab.iiitd.edu.in\/flavordb\/."},{"key":"ref_21","unstructured":"Li, Y., Zemel, R., Brockschmidt, M., and Tarlow, D. (2016, January 2\u20134). Gated graph sequence neural networks. Proceedings of the 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1186\/s13321-021-00498-z","article-title":"Diversity oriented Deep Reinforcement Learning for targeted molecule generation","volume":"13","author":"Pereira","year":"2021","journal-title":"J. Cheminform."},{"key":"ref_23","unstructured":"Nwankpa, C., Ijomah, W., Gachagan, A., and Marshall, S. (2018). Activation Functions: Comparison of trends in Practice and Research for Deep Learning. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1186\/1758-2946-1-8","article-title":"Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions","volume":"1","author":"Ertl","year":"2009","journal-title":"J. Cheminform."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1021\/ci700286x","article-title":"Natural product-likeness score and its application for prioritization of compound libraries","volume":"48","author":"Ertl","year":"2008","journal-title":"J. Chem. Inf. Model."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chen, Y., Stork, C., Hirte, S., and Kirchmair, J. (2019). NP-scout: Machine learning approach for the quantification and visualization of the natural product-likeness of small molecules. Biomolecules, 9.","DOI":"10.3390\/biom9020043"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1038\/s41557-019-0411-x","article-title":"Principle and design of pseudo-natural products","volume":"12","author":"Karageorgis","year":"2020","journal-title":"Nat. Chem."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1017\/S0962492900000015","article-title":"Buhmann Radial Basis Functions","volume":"9","author":"Buhmann","year":"2000","journal-title":"Acta Numer."},{"key":"ref_29","unstructured":"Atance, S.R., Diez, J.V., Engkvist, O., Olsson, S., and Mercado, R. (2021). ChemRxiv, Cambridge Open Engage."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"D1388","DOI":"10.1093\/nar\/gkaa971","article-title":"PubChem in 2021: New data content and improved web interfaces","volume":"49","author":"Kim","year":"2021","journal-title":"Nucleic Acids Res."}],"container-title":["Foods"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2304-8158\/12\/6\/1147\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:51:20Z","timestamp":1760122280000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2304-8158\/12\/6\/1147"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,8]]},"references-count":30,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["foods12061147"],"URL":"https:\/\/doi.org\/10.3390\/foods12061147","relation":{},"ISSN":["2304-8158"],"issn-type":[{"value":"2304-8158","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,8]]}}}