{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T11:46:08Z","timestamp":1768909568036,"version":"3.49.0"},"reference-count":41,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,10,22]],"date-time":"2024-10-22T00:00:00Z","timestamp":1729555200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computation"],"abstract":"<jats:p>Accurate predictions of molecular properties are crucial for advancements in drug discovery and materials science. However, this task is complex and requires effective representations of molecular structures. Recently, Graph Neural Networks (GNNs) have emerged as powerful tools for this purpose, demonstrating significant potential in modeling molecular data. Despite advancements in GNN predictive performance, existing methods lack clarity on how architectural choices, particularly activation functions, affect training dynamics and inference stages in interpreting the predicted results. To address this gap, this paper introduces a novel activation function called the Sine Linear Unit (SLU), aimed at enhancing the predictive capabilities of GNNs in the context of molecular property prediction. To demonstrate the effectiveness of SLU within GNN architecture, we conduct experiments on diverse molecular datasets encompassing various regression and classification tasks. Our findings indicate that SLU consistently outperforms traditional activation functions on hydration free energy (FreeSolv), inhibitory binding of human \u03b2 secretase (BACE), and blood brain barrier penetration (BBBP), achieving the superior performance in each task, with one exception on the GCN model using the QM9 data set. These results underscore SLU\u2019s potential to significantly improve prediction accuracy, making it a valuable addition to the field of molecular modeling.<\/jats:p>","DOI":"10.3390\/computation12110212","type":"journal-article","created":{"date-parts":[[2024,10,22]],"date-time":"2024-10-22T06:11:25Z","timestamp":1729577485000},"page":"212","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Optimizing GNN Architectures Through Nonlinear Activation Functions for Potent Molecular Property Prediction"],"prefix":"10.3390","volume":"12","author":[{"given":"Areen","family":"Rasool","sequence":"first","affiliation":[{"name":"Abdus Salam School of Mathematical Sciences, GC University, Lahore 54600, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8642-0660","authenticated-orcid":false,"given":"Jamshaid Ul","family":"Rahman","sequence":"additional","affiliation":[{"name":"Abdus Salam School of Mathematical Sciences, GC University, Lahore 54600, Pakistan"}]},{"given":"Quaid","family":"Iqbal","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Statistics, Binghamton University\u2014State University of New York, Binghamton, NY 13902, USA"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,22]]},"reference":[{"key":"ref_1","unstructured":"Jin, W., Coley, C., Barzilay, R., and Jaakkola, T. (2017). Predicting organic reaction outcomes with weisfeilerlehman network. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"6091","DOI":"10.1039\/C8SC02339E","article-title":"\u201cFound in Translation\u201d: Predicting outcomes of complex organic chemistry reactions using neural sequence-to-sequence models","volume":"9","author":"Schwaller","year":"2018","journal-title":"Chem. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, X.C., Wu, C.K., Yang, Z.J., Wu, Z.X., Yi, J.C., Hsieh, C.Y., Hou, T.J., and Cao, D.S. (2021). MGBERT: Leveraging unsupervised atomic representation learning for molecular property prediction. Brief. Bioinform., 22.","DOI":"10.1093\/bib\/bbab152"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3156","DOI":"10.1038\/s41467-021-23415-2","article-title":"Masked graph modeling for molecule generation","volume":"12","author":"Mahmood","year":"2021","journal-title":"Nat. Commun."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"8778","DOI":"10.1021\/acs.jmedchem.9b01129","article-title":"Practical high-quality electrostatic potential surfaces for drug discovery using a graph-convolutional deep neural network","volume":"63","author":"Rathi","year":"2019","journal-title":"J. Med. Chem."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1038\/nature25978","article-title":"Planning chemical syntheses with deep neural networks and symbolic AI","volume":"555","author":"Segler","year":"2018","journal-title":"Nature"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"7673","DOI":"10.1021\/acs.chemrev.6b00851","article-title":"Information retrieval and text mining technologies for chemistry","volume":"117","author":"Krallinger","year":"2017","journal-title":"Chem. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Bean, D.M., Wu, H., Iqbal, E., Dzahini, O., Ibrahim, Z.M., Broadbent, M., Stewart, R., and Dobson, R.J.B. (2017). Knowledge graph prediction of unknown adverse drug reactions and validation in electronic health records. Sci. Rep., 7.","DOI":"10.1038\/s41598-017-16674-x"},{"key":"ref_9","unstructured":"Fabian, B., Edlich, T., Gaspar, H., Segler, M., Meyers, J., Fiscato, M., and Ahmed, M. (2020). Molecular representation learning with language models and domain-relevant auxiliary tasks. arXiv."},{"key":"ref_10","unstructured":"Chithrananda, S., Grand, G., and Ramsundar, B. (2020). ChemBERTa: Large-scale self-supervised pretraining for molecular property prediction. arXiv."},{"key":"ref_11","unstructured":"Honda, S., Shi, S., and Ueda, H.R. (2019). Smiles transformer: Pre-trained molecular fingerprint for low data drug discovery. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, S., Guo, Y., Wang, Y., Sun, H., and Huang, J. (2019, January 7\u201310). Smiles-bert: Large scale unsupervised pre-training for molecular property prediction. Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, Niagara Falls, NY, USA.","DOI":"10.1145\/3307339.3342186"},{"key":"ref_13","unstructured":"Shin, B., Park, S., Kang, K., and Ho, J.C. (2019, January 8\u201310). Self-attention based molecule representation for predicting drug-target interaction. Proceedings of the Machine Learning for Healthcare Conference, Ann Arbor, MI, USA. PMLR."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"914923","DOI":"10.1021\/acs.jcim.8b00803","article-title":"Identifying structure-property relationships through smiles syntax analysis with self-attention mechanism","volume":"59","author":"Zheng","year":"2019","journal-title":"J. Chem. Inf. Model."},{"key":"ref_15","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_16","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_17","unstructured":"Wang, H., Li, W., Jin, X., Cho, K., Ji, H., Han, J., and Burke, M.D. (2021). Chemical-reaction-aware molecule representation learning. arXiv."},{"key":"ref_18","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Rahman, J.U., Danish, S., and Lu, D. (2024). Oscillator Simulation with Deep Neural Networks. Mathematics, 12.","DOI":"10.3390\/math12070959"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ul Rahman, J., Danish, S., and Lu, D. (2023). Deep Neural Network-Based Simulation of Sel\u2019kov Model in Glycolysis: A Comprehensive Analysis. Mathematics, 11.","DOI":"10.3390\/math11143216"},{"key":"ref_21","unstructured":"Kipf, T.N., and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv."},{"key":"ref_22","unstructured":"Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., and Dahl, G.E. (2017, January 6\u201311). Neural message passing for quantum chemistry. Proceedings of the International Conference on Machine Learning, Sydney, Australia. PMLR."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ishida, S., Miyazaki, T., Sugaya, Y., and Omachi, S. (2021). Graph neural networks with multiple feature extraction paths for chemical property estimation. Molecules, 26.","DOI":"10.3390\/molecules26113125"},{"key":"ref_24","unstructured":"Duvenaud, D.K., Maclaurin, D., Iparraguirre, J., Bombarell, R., Hirzel, T., Aspuru-Guzik, A., and Adams, R.P. (2015). Convolutional networks on graphs for learning molecular fingerprints. Adv. Neural Inf. Process. Syst., 28."},{"key":"ref_25","unstructured":"Gasteiger, J., Gro\u00df, J., and G\u00fcnnemann, S. (2020). Directional message passing for molecular graphs. arXiv."},{"key":"ref_26","unstructured":"Sch\u00fctt, K., Unke, O., and Gastegger, M. (2021, January 18\u201324). Equivariant message passing for the prediction of tensorial properties and molecular spectra. Proceedings of the International Conference on Machine Learning, Online. PMLR."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"241722","DOI":"10.1063\/1.5019779","article-title":"Schnet-a deep learning architecture for molecules and materials","volume":"148","author":"Sauceda","year":"2018","journal-title":"J. Chem. Phys."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1039\/C7SC02664A","article-title":"MoleculeNet: A benchmark for molecular machine learning","volume":"9","author":"Wu","year":"2018","journal-title":"Chem. Sci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1186\/s13321-024-00846-9","article-title":"MolPROP: Molecular Property prediction with multimodal language and graph fusion","volume":"16","author":"Rollins","year":"2024","journal-title":"J. Cheminform."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"5853","DOI":"10.1021\/acs.jcim.4c00586","article-title":"A New Fingerprint and Graph Hybrid Neural Network for Predicting Molecular Properties","volume":"64","author":"Zhang","year":"2024","journal-title":"J. Chem. Inf. Model."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1186\/s13321-023-00691-2","article-title":"Force field-inspired molecular representation learning for property prediction","volume":"15","author":"Ren","year":"2023","journal-title":"J. Cheminform."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1186\/s13321-020-0414-z","article-title":"A self-attention based message passing neural network for predicting molecular lipophilicity and aqueous solubility","volume":"12","author":"Tang","year":"2020","journal-title":"J. Cheminform."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1038\/35016072","article-title":"Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit","volume":"405","author":"Hahnloser","year":"2000","journal-title":"Nature"},{"key":"ref_34","unstructured":"Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 16\u201321). Rectifier nonlinearities improve neural network acoustic models. Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_35","unstructured":"Rahman, J.U., Makhdoom, F., and Lu, D. (2023). Amplifying Sine Unit: An Oscillatory Activation Function for Deep Neural Networks to Recover Nonlinear Oscillations Efficiently. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1080\/14786447408641058","article-title":"LVII. On the mathematical theory of isomers","volume":"47","author":"Cayley","year":"1874","journal-title":"Lond. Edinb. Dublin Philos. Mag. J. Sci."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1225","DOI":"10.1021\/acs.chemrestox.6b00135","article-title":"ToxCast chemical landscape: Paving the road to 21st century toxicology","volume":"29","author":"Richard","year":"2016","journal-title":"Chem. Res. Toxicol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"8732","DOI":"10.1021\/ja902302h","article-title":"970 million druglike small molecules for virtual screening in the chemical universe database GDB-13","volume":"131","author":"Blum","year":"2009","journal-title":"J. Am. Chem. Soc."},{"key":"ref_39","unstructured":"Sch\u00fctt, K., Kindermans, P.J., Sauceda Felix, H.E., Chmiela, S., Tkatchenko, A., and M\u00fcller, K.R. (2017). Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ddtec.2020.11.009","article-title":"A compact review of molecular property prediction with graph neural networks","volume":"37","author":"Wieder","year":"2020","journal-title":"Drug Discov. Today Technol."},{"key":"ref_41","unstructured":"Xu, K., Hu, W., Leskovec, J., and Jegelka, S. (2018). How powerful are graph neural networks?. arXiv."}],"container-title":["Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-3197\/12\/11\/212\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:17:40Z","timestamp":1760113060000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-3197\/12\/11\/212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,22]]},"references-count":41,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["computation12110212"],"URL":"https:\/\/doi.org\/10.3390\/computation12110212","relation":{},"ISSN":["2079-3197"],"issn-type":[{"value":"2079-3197","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,22]]}}}