{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:34:50Z","timestamp":1760243690905,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,8,30]],"date-time":"2022-08-30T00:00:00Z","timestamp":1661817600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2017YFC0907 505"],"award-info":[{"award-number":["2017YFC0907 505"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Network data analysis is a crucial method for mining complicated object interactions. In recent years, random walk and neural-language-model-based network representation learning (NRL) approaches have been widely used for network data analysis. However, these NRL approaches suffer from the following deficiencies: firstly, because the random walk procedure is based on symmetric node similarity and fixed probability distribution, the sampled vertices\u2019 sequences may lose local community structure information; secondly, because the feature extraction capacity of the shallow neural language model is limited, they can only extract the local structural features of networks; and thirdly, these approaches require specially designed mechanisms for different downstream tasks to integrate vertex attributes of various types. We conducted an in-depth investigation to address the aforementioned issues and propose a novel general NRL framework called dynamic structure and vertex attribute fusion network embedding, which firstly defines an asymmetric similarity and h-hop dynamic random walk strategy to guide the random walk process to preserve the network\u2019s local community structure in walked vertex sequences. Next, we train a self-attention-based sequence prediction model on the walked vertex sequences to simultaneously learn the vertices\u2019 local and global structural features. Finally, we introduce an attributes-driven Laplacian space optimization to converge the process of structural feature extraction and attribute feature extraction. The proposed approach is exhaustively evaluated by means of node visualization and classification on multiple benchmark datasets, and achieves superior results compared to baseline approaches.<\/jats:p>","DOI":"10.3390\/e24091213","type":"journal-article","created":{"date-parts":[[2022,8,30]],"date-time":"2022-08-30T21:25:18Z","timestamp":1661894718000},"page":"1213","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Improving Network Representation Learning via Dynamic Random Walk, Self-Attention and Vertex Attributes-Driven Laplacian Space Optimization"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1510-2478","authenticated-orcid":false,"given":"Shengxiang","family":"Hu","sequence":"first","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bofeng","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Shanghai Polytechnic University, Shanghai 201209, China"},{"name":"School of Computer Science and Technology, Kashi University, Kashi 844008, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hehe","family":"Lv","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Furong","family":"Chang","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Yangzhou Polytechnic Institute, Yangzhou 225127, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chenyang","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liangrui","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guobing","family":"Zou","sequence":"additional","affiliation":[{"name":"School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"117436","DOI":"10.1016\/j.eswa.2022.117436","article-title":"Graph-based modeling using association rule mining to detect influential users in social networks","volume":"2022","author":"Agouti","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"117128","DOI":"10.1016\/j.eswa.2022.117128","article-title":"Recommendation Networks of Homogeneous Products on An E-commerce Platform: Measurement and Competition Effects","volume":"201","author":"Zhu","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_3","first-page":"93","article-title":"Collective classification in network data","volume":"29","author":"Sen","year":"2008","journal-title":"AI Mag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4426","DOI":"10.1073\/pnas.1818013116","article-title":"Evolution of resilience in protein interactomes across the tree of life","volume":"116","author":"Zitnik","year":"2019","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1109\/TPAMI.2015.2456913","article-title":"Robust vertex classification","volume":"38","author":"Chen","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"115991","DOI":"10.1016\/j.eswa.2021.115991","article-title":"Link prediction by deep non-negative matrix factorization","volume":"188","author":"Chen","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/TBDATA.2018.2850013","article-title":"Network representation learning: A survey","volume":"6","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Big Data"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1616","DOI":"10.1109\/TKDE.2018.2807452","article-title":"A comprehensive survey of graph embedding: Problems, techniques, and applications","volume":"30","author":"Cai","year":"2018","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Perozzi, B., Al-Rfou, R., and Skiena, S. (2014, January 24\u201327). Deepwalk: Online learning of social representations. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.","DOI":"10.1145\/2623330.2623732"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Grover, A., and Leskovec, J. (2016, January 13\u201317). node2vec: Scalable feature learning for networks. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939754"},{"key":"ref_11","unstructured":"Ribeiro, L.F., Saverese, P.H., and Figueiredo, D.R. (2017, January 13\u201317). struc2vec: Learning node representations from structural identity. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada."},{"key":"ref_12","unstructured":"Sun, X., Song, Z., Dong, J., Yu, Y., Plant, C., and B\u00f6hm, C. (February, January 27). Network Structure and Transfer Behaviors Embedding via Deep Prediction Model. Proceedings of the AAAI Conference on Artificial Intelligence, Hilton Hawaiian Village, Honolulu, HI, USA."},{"key":"ref_13","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.patcog.2019.01.006","article-title":"Wider or deeper: Revisiting the resnet model for visual recognition","volume":"90","author":"Wu","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_15","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_16","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1373","DOI":"10.1162\/089976603321780317","article-title":"Laplacian eigenmaps for dimensionality reduction and data representation","volume":"15","author":"Belkin","year":"2003","journal-title":"Neural Comput."},{"key":"ref_18","unstructured":"Tang, L., and Liu, H. (July, January 28). Relational learning via latent social dimensions. Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France."},{"key":"ref_19","unstructured":"Donnat, C., Zitnik, M., Hallac, D., and Leskovec, J. (2018). Spectral graph wavelets for structural role similarity in networks. arXiv."},{"key":"ref_20","unstructured":"Yang, C., Liu, Z., Zhao, D., Sun, M., and Chang, E. (2015, January 25\u201331). Network representation learning with rich text information. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, D., Yin, J., Zhu, X., and Zhang, C. (2016, January 12\u201315). Homophily, structure, and content augmented network representation learning. Proceedings of the 2016 IEEE 16th International Conference on Data Mining (ICDM), Barcelona, Spain.","DOI":"10.1109\/ICDM.2016.0072"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"i60","DOI":"10.1093\/bioinformatics\/btu269","article-title":"Inductive matrix completion for predicting gene\u2013disease associations","volume":"30","author":"Natarajan","year":"2014","journal-title":"Bioinformatics"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Cao, S., Lu, W., and Xu, Q. (2015, January 18\u201323). Grarep: Learning graph representations with global structural information. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, Melbourne, Australia.","DOI":"10.1145\/2806416.2806512"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, J., Zhu, J., and Zhang, B. (2016, January 7\u201312). Discriminative deep random walk for network classification. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany.","DOI":"10.18653\/v1\/P16-1095"},{"key":"ref_25","first-page":"12","article-title":"Tri-party deep network representation","volume":"11","author":"Pan","year":"2016","journal-title":"Network"},{"key":"ref_26","unstructured":"Rozemberczki, B., and Sarkar, R. (2020, January 1\u20133). Fast sequence-based embedding with diffusion graphs. Proceedings of the 9th International Workshop on Complex Networks, Madrid, Spain."},{"key":"ref_27","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013, January 5\u201310). Distributed representations of words and phrases and their compositionality. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_28","unstructured":"Cormen, T.H., Leiserson, C.E., Rivest, R.L., and Stein, C. (2009). Introduction to Algorithms, MIT Press."},{"key":"ref_29","unstructured":"Le, Q., and Mikolov, T. (2014, January 13\u201315). Distributed representations of sentences and documents. Proceedings of the International Conference on Machine Learning, PMLR, Barcelona, Spain."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Graves, A. (2013). Generating Sequences with Recurrent Neural Networks. arXiv.","DOI":"10.1007\/978-3-642-24797-2_3"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wang, D., Cui, P., and Zhu, W. (2016, January 13\u201317). Structural deep network embedding. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939753"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.aiopen.2021.02.001","article-title":"Network representation learning: A macro and micro view","volume":"2","author":"Liu","year":"2021","journal-title":"AI Open"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., and Mei, Q. (2015, January 18\u201322). LINE: Large-scale information network embedding. Proceedings of the 24th International Conference on World Wide Web, Florence, Italy.","DOI":"10.1145\/2736277.2741093"},{"key":"ref_34","unstructured":"Krause, B., Kahembwe, E., Murray, I., and Renals, S. (2019). Dynamic Evaluation of Transformer Language Models. arXiv."},{"key":"ref_35","unstructured":"Rae, J.W., Potapenko, A., Jayakumar, S.M., and Lillicrap, T.P. (2019). Compressive transformers for long-range sequence modelling. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Jamin, A., and Humeau-Heurtier, A. (2019). (Multiscale) cross-entropy methods: A review. Entropy, 22.","DOI":"10.3390\/e22010045"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Belkin, M., and Niyogi, P. (2002, January 9\u201314). Laplacian eigenmaps and spectral techniques for embedding and clustering. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.","DOI":"10.7551\/mitpress\/1120.003.0080"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"809","DOI":"10.1109\/TNNLS.2015.2424995","article-title":"Extreme learning machine for multilayer perceptron","volume":"27","author":"Tang","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_39","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_40","unstructured":"Rozemberczki, B., Allen, C., and Sarkar, R. (2019). Multi-Scale Attributed Node Embedding. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Lu, X., Tsao, Y., Matsuda, S., and Hori, C. (2013, January 25\u201329). Speech enhancement based on deep denoising autoencoder. Proceedings of the Interspeech, Lyon, France.","DOI":"10.21437\/Interspeech.2013-130"},{"key":"ref_42","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Hinton","year":"2008","journal-title":"J. Mach. Learn. Res."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/9\/1213\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:20:08Z","timestamp":1760142008000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/9\/1213"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,30]]},"references-count":42,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["e24091213"],"URL":"https:\/\/doi.org\/10.3390\/e24091213","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2022,8,30]]}}}