{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T02:35:05Z","timestamp":1750818905129,"version":"3.37.3"},"reference-count":26,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2024,10,12]],"date-time":"2024-10-12T00:00:00Z","timestamp":1728691200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2023YFB3001501"],"award-info":[{"award-number":["2023YFB3001501"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62302133"],"award-info":[{"award-number":["62302133"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Yangtze River Delta Project","award":["2023ZY1068"],"award-info":[{"award-number":["2023ZY1068"]}]},{"DOI":"10.13039\/100022963","name":"Key Research and Development Program of Zhejiang Province","doi-asserted-by":"publisher","award":["2024C01104","2024C01026"],"award-info":[{"award-number":["2024C01104","2024C01026"]}],"id":[{"id":"10.13039\/100022963","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004731","name":"Natural Science Foundation of Zhejiang Province","doi-asserted-by":"publisher","award":["LQ23F020015"],"award-info":[{"award-number":["LQ23F020015"]}],"id":[{"id":"10.13039\/501100004731","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,2,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The exponential growth in data and parameters in modern neural networks has created the need to distribute these models across multiple devices for efficient training, resulting in the device placement problem. Existing graph encoding approaches for device placement suffer from low efficiency when searching for optimal parallel strategies, primarily due to suboptimal positional information retrieval. To address these challenges, we propose the Laplacian Principal Component Analysis-graph attention networks (LPCA-GAT) model. Firstly, we employ GAT to capture complex relationships between nodes and generate node encodings. Secondly, we leverage LPCA on the graph Laplacian matrix to extract crucial low-dimensional positional information. Finally, by integrating these two components, we obtain the final node encodings. This enhances the representation capability of nodes within the graph, enabling efficient device placement. The experimental results demonstrate that LPCA-GAT achieves superior device placement results, specifically accelerating execution and computation time by 13.24% and 96.38%, respectively, leading to significant improvements in both operational efficiency and performance.<\/jats:p>","DOI":"10.1093\/comjnl\/bxae102","type":"journal-article","created":{"date-parts":[[2024,10,13]],"date-time":"2024-10-13T06:51:22Z","timestamp":1728802282000},"page":"175-186","source":"Crossref","is-referenced-by-count":1,"title":["Device placement using Laplacian PCA and graph attention networks"],"prefix":"10.1093","volume":"68","author":[{"given":"Meng","family":"Han","sequence":"first","affiliation":[{"name":"School of Computer Science, Hangzhou Dianzi University , Hangzhou Economic Development Zone, No. 1158, No. 2 Street, Baiyang Street, Hangzhou, Zhejiang 310018 ,","place":["China"]}]},{"given":"Yan","family":"Zeng","sequence":"additional","affiliation":[{"name":"School of Computer Science, Hangzhou Dianzi University , Hangzhou Economic Development Zone, No. 1158, No. 2 Street, Baiyang Street, Hangzhou, Zhejiang 310018 ,","place":["China"]}]},{"given":"Hao","family":"Shu","sequence":"additional","affiliation":[{"name":"School of Software, Shanxi Agricultural University , No.1, Mingxian South Road, Jinzhong, Shanxi 030801 ,","place":["China"]}]},{"given":"Lupeng","family":"Yue","sequence":"additional","affiliation":[{"name":"School of Computer Science, Hangzhou Dianzi University , Hangzhou Economic Development Zone, No. 1158, No. 2 Street, Baiyang Street, Hangzhou, Zhejiang 310018 ,","place":["China"]}]},{"given":"Jilin","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Hangzhou Dianzi University , Hangzhou Economic Development Zone, No. 1158, No. 2 Street, Baiyang Street, Hangzhou, Zhejiang 310018 ,","place":["China"]}]},{"given":"Jian","family":"Wan","sequence":"additional","affiliation":[{"name":"School of Computer Science, Hangzhou Dianzi University , Hangzhou Economic Development Zone, No. 1158, No. 2 Street, Baiyang Street, Hangzhou, Zhejiang 310018 ,","place":["China"]}]},{"given":"Yongjian","family":"Ren","sequence":"additional","affiliation":[{"name":"School of Computer Science, Hangzhou Dianzi University , Hangzhou Economic Development Zone, No. 1158, No. 2 Street, Baiyang Street, Hangzhou, Zhejiang 310018 ,","place":["China"]}]}],"member":"286","published-online":{"date-parts":[[2024,10,12]]},"reference":[{"key":"2025021705264203700_ref1","doi-asserted-by":"publisher","first-page":"8638","DOI":"10.1609\/aaai.v36i8.20842","article-title":"Prune and tune ensembles: low-cost ensemble learning with sparse independent subnetworks","volume-title":"Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI)","author":"Whitaker","year":"2022"},{"key":"2025021705264203700_ref2","doi-asserted-by":"crossref","first-page":"4821","DOI":"10.18653\/v1\/2022.acl-long.331","article-title":"Compression of generative pre-trained language models via quantization","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL)","author":"Tao","year":"2022"},{"key":"2025021705264203700_ref3","first-page":"4602","article-title":"Symbolic knowledge distillation: from general language models to commonsense models","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL)","author":"West","year":"2022"},{"key":"2025021705264203700_ref4","first-page":"5055","article-title":"LAG: Lazily Aggregated Gradient for communication-efficient distributed learning","volume-title":"Proceedings of the 32nd International Conference on Neural Information Processing Systems (NeurIPS)","author":"Chen","year":"2018"},{"key":"2025021705264203700_ref5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1909.08053","article-title":"Megatron-LM: training multi-billion parameter language models using model parallelism","author":"Shoeybi","year":"2019"},{"key":"2025021705264203700_ref6","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1802.05799","article-title":"Horovod: fast and easy distributed deep learning in TensorFlow","author":"Sergeev","year":"2018"},{"key":"2025021705264203700_ref7","first-page":"1676","article-title":"Spotlight: optimizing device placement for training deep neural networks","volume-title":"Proceedings of the 35th International Conference on Machine Learning (ICML)","author":"Gao","year":"2018"},{"key":"2025021705264203700_ref8","article-title":"Reinforced genetic algorithm learning for optimizing computation graphs","volume-title":"Proceedings of the 8th International Conference International Conference on Learning Representations (ICLR)","author":"Paliwal","year":"2020"},{"key":"2025021705264203700_ref9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1910.01578","article-title":"GDP: generalized device placement for dataflow graphs","author":"Zhou","year":"2019"},{"key":"2025021705264203700_ref10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3472456.3472523","article-title":"Accelerated device placement optimization with contrastive learning","volume-title":"Proceedings of the 50th International Conference on Parallel Processing (ICPP)","author":"Lan","year":"2021"},{"key":"2025021705264203700_ref11","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1016\/j.future.2023.05.032","article-title":"Heterps: distributed deep learning with reinforcement learning based scheduling in heterogeneous environments","volume":"148","author":"Liu","year":"2023","journal-title":"Future Gener Comput Syst"},{"key":"2025021705264203700_ref12","first-page":"3981","article-title":"Placeto: learning generalizable device placement algorithms for distributed machine learning","volume-title":"Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS)","author":"Addanki","year":"2019"},{"key":"2025021705264203700_ref13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3429882.3430104","article-title":"Graph representation matters in device placement","volume-title":"Proceedings of the Workshop on Distributed Infrastructures for Deep Learning (DIDL)","author":"Mitropolitsky","year":"2020"},{"key":"2025021705264203700_ref14","doi-asserted-by":"publisher","first-page":"2484","DOI":"10.1093\/comjnl\/bxab085","article-title":"Joining formal concept analysis to feature extraction for data pruning in cloud of things","volume":"65","author":"Moulahi","year":"2021","journal-title":"Comput J"},{"key":"2025021705264203700_ref15","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/bxae060","article-title":"Global-local graph attention: Unifying global and local attention for node classification","volume":"bxae060","author":"Lin","year":"2024","journal-title":"Comput J"},{"key":"2025021705264203700_ref16","first-page":"2430","article-title":"Device placement optimization with reinforcement learning","volume-title":"Proceedings of the 34th International Conference on Machine Learning (ICML)","author":"Mirhoseini","year":"2017"},{"key":"2025021705264203700_ref17","doi-asserted-by":"publisher","first-page":"178","DOI":"10.1108\/aa.2001.21.2.178.2","article-title":"Learning with recurrent neural networks","volume":"21","author":"Hammer","year":"2001","journal-title":"Assembly Autom"},{"key":"2025021705264203700_ref18","article-title":"A hierarchical model for device placement","volume-title":"Proceedings of the 6th International Conference on Learning Representations (ICLR)","author":"Mirhoseini","year":"2018"},{"key":"2025021705264203700_ref19","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"2025021705264203700_ref20","first-page":"1299","article-title":"Aware: Adaptive distributed training with computation, communication and position awareness for deep learning model","volume-title":"2022 IEEE 24th International Conference on High Performance Computing & Communications; 8th International Conference on Data Science & Systems; 20th International Conference on Smart City; 8th International Conference on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC\/DSS\/SmartCity\/DependSys)","author":"Zeng","year":"2022"},{"key":"2025021705264203700_ref21","first-page":"770","article-title":"Deep residual learning for image recognition","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"He","year":"2016"},{"key":"2025021705264203700_ref22","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1609.08144","article-title":"Google\u2019s neural machine translation system: bridging the gap between human and machine translation","author":"Wu","year":"2016"},{"key":"2025021705264203700_ref23","article-title":"Very deep convolutional networks for large-scale image recognition","volume-title":"Proceedings of the 3rd International Conference on Learning Representations (ICLR)","author":"Simonyan","year":"2015"},{"key":"2025021705264203700_ref24","first-page":"2818","article-title":"Rethinking the inception architecture for computer vision","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Szegedy","year":"2016"},{"key":"2025021705264203700_ref25","first-page":"8697","article-title":"Learning transferable architectures for scalable image recognition","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Zoph","year":"2018"},{"key":"2025021705264203700_ref26","first-page":"19","article-title":"Progressive neural architecture search","volume-title":"Proceedings of the 15th European Conference on Computer Vision (ECCV)","author":"Liu","year":"2018"}],"container-title":["The Computer Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/68\/2\/175\/59730916\/bxae102.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/68\/2\/175\/59730916\/bxae102.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,17]],"date-time":"2025-02-17T05:27:00Z","timestamp":1739770020000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/comjnl\/article\/68\/2\/175\/7819324"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,12]]},"references-count":26,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,10,12]]},"published-print":{"date-parts":[[2025,2,9]]}},"URL":"https:\/\/doi.org\/10.1093\/comjnl\/bxae102","relation":{},"ISSN":["0010-4620","1460-2067"],"issn-type":[{"type":"print","value":"0010-4620"},{"type":"electronic","value":"1460-2067"}],"subject":[],"published-other":{"date-parts":[[2025,2]]},"published":{"date-parts":[[2024,10,12]]}}}