{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T05:02:43Z","timestamp":1750309363470,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":43,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,8,24]],"date-time":"2024-08-24T00:00:00Z","timestamp":1724457600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key R&D Program of China under Grant","award":["No.2023YFF0905400"],"award-info":[{"award-number":["No.2023YFF0905400"]}]},{"name":"National Natural Science Foundation of China through grants","award":["No.U2341229, No.62076138"],"award-info":[{"award-number":["No.U2341229, No.62076138"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,8,25]]},"DOI":"10.1145\/3637528.3671986","type":"proceedings-article","created":{"date-parts":[[2024,8,25]],"date-time":"2024-08-25T04:55:12Z","timestamp":1724561712000},"page":"4179-4190","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Subspace Selection based Prompt Tuning with Nonconvex Nonsmooth Black-Box Optimization"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-3833-8555","authenticated-orcid":false,"given":"Haozhen","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Jilin University, Changchun, Jilin, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1188-6452","authenticated-orcid":false,"given":"Hualin","family":"Zhang","sequence":"additional","affiliation":[{"name":"Mohamed bin Zayed University of Artificial Intelligence, Masdar, United Arab Emirates"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7165-3143","authenticated-orcid":false,"given":"Bin","family":"Gu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Jilin University & Mohamed bin Zayed University of Artificial Intelligence, Changchun, Jilin, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2697-8093","authenticated-orcid":false,"given":"Yi","family":"Chang","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Jilin University & Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Ministry of Education, Changchun, Jilin, China"}]}],"member":"320","published-online":{"date-parts":[[2024,8,24]]},"reference":[{"key":"e_1_3_2_2_1_1","unstructured":"Alekh Agarwal Ofer Dekel and Lin Xiao. 2010. Optimal Algorithms for Online Convex Optimization with Multi-Point Bandit Feedback.. In Colt. Citeseer 28--40."},{"volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Aghajanyan Armen","key":"e_1_3_2_2_2_1","unstructured":"Armen Aghajanyan, Sonal Gupta, and Luke Zettlemoyer. 2021. Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, 7319--7328."},{"key":"e_1_3_2_2_3_1","volume-title":"Gradients without backpropagation. arXiv preprint arXiv:2202.08587","author":"Baydin Atilim G\u00fcnecs","year":"2022","unstructured":"Atilim G\u00fcnecs Baydin, Barak A Pearlmutter, Don Syme, Frank Wood, and Philip Torr. 2022. Gradients without backpropagation. arXiv preprint arXiv:2202.08587 (2022)."},{"key":"e_1_3_2_2_4_1","volume-title":"A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326","author":"Bowman Samuel R","year":"2015","unstructured":"Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326 (2015)."},{"key":"e_1_3_2_2_5_1","unstructured":"Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. Advances in neural information processing systems Vol. 33 (2020) 1877--1901."},{"key":"e_1_3_2_2_6_1","volume-title":"DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training. arXiv preprint arXiv:2310.02025","author":"Chen Aochuan","year":"2023","unstructured":"Aochuan Chen, Yimeng Zhang, Jinghan Jia, James Diffenderfer, Jiancheng Liu, Konstantinos Parasyris, Yihua Zhang, Zheng Zhang, Bhavya Kailkhura, and Sijia Liu. 2023. DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training. arXiv preprint arXiv:2310.02025 (2023)."},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3128572.3140448"},{"key":"e_1_3_2_2_8_1","first-page":"1","article-title":"Palm: Scaling language modeling with pathways","volume":"24","author":"Chowdhery Aakanksha","year":"2023","unstructured":"Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, Vol. 24, 240 (2023), 1--113.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_2_9_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Association for Computational Linguistics, Minneapolis, Minnesota, 4171--4186."},{"key":"e_1_3_2_2_10_1","volume-title":"Black-box prompt learning for pre-trained language models. arXiv preprint arXiv:2201.08531","author":"Diao Shizhe","year":"2022","unstructured":"Shizhe Diao, Zhichao Huang, Ruijia Xu, Xuechun Li, Yong Lin, Xiao Zhou, and Tong Zhang. 2022. Black-box prompt learning for pre-trained language models. arXiv preprint arXiv:2201.08531 (2022)."},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2015.2409256"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/3586589.3586709"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1137\/120880811"},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-014-0846-1"},{"key":"e_1_3_2_2_15_1","volume-title":"Completely derandomized self-adaptation in evolution strategies. Evolutionary computation","author":"Hansen Nikolaus","year":"2001","unstructured":"Nikolaus Hansen and Andreas Ostermeier. 2001. Completely derandomized self-adaptation in evolution strategies. Evolutionary computation, Vol. 9, 2 (2001), 159--195."},{"key":"e_1_3_2_2_16_1","volume-title":"The forward-forward algorithm: Some preliminary investigations. arXiv preprint arXiv:2212.13345","author":"Hinton Geoffrey","year":"2022","unstructured":"Geoffrey Hinton. 2022. The forward-forward algorithm: Some preliminary investigations. arXiv preprint arXiv:2212.13345 (2022)."},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33011503"},{"key":"e_1_3_2_2_18_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_3_2_2_19_1","volume-title":"The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691","author":"Lester Brian","year":"2021","unstructured":"Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691 (2021)."},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.clinicalnlp-1.17"},{"key":"e_1_3_2_2_21_1","volume-title":"Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190","author":"Li Xiang Lisa","year":"2021","unstructured":"Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 (2021)."},{"key":"e_1_3_2_2_22_1","volume-title":"GPT understands, too. AI Open","author":"Liu Xiao","year":"2023","unstructured":"Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2023. GPT understands, too. AI Open (2023)."},{"key":"e_1_3_2_2_23_1","volume-title":"Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)."},{"key":"e_1_3_2_2_24_1","volume-title":"Fine-Tuning Language Models with Just Forward Passes. arXiv preprint arXiv:2305.17333","author":"Malladi Sadhika","year":"2023","unstructured":"Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D Lee, Danqi Chen, and Sanjeev Arora. 2023. Fine-Tuning Language Models with Just Forward Passes. arXiv preprint arXiv:2305.17333 (2023)."},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10208-015-9296-2"},{"key":"e_1_3_2_2_26_1","volume-title":"To tune or not to tune? adapting pretrained representations to diverse tasks. arXiv preprint arXiv:1903.05987","author":"Peters Matthew E","year":"2019","unstructured":"Matthew E Peters, Sebastian Ruder, and Noah A Smith. 2019. To tune or not to tune? adapting pretrained representations to diverse tasks. arXiv preprint arXiv:1903.05987 (2019)."},{"key":"e_1_3_2_2_27_1","unstructured":"Yujia Qin Xiaozhi Wang Yusheng Su Yankai Lin Ning Ding Zhiyuan Liu Juanzi Li Lei Hou Peng Li Maosong Sun et al. 2021. Exploring lowdimensional intrinsic task subspace via prompt tuning. arXiv preprint arXiv:2110.07867 (2021)."},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/3455716.3455856"},{"key":"e_1_3_2_2_29_1","volume-title":"Scaling Forward Gradient With Local Losses. In The Eleventh International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=JxpBP1JM15-","author":"Ren Mengye","year":"2023","unstructured":"Mengye Ren, Simon Kornblith, Renjie Liao, and Geoffrey Hinton. 2023. Scaling Forward Gradient With Local Losses. In The Eleventh International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=JxpBP1JM15-"},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02431-3"},{"key":"e_1_3_2_2_31_1","volume-title":"Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864","author":"Salimans Tim","year":"2017","unstructured":"Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. 2017. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864 (2017)."},{"key":"e_1_3_2_2_32_1","volume-title":"Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642","author":"Socher Richard","year":"2013","unstructured":"Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing. 1631--1642."},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/9.119632"},{"key":"e_1_3_2_2_34_1","volume-title":"FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models. arXiv preprint arXiv:2310.01467","author":"Sun Jingwei","year":"2023","unstructured":"Jingwei Sun, Ziyue Xu, Hongxu Yin, Dong Yang, Daguang Xu, Yiran Chen, and Holger R Roth. 2023. FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models. arXiv preprint arXiv:2310.01467 (2023)."},{"volume-title":"BBTv2: Towards a Gradient-Free Future with Large Language Models","author":"Sun Tianxiang","key":"e_1_3_2_2_35_1","unstructured":"Tianxiang Sun, Zhengfu He, Hong Qian, Yunhua Zhou, Xuanjing Huang, and Xipeng Qiu. 2022. BBTv2: Towards a Gradient-Free Future with Large Language Models. Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 3916--3930."},{"key":"e_1_3_2_2_36_1","volume-title":"International Conference on Machine Learning. PMLR","author":"Sun Tianxiang","year":"2022","unstructured":"Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. 2022. Black-box tuning for language-model-as-a-service. In International Conference on Machine Learning. PMLR, 20841--20855."},{"key":"e_1_3_2_2_37_1","volume-title":"GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461","author":"Wang Alex","year":"2018","unstructured":"Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461 (2018)."},{"key":"e_1_3_2_2_38_1","volume-title":"Advances in Neural Information Processing Systems","volume":"32","author":"Xu Yi","year":"2019","unstructured":"Yi Xu, Rong Jin, and Tianbao Yang. 2019. Non-asymptotic analysis of stochastic methods for non-smooth non-convex regularized problems. Advances in Neural Information Processing Systems, Vol. 32 (2019)."},{"key":"e_1_3_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2023\/187"},{"key":"e_1_3_2_2_40_1","volume-title":"Diverse few-shot text classification with multiple metrics. arXiv preprint arXiv:1805.07513","author":"Yu Mo","year":"2018","unstructured":"Mo Yu, Xiaoxiao Guo, Jinfeng Yi, Shiyu Chang, Saloni Potdar, Yu Cheng, Gerald Tesauro, Haoyu Wang, and Bowen Zhou. 2018. Diverse few-shot text classification with multiple metrics. arXiv preprint arXiv:1805.07513 (2018)."},{"key":"e_1_3_2_2_41_1","volume-title":"Pangu-\u03b1: Large-scale autoregressive pretrained Chinese language models with auto-parallel computation. arXiv preprint arXiv:2104.12369","author":"Zeng Wei","year":"2021","unstructured":"Wei Zeng, Xiaozhe Ren, Teng Su, Hui Wang, Yi Liao, Zhiwei Wang, Xin Jiang, ZhenZhang Yang, Kaisheng Wang, Xiaoda Zhang, et al. 2021. Pangu-\u03b1: Large-scale autoregressive pretrained Chinese language models with auto-parallel computation. arXiv preprint arXiv:2104.12369 (2021)."},{"key":"e_1_3_2_2_42_1","volume-title":"Character-level convolutional networks for text classification. Advances in neural information processing systems","author":"Zhang Xiang","year":"2015","unstructured":"Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in neural information processing systems, Vol. 28 (2015)."},{"key":"e_1_3_2_2_43_1","volume-title":"Black-box Prompt Tuning with Subspace Learning. arXiv preprint arXiv:2305.03518","author":"Zheng Yuanhang","year":"2023","unstructured":"Yuanhang Zheng, Zhixing Tan, Peng Li, and Yang Liu. 2023. Black-box Prompt Tuning with Subspace Learning. arXiv preprint arXiv:2305.03518 (2023)."}],"event":{"name":"KDD '24: The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"],"location":"Barcelona Spain","acronym":"KDD '24"},"container-title":["Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3637528.3671986","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3637528.3671986","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:06:05Z","timestamp":1750291565000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3637528.3671986"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,24]]},"references-count":43,"alternative-id":["10.1145\/3637528.3671986","10.1145\/3637528"],"URL":"https:\/\/doi.org\/10.1145\/3637528.3671986","relation":{},"subject":[],"published":{"date-parts":[[2024,8,24]]},"assertion":[{"value":"2024-08-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}