{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T17:32:20Z","timestamp":1778607140044,"version":"3.51.4"},"reference-count":32,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,3,21]],"date-time":"2022-03-21T00:00:00Z","timestamp":1647820800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61862060"],"award-info":[{"award-number":["61862060"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61462079"],"award-info":[{"award-number":["61462079"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61562086"],"award-info":[{"award-number":["61562086"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In most of the existing multi-task learning (MTL) models, multiple tasks\u2019 public information is learned by sharing parameters across hidden layers, such as hard sharing, soft sharing, and hierarchical sharing. One promising approach is to introduce model pruning into information learning, such as sparse sharing, which is regarded as being outstanding in knowledge transferring. However, the above method performs inefficiently in conflict tasks, with inadequate learning of tasks\u2019 private information, or through suffering from negative transferring. In this paper, we propose a multi-task learning model (Pruning-Based Feature Sharing, PBFS) that merges a soft parameter sharing structure with model pruning and adds a prunable shared network among different task-specific subnets. In this way, each task can select parameters in a shared subnet, according to its requirements. Experiments are conducted on three benchmark public datasets and one synthetic dataset; the impact of the different subnets\u2019 sparsity and tasks\u2019 correlations to the model performance is analyzed. Results show that the proposed model\u2019s information sharing strategy is helpful to transfer learning and superior to the several comparison models.<\/jats:p>","DOI":"10.3390\/e24030432","type":"journal-article","created":{"date-parts":[[2022,3,21]],"date-time":"2022-03-21T13:24:59Z","timestamp":1647869099000},"page":"432","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Task\u2019s Choice: Pruning-Based Feature Sharing (PBFS) for Multi-Task Learning"],"prefix":"10.3390","volume":"24","author":[{"given":"Ying","family":"Chen","sequence":"first","affiliation":[{"name":"School of Software, Xinjiang University, Urumqi 830008, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiong","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Software, Xinjiang University, Urumqi 830008, China"},{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yutong","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Software, Xinjiang University, Urumqi 830008, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiaying","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xusheng","family":"Du","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ma, X., Zhao, L., Huang, G., Wang, Z., Hu, Z., Zhu, X., and Gai, K. (2018, January 8\u201312). Entire space multi-task model: An effective approach for estimating post-click conversion rate. Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, MI, USA.","DOI":"10.1145\/3209978.3210104"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hadash, G., Shalom, O.S., and Osadchy, R. (2018, January 2). Rank and rate: Multi-task learning for recommender systems. Proceedings of the 12th ACM Conference on Recommender Systems, Vancouver, BC, Canada.","DOI":"10.1145\/3240323.3240406"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Misra, I., Shrivastava, A., Gupta, A., and Hebert, M. (2016, January 27\u201330). Cross-stitch networks for multi-task learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.433"},{"key":"ref_4","unstructured":"Negrinho, R., and Gordon, G. (2017). Deeparchitect: Automatically designing and training deep architectures. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask learning","volume":"28","author":"Caruana","year":"1997","journal-title":"Mach. Learn."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Collobert, R., and Weston, J. (2008, January 5\u20139). A unified architecture for natural language processing: Deep neural networks with multitask learning. Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland.","DOI":"10.1145\/1390156.1390177"},{"key":"ref_7","unstructured":"Subramanian, S., Trischler, A., Bengio, Y., and Pal, C.J. (2018). Learning general purpose distributed sentence representations via large scale multi-task learning. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liu, X., He, P., Chen, W., and Gao, J. (2019). Multi-task deep neural networks for natural language understanding. arXiv.","DOI":"10.18653\/v1\/P19-1441"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., and Chi, E.H. (2018, January 19\u201323). Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.","DOI":"10.1145\/3219819.3220007"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"S\u00f8gaard, A., and Goldberg, Y. (2016, January 7\u201312). Deep multi-task learning with low level tasks supervised at lower layers. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Berlin, Germany.","DOI":"10.18653\/v1\/P16-2038"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hashimoto, K., Xiong, C., Tsuruoka, Y., and Socher, R. (2016). A joint many-task model: Growing a neural network for multiple nlp tasks. arXiv.","DOI":"10.18653\/v1\/D17-1206"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Sun, T., Shao, Y., Li, X., Liu, P., Yan, H., Qiu, X., and Huang, X. (2020, January 7\u201312). Learning sparse sharing architectures for multiple tasks. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i05.6424"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Tang, H., Liu, J., Zhao, M., and Gong, X. (2020, January 22\u201326). Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations. Proceedings of the Fourteenth ACM Conference on Recommender Systems, Virtual Event, Brazil.","DOI":"10.1145\/3383313.3412236"},{"key":"ref_14","unstructured":"Xiao, X., Chen, H., Liu, Y., Yao, X., Liu, P., Fan, C., Ji, N., and Jiang, X. (2020). LT4REC: A Lottery Ticket Hypothesis Based Multi-task Practice for Video Recommendation System. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Qin, Z., Cheng, Y., Zhao, Z., Chen, Z., Metzler, D., and Qin, J. (2020, January 23\u201327). Multitask mixture of sequential experts for user activity streams. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, San Diego, CA, USA.","DOI":"10.1145\/3394486.3403359"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"827","DOI":"10.1016\/j.giq.2007.08.001","article-title":"Information asymmetry and information sharing","volume":"24","author":"Clarkson","year":"2007","journal-title":"Gov. Inf. Q."},{"key":"ref_17","unstructured":"Ruder, S., Bingel, J., Augenstein, I., and S\u00f8gaard, A. (2017). Sluice networks: Learning what to share between loosely related tasks. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1162\/neco.1991.3.1.79","article-title":"Adaptive mixtures of local experts","volume":"3","author":"Jacobs","year":"1991","journal-title":"Neural Comput."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhao, J., Du, B., Sun, L., Zhuang, F., Lv, W., and Xiong, H. (2019, January 4\u20138). Multiple relational attention network for multi-task learning. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330861"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, L.J., Fei-Fei, L., Yuille, A., Huang, J., and Murphy, K. (2018, January 8\u201314). Progressive neural architecture search. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_2"},{"key":"ref_21","unstructured":"Vanhoucke, V., Senior, A., and Mao, M.Z. (2022, March 17). Improving the Speed of Neural Networks on CPUs. In Proceedings of the Deep Learning and Unsupervised Feature NIPS Workshop. Available online: http:\/\/research.google.com\/pubs\/archive\/37631.pdf."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. (2018, January 18\u201322). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00286"},{"key":"ref_23","first-page":"598","article-title":"Optimal brain damage","volume":"2","author":"LeCun","year":"1989","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_24","unstructured":"Hassibi, B., and Stork, D.G. (1993). Second Order Derivatives for Network Pruning: Optimal Brain Surgeon, Morgan Kaufmann."},{"key":"ref_25","unstructured":"Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H.P. (2016). Pruning filters for efficient convnets. arXiv."},{"key":"ref_26","unstructured":"Liu, H., Simonyan, K., Vinyals, O., Fernando, C., and Kavukcuoglu, K. (2017). Hierarchical representations for efficient architecture search. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Elsken, T., Metzen, J.H., and Hutter, F. (2018). Efficient multi-objective neural architecture search via lamarckian evolution. arXiv.","DOI":"10.1007\/978-3-030-05318-5_3"},{"key":"ref_28","unstructured":"Ma, J., Zhao, Z., Chen, J., Li, A., Hong, L., and Chi, E.H. (February, January 27). Snr: Sub-network routing for flexible parameter sharing in multi-task learning. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_29","unstructured":"Frankle, J., and Carbin, M. (2018). The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., and Zhang, C. (2017, January 22\u201329). Learning efficient convolutional networks through network slimming. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.298"},{"key":"ref_31","unstructured":"Asuncion, A., and Newman, D. (2022, March 17). UCI Machine Learning Repository. Available online: http:\/\/archive.ics.uci.edu\/ml."},{"key":"ref_32","unstructured":"Cortez, P., and Silva, A.M.G. (2022, March 17). Using Data Mining to Predict Secondary School Student Performance. Available online: http:\/\/archive.ics.uci.edu\/ml\/datasets\/Student+Performance."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/3\/432\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:40:23Z","timestamp":1760136023000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/3\/432"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,21]]},"references-count":32,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["e24030432"],"URL":"https:\/\/doi.org\/10.3390\/e24030432","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,21]]}}}