{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T08:46:10Z","timestamp":1769071570779,"version":"3.49.0"},"reference-count":32,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,6,15]],"date-time":"2023-06-15T00:00:00Z","timestamp":1686787200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:p>Embedding is widely used in recommendation models to learn feature representations. However, the traditional embedding technique that assigns a fixed size to all categorical features may be suboptimal due to the following reasons. In recommendation domain, the majority of categorical features' embeddings can be trained with less capacity without impacting model performance, thereby storing embeddings with equal length may incur unnecessary memory usage. Existing work that tries to allocate customized sizes for each feature usually either simply scales the embedding size with feature's popularity or formulates this size allocation problem as an architecture selection problem. Unfortunately, most of these methods either have large performance drop or incur significant extra time cost for searching proper embedding sizes. In this article, instead of formulating the size allocation problem as an architecture selection problem, we approach the problem from a pruning perspective and propose <jats:bold>P<\/jats:bold>runing-based <jats:bold>M<\/jats:bold>ulti-size <jats:bold>E<\/jats:bold>mbedding (PME) framework. During the search phase, we prune the dimensions that have the least impact on model performance in the embedding to reduce its capacity. Then, we show that the customized size of each token can be obtained by transferring the capacity of its pruned embedding with significant less search cost. Experimental results validate that PME can efficiently find proper sizes and hence achieve strong performance while significantly reducing the number of parameters in the embedding layer.<\/jats:p>","DOI":"10.3389\/fdata.2023.1195742","type":"journal-article","created":{"date-parts":[[2023,6,15]],"date-time":"2023-06-15T05:48:29Z","timestamp":1686808109000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["PME: pruning-based multi-size embedding for recommender systems"],"prefix":"10.3389","volume":"6","author":[{"given":"Zirui","family":"Liu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qingquan","family":"Song","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Soo-Hyun","family":"Choi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rui","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xia","family":"Hu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,6,15]]},"reference":[{"key":"B1","author":"Buda","year":"2010","journal-title":"Life Time of Correlations and its Applications"},{"key":"B2","first-page":"5609","article-title":"\u201cHashnet: Deep learning to hash by continuation,\u201d","volume-title":"IEEE International Conference on Computer Vision, ICCV 2017","author":"Cao","year":"2017"},{"key":"B3","first-page":"8532","article-title":"\u201c\u201cLearning-compression\u201d algorithms for neural net pruning,\u201d","author":"Carreira-Perpin\u00e1n","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B4","first-page":"1617","article-title":"\u201cDifferentiable product quantization for end-to-end embedding compression,\u201d","volume-title":"Proceedings of the 37th International Conference on Machine Learning, ICML 2020, Virtual Event (PMLR), vol. 119 of Proceedings of Machine Learning Research","author":"Chen","year":"2020"},{"key":"B5","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1145\/2988450.2988454","article-title":"\u201cWide & deep learning for recommender systems,\u201d","author":"Cheng","year":"2016","journal-title":"Proceedings of the 1st Workshop on Deep Learning for Recommender Systems"},{"key":"B6","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1145\/2959100.2959190","article-title":"\u201cDeep neural networks for youtube recommendations,\u201d","volume-title":"Proceedings of the 10th ACM Conference on Recommender Systems","author":"Covington","year":"2016"},{"key":"B7","article-title":"\u201cRandom offset block embedding (ROBE) for compressed embedding tables in deep learning recommendation systems,\u201d","volume-title":"Proceedings of Machine Learning and Systems 2022, MLSys 2022","author":"Desai","year":"2022"},{"key":"B8","article-title":"\u201cThe lottery ticket hypothesis: Finding sparse, trainable neural networks,\u201d","volume-title":"7th International Conference on Learning Representations, ICLR 2019","author":"Frankle","year":"2019"},{"key":"B9","first-page":"2786","article-title":"\u201cMixed dimension embeddings with application to memory-efficient recommendation systems,\u201d","volume-title":"IEEE International Symposium on Information Theory, ISIT 2021","author":"Ginart","year":"2021"},{"key":"B10","first-page":"1725","article-title":"\u201cDeepfm: A factorization-machine based neural network for CTR prediction,\u201d","volume-title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017","author":"Guo","year":"2017"},{"key":"B11","article-title":"\u201cDeep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,\u201d","volume-title":"4th International Conference on Learning Representations, ICLR 2016","author":"Han","year":"2016"},{"key":"B12","doi-asserted-by":"crossref","first-page":"2387","DOI":"10.1145\/3394486.3403288","article-title":"\u201cNeural input search for large scale recommendation models,\u201d","volume-title":"KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event","author":"Joglekar","year":"2020"},{"key":"B13","doi-asserted-by":"crossref","first-page":"562","DOI":"10.1145\/3366424.3383416","article-title":"\u201cLearning multi-granular quantized embeddings for large-vocab categorical features in recommender systems,\u201d","volume-title":"Companion of The 2020 Web Conference 2020","author":"Kang","year":"2020"},{"key":"B14","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1109\/MC.2009.263","article-title":"Matrix factorization techniques for recommender systems","volume":"42","author":"Koren","year":"2009","journal-title":"Computer"},{"key":"B15","first-page":"5544","article-title":"\u201cSoft threshold weight reparameterization for learnable sparsity,\u201d","author":"Kusupati","year":"2020","journal-title":"Proceedings of the 37th International Conference on Machine Learning, ICML 2020, Virtual Event (PMLR), vol. 119 of Proceedings of Machine Learning Research"},{"key":"B16","article-title":"\u201cSnip: single-shot network pruning based on connection sensitivity,\u201d","volume-title":"7th International Conference on Learning Representations, ICLR 2019","author":"Lee","year":"2019"},{"key":"B17","doi-asserted-by":"crossref","first-page":"1754","DOI":"10.1145\/3219819.3220023","article-title":"\u201cxdeepfm: Combining explicit and implicit feature interactions for recommender systems,\u201d","volume-title":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018","author":"Lian","year":"2018"},{"key":"B18","first-page":"27","article-title":"\u201cDeep learning of binary hash codes for fast image retrieval,\u201d","volume-title":"2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2015","author":"Lin","year":"2015"},{"key":"B19","article-title":"\u201cDARTS: differentiable architecture search,\u201d","volume-title":"7th International Conference on Learning Representations, ICLR 2019","author":"Liu","year":"2019"},{"key":"B20","article-title":"\u201cLearnable embedding sizes for recommender systems,\u201d","volume-title":"9th International Conference on Learning Representations, ICLR 2021, Virtual Event","author":"Liu","year":"2021"},{"key":"B21","article-title":"\u201cLow rank approximation - algorithms, implementation, applications,\u201d","volume-title":"Communications and Control Engineering","author":"Markovsky","year":"2012"},{"key":"B22","article-title":"Deep learning inference in facebook data centers: Characterization, performance optimizations and hardware implications","author":"Park","year":"2018","journal-title":"arXiv [Preprint]"},{"key":"B23","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1145\/1454008.1454012","article-title":"\u201cThe long tail of recommender systems and how to leverage it,\u201d","volume-title":"Proceedings of the 2008 ACM Conference on Recommender Systems, RecSys 2008","author":"Park","year":"2008"},{"key":"B24","first-page":"8024","article-title":"\u201cPytorch: An imperative style, high-performance deep learning library,\u201d","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019","author":"Paszke","year":"2019"},{"key":"B25","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1145\/3394486.3403059","article-title":"\u201cCompositional embeddings using complementary partitions for memory-efficient recommendation systems,\u201d","volume-title":"KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event","author":"Shi","year":"2020"},{"key":"B26","first-page":"1161","article-title":"\u201cAutoint: Automatic feature interaction learning via self-attentive neural networks,\u201d","volume-title":"Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019","author":"Song","year":"2019"},{"key":"B27","article-title":"\u201cPicking winning tickets before training by preserving gradient flow,\u201d","volume-title":"8th International Conference on Learning Representations, ICLR 2020","author":"Wang","year":"2020"},{"key":"B28","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1109\/TPAMI.2017.2699960","article-title":"A survey on learning to Hash","volume":"40","author":"Wang","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"B29","article-title":"\u201cTt-rec: Tensor train compression for deep learning recommendation models,\u201d","volume-title":"Proceedings of Machine Learning and Systems 2021, MLSys 2021, virtual","author":"Yin","year":"2021"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3158369","article-title":"Deep learning based recommender system: a survey and new perspectives","volume":"52","author":"Zhang","year":"2019","journal-title":"ACM Comput. Surv."},{"key":"B31","article-title":"Memory-efficient embedding for recommendations","author":"Zhao","year":"","journal-title":"arXiv [Preprint]"},{"key":"B32","article-title":"AutoEMB: automated embedding dimensionality search in streaming recommendations","author":"Zhao","year":"","journal-title":"arXiv preprint arXiv:2002.11252"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2023.1195742\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,15]],"date-time":"2023-06-15T05:48:52Z","timestamp":1686808132000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2023.1195742\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,15]]},"references-count":32,"alternative-id":["10.3389\/fdata.2023.1195742"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2023.1195742","relation":{},"ISSN":["2624-909X"],"issn-type":[{"value":"2624-909X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,15]]},"article-number":"1195742"}}