{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T20:59:30Z","timestamp":1782507570143,"version":"3.54.5"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:p>Machine learning (ML) has become a cornerstone of key applications at ByteDance. As model complexity and data volumes surge, data management for large-scale ML workloads faces substantial challenges, particularly with recent advances in large recommendation models (LRMs) and large multimodal models (LMMs). Traditional approaches exhibit limitations in storage efficiency, metadata scalability, update mechanisms, and integration with ML frameworks. To address these challenges, we propose Magnus, a holistic data management system built upon Apache Iceberg. Magnus integrates innovative optimizations across resource-efficient storage formats optimized for large wide tables and multimodal data, built-in support for vector and inverted indexes to accelerate data retrieval, scalable metadata planning with Git-like branching and tagging capabilities, and high-performance update\/upsert based on lightweight merge-on-read (MOR) strategies. Additionally, Magnus provides native support and specialized enhancement for LRM and LMM training workloads. Experimental results demonstrate significant performance gains in real-world ML scenarios. Magnus has been deployed at ByteDance for over five years, enabling robust and efficient data infrastructure for large-scale ML workloads.<\/jats:p>","DOI":"10.14778\/3750601.3750620","type":"journal-article","created":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T13:38:05Z","timestamp":1758029885000},"page":"4964-4977","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Magnus: A Holistic Approach to Data Management for Large-Scale Machine Learning Workloads"],"prefix":"10.14778","volume":"18","author":[{"given":"Jun","family":"Song","sequence":"first","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jingyi","family":"Ding","sequence":"additional","affiliation":[{"name":"Zhejiang University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Irshad","family":"Kandy","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanghao","family":"Lin","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhongjia","family":"Wei","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zilong","family":"Zhou","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhiwei","family":"Peng","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jixi","family":"Shan","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongyue","family":"Mao","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiuqi","family":"Huang","sequence":"additional","affiliation":[{"name":"Zhejiang University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xun","family":"Song","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cheng","family":"Chen","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanjia","family":"Li","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tianhao","family":"Yang","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Jia","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaohong","family":"Dong","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kang","family":"Lei","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rui","family":"Shi","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pengwei","family":"Zhao","sequence":"additional","affiliation":[{"name":"ByteDance Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Chen","sequence":"additional","affiliation":[{"name":"Zhejiang University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,9,16]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Amazon. 2025. Amazon S3. https:\/\/aws.amazon.com\/s3\/ An object storage service offering industry-leading scalability data availability security and performance."},{"key":"e_1_2_1_2_1","unstructured":"AnyScale. 2025. Ray. https:\/\/www.ray.io\/ An AI compute engine."},{"key":"e_1_2_1_3_1","unstructured":"Apache Software Foundation. 2025. Apache Flink. https:\/\/flink.apache.org\/ A framework and distributed processing engine for stateful computations over unbounded and bounded data streams."},{"key":"e_1_2_1_4_1","unstructured":"Apache Software Foundation. 2025. Apache Hadoop. https:\/\/hadoop.apache.org\/ A framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models."},{"key":"e_1_2_1_5_1","unstructured":"Apache Software Foundation. 2025. Apache Hudi. https:\/\/hudi.apache.org\/ An open source data lake platform."},{"key":"e_1_2_1_6_1","unstructured":"Apache Software Foundation. 2025. Apache Iceberg. https:\/\/iceberg.apache.org\/ The open table format for analytic datasets."},{"key":"e_1_2_1_7_1","unstructured":"Apache Software Foundation. 2025. Apache ORC. http:\/\/orc.apache.org The smallest fastest columnar storage for Hadoop workloads."},{"key":"e_1_2_1_8_1","unstructured":"Apache Software Foundation. 2025. Apache Parquet. https:\/\/parquet.apache.org\/ An open source column-oriented data file format designed for efficient data storage and retrieval."},{"key":"e_1_2_1_9_1","unstructured":"Apache Software Foundation. 2025. Apache Spark. https:\/\/spark.apache.org A multi-language engine for executing data engineering data science and machine learning on single-node machines or clusters."},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","first-page":"3411","DOI":"10.14778\/3415478.3415560","article-title":"Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores","volume":"13","author":"Armbrust Michael","year":"2020","unstructured":"Michael Armbrust, Tathagata Das, Liwen Sun, Burak Yavuz, Shixiong Zhu, Mukul Murthy, Joseph Torres, Herman van Hovell, Adrian Ionescu, Alicja \u0141uszczak, et al. 2020. Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores. Proceedings of the VLDB Endowment (VLDB) 13, 12 (2020), 3411\u20133424. http:\/\/www.vldb.org\/pvldb\/vol13\/p3411-armbrust.pdf","journal-title":"Proceedings of the VLDB Endowment (VLDB)"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526054"},{"key":"e_1_2_1_12_1","volume-title":"Conference on Innovative Data Systems Research (CIDR). www.cidrdb.org. https:\/\/www.cidrdb.org\/cidr2023\/papers\/p77-chattopadhyay.pdf","author":"Chattopadhyay Biswapesh","year":"2023","unstructured":"Biswapesh Chattopadhyay, Pedro Pedreira, Sameer Agarwal, Yutian Sun, Suketu Vakharia, Peng Li, Weiran Liu, and Sundaram Narayanan. 2023. Shared Foundations: Modernizing Meta's Data Lakehouse. In Conference on Innovative Data Systems Research (CIDR). www.cidrdb.org. https:\/\/www.cidrdb.org\/cidr2023\/papers\/p77-chattopadhyay.pdf"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.12740"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.14778\/3611540.3611545"},{"key":"e_1_2_1_15_1","unstructured":"Databricks. 2025. Databricks Feature Store. https:\/\/www.databricks.com\/product\/feature-store The first feature store co-designed with a data platform and MLOps framework."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626246.3653389"},{"key":"e_1_2_1_17_1","unstructured":"Chaorui Deng Deyao Zhu Kunchang Li Chenhui Gou Feng Li Zeyu Wang Shu Zhong Weihao Yu Xiaonan Nie Ziang Song Guang Shi and Haoqi Fan. 2025. Emerging Properties in Unified Multimodal Pretraining. arXiv:2505.14683 [cs.CV] https:\/\/arxiv.org\/abs\/2505.14683"},{"key":"e_1_2_1_18_1","doi-asserted-by":"crossref","first-page":"3083","DOI":"10.14778\/3476311.3476385","article-title":"Big Metadata: When Metadata is Big Data","volume":"14","author":"Edara Pavan","year":"2021","unstructured":"Pavan Edara and Mosha Pasumansky. 2021. Big Metadata: When Metadata is Big Data. Proceedings of the VLDB Endowment (VLDB) 14, 12 (2021), 3083\u20133095. http:\/\/www.vldb.org\/pvldb\/vol14\/p3083-edara.pdf","journal-title":"Proceedings of the VLDB Endowment (VLDB)"},{"key":"e_1_2_1_19_1","unstructured":"Apache Software Foundation. 2025. Doris storage file format optimization. https:\/\/doris.apache.org\/community\/design\/doris_storage_optimization\/"},{"key":"e_1_2_1_20_1","unstructured":"Apache Software Foundation. 2025. HFile. https:\/\/hbase.apache.org\/devapidocs\/org\/apache\/hadoop\/hbase\/io\/hfile\/HFile.html File format for hbase."},{"key":"e_1_2_1_21_1","unstructured":"LF AI & Data Foundation. 2025. Milvus. https:\/\/milvus.io\/ An open-source vector database built for GenAI applications. Install with pip perform high-speed searches and scale to tens of billions of vectors with minimal performance loss."},{"key":"e_1_2_1_22_1","unstructured":"Yu Gao Haoyuan Guo Tuyen Hoang Weilin Huang Lu Jiang Fangyuan Kong Huixia Li Jiashi Li Liang Li Xiaojie Li Xunsong Li Yifu Li Shanchuan Lin Zhijie Lin Jiawei Liu Shu Liu Xiaonan Nie Zhiwu Qing Yuxi Ren Li Sun Zhi Tian Rui Wang Sen Wang Guoqiang Wei Guohong Wu Jie Wu Ruiqi Xia Fei Xiao Xuefeng Xiao Jiangqiao Yan Ceyuan Yang Jianchao Yang Runkai Yang Tao Yang Yihang Yang Zilyu Ye Xuejiao Zeng Yan Zeng Heng Zhang Yang Zhao Xiaozheng Zheng Peihao Zhu Jiaxin Zou and Feilong Zuo. 2025. Seedance 1.0: Exploring the Boundaries of Video Generation Models. arXiv:2506.09113 [cs.CV] https:\/\/arxiv.org\/abs\/2506.09113"},{"key":"e_1_2_1_23_1","unstructured":"Google. 2025. FlatBuffers. https:\/\/flatbuffers.dev\/ An efficient cross platform serialization library for C++ C# C Go Java Kotlin JavaScript Lobster Lua TypeScript PHP Python Rust and Swift."},{"key":"e_1_2_1_24_1","unstructured":"Google. 2025. Google BigQuery. https:\/\/cloud.google.com\/bigquery\/ The autonomous data to AI platform automating the entire data life cycle from ingestion to AI-driven insights."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2023.3270101"},{"key":"e_1_2_1_26_1","volume-title":"Deep Lake: A Lakehouse for Deep Learning. arXiv:2209.10785 [cs.DC] https:\/\/arxiv.org\/abs\/2209.10785","author":"Hambardzumyan Sasun","year":"2022","unstructured":"Sasun Hambardzumyan, Abhinav Tuli, Levon Ghukasyan, Fariz Rahman, Hrant Topchyan, David Isayan, Mark McQuade, Mikayel Harutyunyan, Tatevik Hakobyan, Ivo Stranic, and Davit Buniatyan. 2022. Deep Lake: A Lakehouse for Deep Learning. arXiv:2209.10785 [cs.DC] https:\/\/arxiv.org\/abs\/2209.10785"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591687"},{"key":"e_1_2_1_28_1","volume-title":"Analyzing and Comparing Lakehouse Storage Systems. In Conference on Innovative Data Systems Research (CIDR). www.cidrdb.org. https:\/\/www.cidrdb.org\/cidr2023\/papers\/p92-jain.pdf","author":"Jain Paras","year":"2023","unstructured":"Paras Jain, Peter Kraft, Conor Power, Tathagata Das, Ion Stoica, and Matei Zaharia. 2023. Analyzing and Comparing Lakehouse Storage Systems. In Conference on Innovative Data Systems Research (CIDR). www.cidrdb.org. https:\/\/www.cidrdb.org\/cidr2023\/papers\/p92-jain.pdf"},{"key":"e_1_2_1_29_1","volume-title":"USENIX Symposium on Networked Systems Design and Implementation (NSDI). USENIX Association, 745\u2013760","author":"Jiang Ziheng","year":"2024","unstructured":"Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, Zhi Zhang, Yanghua Peng, Xiang Li, Cong Xie, Shibiao Nong, Yulu Jia, Sun He, Hongmin Chen, Zhihao Bai, Qi Hou, Shipeng Yan, Ding Zhou, Yiyao Sheng, Zhuo Jiang, Haohan Xu, Haoran Wei, Zhang Zhang, Pengfei Nie, Leqi Zou, Sida Zhao, Liang Xiang, Zherui Liu, Zhe Li, Xiaoying Jia, Jianxi Ye, Xin Jin, and Xin Liu. 2024. MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs. In USENIX Symposium on Networked Systems Design and Implementation (NSDI). USENIX Association, 745\u2013760. https:\/\/www.usenix.org\/conference\/nsdi24\/presentation\/jiang-ziheng"},{"key":"e_1_2_1_30_1","unstructured":"LanceDB. 2025. LanceDB. https:\/\/lancedb.com\/ A unified data platform designed for multimodal data and built for enterprise scale."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626246.3653388"},{"key":"e_1_2_1_32_1","volume-title":"Abadi","author":"Liao Gang","year":"2024","unstructured":"Gang Liao, Ye Liu, Jianjun Chen, and Daniel J. Abadi. 2024. Bullion: A Column Store for Machine Learning. arXiv:2404.08901 [cs.DB] https:\/\/arxiv.org\/abs\/2404.08901"},{"key":"e_1_2_1_33_1","volume-title":"Monolith: Real Time Recommendation System with Collisionless Embedding Table. 3303","author":"Liu Zhuoran","year":"2022","unstructured":"Zhuoran Liu, Leqi Zou, Xuan Zou, Caihua Wang, Biao Zhang, Da Tang, Bolin Zhu, Yijie Zhu, Peng Wu, Ke Wang, and Youlong Cheng. 2022. Monolith: Real Time Recommendation System with Collisionless Embedding Table. 3303 (2022). https:\/\/ceur-ws.org\/Vol-3303\/paper8.pdf"},{"key":"e_1_2_1_34_1","doi-asserted-by":"crossref","first-page":"3461","DOI":"10.14778\/3415478.3415568","article-title":"Dremel: A Decade of Interactive SQL Analysis at Web Scale","volume":"13","author":"Melnik Sergey","year":"2020","unstructured":"Sergey Melnik, Andrey Gubarev, Jing Jing Long, Geoffrey Romer, Shiva Shivakumar, Matt Tolton, Theo Vassilakis, Hossein Ahmadi, Dan Delorey, Slava Min, Mosha Pasumansky, and Jeff Shute. 2020. Dremel: A Decade of Interactive SQL Analysis at Web Scale. Proceedings of the VLDB Endowment (VLDB) 13, 12 (2020), 3461\u20133472. http:\/\/www.vldb.org\/pvldb\/vol13\/p3461-melnik.pdf","journal-title":"Proceedings of the VLDB Endowment (VLDB)"},{"key":"e_1_2_1_35_1","unstructured":"Elastic NV. 2025. Elasticsearch. https:\/\/www.elastic.co\/elasticsearch Free and open source distributed RESTful search engine."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685834"},{"key":"e_1_2_1_37_1","volume-title":"Lance: Efficient Random Access in Columnar Storage through Adaptive Structural Encodings. arXiv:2504.15247 [cs.DB] https:\/\/arxiv.org\/abs\/2504.15247","author":"Pace Weston","year":"2025","unstructured":"Weston Pace, Chang She, Lei Xu, Will Jones, Albert Lockett, Jun Wang, and Raunak Shah. 2025. Lance: Efficient Random Access in Columnar Storage through Adaptive Structural Encodings. arXiv:2504.15247 [cs.DB] https:\/\/arxiv.org\/abs\/2504.15247"},{"key":"e_1_2_1_38_1","unstructured":"Meta Platforms. 2025. The Nimble File Format. https:\/\/github.com\/facebookincubator\/nimble New file format for storage of large columnar datasets."},{"key":"e_1_2_1_39_1","unstructured":"Meta Platforms. 2025. Velox. https:\/\/velox-lib.io\/ An open-source composable execution engine for data systems."},{"key":"e_1_2_1_40_1","volume-title":"Primus: Unified Training System for Large-Scale Deep Learning Recommendation Models. In USENIX Annual Technical Conference (ATC). USENIX Association, 905\u2013922","author":"Shan Jixi","year":"2025","unstructured":"Jixi Shan, Xiuqi Huang, Yang Guo, Hongyue Mao, Ho-Pang Hsu, Hang Cheng, Can Wang, Jun Song, Rui Shi, Xiaofeng Gao, Jingwei Xu, Shiru Ren, Jiaxiao Zheng, Hua Huang, Lele Yu, Peng Xu, and Guihai Chen. 2025. Primus: Unified Training System for Large-Scale Deep Learning Recommendation Models. In USENIX Annual Technical Conference (ATC). USENIX Association, 905\u2013922. https:\/\/www.usenix.org\/conference\/atc25\/presentation\/shan-jixi"},{"key":"e_1_2_1_41_1","unstructured":"Spiral. 2025. Vortex. https:\/\/github.com\/vortex-data\/vortex An extensible state of the art columnar file format."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3580305.3599797"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE60146.2024.00386"},{"key":"e_1_2_1_44_1","unstructured":"The Mosaic ML Team. 2022. streaming. https:\/\/github.com\/mosaicml\/streaming\/Fast accurate streaming of training data from cloud storage."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE65448.2025.00139"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685804"},{"key":"e_1_2_1_47_1","unstructured":"Ling Yang Ye Tian Bowen Li Xinchen Zhang Ke Shen Yunhai Tong and Mengdi Wang. 2025. MMaDA: Multimodal Large Diffusion Language Models. arXiv:2505.15809 [cs.CV] https:\/\/arxiv.org\/abs\/2505.15809"},{"key":"e_1_2_1_48_1","volume-title":"Conference on Innovative Data Systems Research (CIDR). www.cidrdb.org. http:\/\/cidrdb.org\/cidr2021\/papers\/cidr2021_paper17","author":"Zaharia Matei","year":"2021","unstructured":"Matei Zaharia, Ali Ghodsi, Reynold Xin, and Michael Armbrust. 2021. Lake-house: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics. In Conference on Innovative Data Systems Research (CIDR). www.cidrdb.org. http:\/\/cidrdb.org\/cidr2021\/papers\/cidr2021_paper17.pdf"},{"key":"e_1_2_1_49_1","volume-title":"International Conference on Machine Learning (ICML). PMLR, Article 2414","author":"Zhai Jiaqi","year":"2024","unstructured":"Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, Yinghai Lu, and Yu Shi. 2024. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations. In International Conference on Machine Learning (ICML). PMLR, Article 2414, 26 pages. https:\/\/proceedings.mlr.press\/v235\/zhai24a.html"},{"key":"e_1_2_1_50_1","volume-title":"International Conference on Machine Learning (ICML). PMLR, Article 2455","author":"Zhang Buyun","year":"2024","unstructured":"Buyun Zhang, Liang Luo, Yuxin Chen, Jade Nie, Xi Liu, Shen Li, Yanli Zhao, Yuchen Hao, Yantao Yao, Ellie Dingqiao Wen, Jongsoo Park, Maxim Naumov, and Wenlin Chen. 2024. Wukong: Towards a Scaling Law for Large-Scale Recommendation. In International Conference on Machine Learning (ICML). PMLR, Article 2455, 14 pages. https:\/\/proceedings.mlr.press\/v235\/zhang24ao.html"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3470496.3533044"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3750601.3750620","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T13:41:30Z","timestamp":1758030090000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3750601.3750620"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8]]},"references-count":51,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2025,8]]}},"alternative-id":["10.14778\/3750601.3750620"],"URL":"https:\/\/doi.org\/10.14778\/3750601.3750620","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2025,8]]},"assertion":[{"value":"2025-09-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}