{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,29]],"date-time":"2026-08-29T04:46:58Z","timestamp":1787978818564,"version":"build-2784847793"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T00:00:00Z","timestamp":1781654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,7,31]]},"abstract":"<jats:p>The proliferation of Machine Learning (ML) models and their open source implementations has transformed AI research and applications. Platforms like Hugging Face (HF) enable this evolving ecosystem, yet a large-scale longitudinal study of how these models change is lacking. This study addresses this gap by analyzing over 680,000 commits from 100,000 models and 2,251 releases from 202 of these models on HF using repository mining and longitudinal methods. We apply an extended ML change taxonomy to classify commits and use Bayesian networks to model temporal patterns in commit and release activities. Our findings show that commit activities align with established data science methodologies, such as the Cross-Industry Standard Process for Data Mining (CRISP-DM), emphasizing iterative refinement. Release patterns tend to consolidate significant updates, particularly in model outputs, sharing, and documentation, distinguishing them from granular commits. Furthermore, projects with higher popularity exhibit distinct evolutionary paths, often starting from a more mature baseline with fewer foundational commits in their public history. In contrast, those with intensive collaboration show unique documentation and technical evolution patterns. These insights enhance the understanding of model changes on community platforms and provide valuable guidance for best practices in model maintenance.<\/jats:p>","DOI":"10.1145\/3767157","type":"journal-article","created":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T13:17:28Z","timestamp":1758028648000},"page":"1-50","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["How Do Machine Learning Models Change?"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-6385-9255","authenticated-orcid":false,"given":"Joel","family":"Casta\u00f1o","sequence":"first","affiliation":[{"name":"Universitat Polit\u00e8cnica de Catalunya, Barcelona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5034-582X","authenticated-orcid":false,"given":"Rafael","family":"Caba\u00f1as","sequence":"additional","affiliation":[{"name":"Department of Mathematics and CDTIME, University of Almer\u00eda, Almer\u00eda, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4982-8725","authenticated-orcid":false,"given":"Antonio","family":"Salmer\u00f3n","sequence":"additional","affiliation":[{"name":"Department of Mathematics and CDTIME, University of Almer\u00eda, Almer\u00eda, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4367-7201","authenticated-orcid":false,"given":"David","family":"Lo","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9928-133X","authenticated-orcid":false,"given":"Silverio","family":"Mart\u00ednez-Fern\u00e1ndez","sequence":"additional","affiliation":[{"name":"Universitat Polit\u00e8cnica de Catalunya, Barcelona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,17]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Huggingface. 2024. HfApi Client. Retrieved November 14 2024 from https:\/\/huggingface.co\/docs\/huggingface_hub\/package_reference\/hf_api"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2024.103079"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER56733.2023.00080"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Adekunle Ajibode Abdul Ali Bangash Filipe Roseiro Cogo Bram Adams and Ahmed E. Hassan. 2024. Towards semantic versioning of open pre-trained language model releases on hugging face. arXiv:2409.10472. Retrieved from https:\/\/arxiv.org\/abs\/2409.10472","DOI":"10.1007\/s10664-025-10631-3"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","unstructured":"Alexandra Gonz\u00e1lez \u00c1lvarez Joel Casta\u00f1o Xavier Franch and Silverio Mart\u00ednez-Fern\u00e1ndez. 2024. Impact of ML optimization tactics on greener pre-trained ML models. arXiv:2409.12878. Retrieved from https:\/\/arxiv.org\/abs\/2409.12878","DOI":"10.1007\/s00607-025-01437-8"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-022-10282-8"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1088\/1742-5468\/2008\/10\/P10008"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TechDebt52882.2021.00016"},{"key":"e_1_3_2_10_2","first-page":"528","volume-title":"Encyclopedia of Software Engineering","author":"Basili Victor R.","year":"1994","unstructured":"Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach. 1994. The goal question metric approach. In Encyclopedia of Software Engineering, 528\u2013532."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2024.107404"},{"key":"e_1_3_2_12_2","unstructured":"Joel Casta\u00f1o Silverio Mart\u00ednez-Fern\u00e1ndez Xavier Franch and Justus Bogner. 2023. Analyzing the evolution and maintenance of ML models on hugging face. arXiv:2311.13380. Retrieved from https:\/\/arxiv.org\/abs\/2311.13380"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM56168.2023.10304801"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.14128996"},{"key":"e_1_3_2_15_2","unstructured":"Peter Chapman. 2000. CRISP-DM 1.0: Step-by-Step Data Mining Guide. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:59777418"},{"key":"e_1_3_2_16_2","volume-title":"Sampling Techniques","author":"Cochran William Gemmell","year":"1977","unstructured":"William Gemmell Cochran. 1977. Sampling Techniques. John Wiley & Sons."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1049\/sfw2.12121"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00091"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453478"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510225"},{"key":"e_1_3_2_22_2","unstructured":"GitHub Docs. 2024. About Releases on GitHub. Retrieved November 14 2024 from https:\/\/docs.github.com\/en\/repositories\/releasing-projects-on-github\/about-releases"},{"key":"e_1_3_2_23_2","unstructured":"Hugging Face Docs. 2024. Hugging Face Repository Guide. Retrieved November 14 2024 from https:\/\/huggingface.co\/docs\/huggingface_hub\/v0.13.2\/guides\/repository"},{"key":"e_1_3_2_24_2","unstructured":"Hugging Face. 2024. LMSYS Arena Leaderboard. Retrieved November 14 2024 from https:\/\/huggingface.co\/spaces\/lmsys\/chatbot-arena-leaderboard"},{"key":"e_1_3_2_25_2","unstructured":"Nir Friedman Kevin Murphy and Stuart Russell. 2013. Learning the structure of dynamic probabilistic networks. arXiv:1301.7374. Retrieved from https:\/\/arxiv.org\/abs\/1301.7374"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3239235.3240502"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","unstructured":"Haoyu Gao Mansooreh Zahedi Christoph Treude Sarita Rosenstock and Marc Cheong. 2024. Documenting ethical considerations in open source AI models. arXiv:2406.18071. Retrieved from https:\/\/arxiv.org\/abs\/2406.18071","DOI":"10.1145\/3674805.3686679"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2021.106566"},{"key":"e_1_3_2_29_2","doi-asserted-by":"crossref","unstructured":"Zoubin Ghahramani. 1997. Learning dynamic Bayesian networks. In International School on Neural Networks Initiated by IIASS and EMFCSC 168\u2013197.","DOI":"10.1007\/BFb0053999"},{"key":"e_1_3_2_30_2","unstructured":"Google. 2024. Gemini API Pricing. Retrieved November 14 2024 from https:\/\/ai.google.dev\/pricing"},{"key":"e_1_3_2_31_2","unstructured":"Google. 2025. Gemini Flash. Retrieved November 14 2024 from https:\/\/deepmind.google\/technologies\/gemini\/flash\/"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2011.63"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2009.5090025"},{"key":"e_1_3_2_34_2","first-page":"99","volume-title":"2008 International Working Conference on Mining Software Repositories","author":"Hindle Abram","year":"2008","unstructured":"Abram Hindle, Daniel M. German, and Ric Holt. 2008. What do large commits tell us? A taxonomical study of large commits. In 2008 International Working Conference on Mining Software Repositories, 99\u2013108."},{"key":"e_1_3_2_35_2","unstructured":"Hugging Face Inc. 2023. Hugging Face Hub Documentation. Retrieved from https:\/\/huggingface.co\/docs\/hub\/index"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-024-54894-0"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-68282-2"},{"key":"e_1_3_2_38_2","unstructured":"Wenxin Jiang Chingwo Cheung George K. Thiruvathukal and James C. Davis. 2023. Exploring naming conventions (and defects) of pre-trained deep learning models in hugging face and other model hubs. arXiv:2310.01642. Retrieved from https:\/\/arxiv.org\/abs\/2310.01642"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00206"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644907"},{"key":"e_1_3_2_41_2","doi-asserted-by":"crossref","unstructured":"Jason Jones Wenxin Jiang Nicholas Synovic George K. Thiruvathukal and James C. Davis. 2024. What do we know about hugging face? A systematic literature review and quantitative validation of qualitative claims. arXiv:2406.08205. Retrieved from https:\/\/arxiv.org\/abs\/2406.08205","DOI":"10.1145\/3674805.3686665"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISI58743.2023.10297271"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.5555\/1795555"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.2307\/2529310"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSA-C57050.2023.00033"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3691620.3694999"},{"key":"e_1_3_2_47_2","unstructured":"Daoyuan Li Li Li Dongsun Kim Tegawend\u00e9 F. Bissyand\u00e9 David Lo and Yves Le Traon. 2016. Watch out for this commit! A study of influential software changes. arXiv:1606.03266. Retrieved from https:\/\/arxiv.org\/abs\/1606.03266"},{"key":"e_1_3_2_48_2","volume-title":"Version Control with Git: Powerful Tools and Techniques for Collaborative Software Development","author":"Loeliger Jon","year":"2012","unstructured":"Jon Loeliger and Matthew McCullough. 2012. Version Control with Git: Powerful Tools and Techniques for Collaborative Software Development. \u201cO\u2019Reilly Media, Inc.\u201d"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2962680"},{"key":"e_1_3_2_50_2","volume-title":"Dynamic Bayesian Networks: Representation, Inference and Learning","author":"Murphy Kevin Patrick","year":"2002","unstructured":"Kevin Patrick Murphy. 2002. Dynamic Bayesian Networks: Representation, Inference and Learning. University of California, Berkeley."},{"key":"e_1_3_2_51_2","volume-title":"Machine Learning. A Probabilistic Perspective","author":"Murphy Kevin Patrick","year":"2012","unstructured":"Kevin Patrick Murphy. 2012. Machine Learning. A Probabilistic Perspective. MIT Press."},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-012-9218-8"},{"key":"e_1_3_2_53_2","unstructured":"OpenAI. 2024. OpenAI API Pricing. Retrieved November 14 2024 from https:\/\/openai.com\/api\/pricing\/"},{"key":"e_1_3_2_54_2","first-page":"1","article-title":"The power of outliers (and why researchers should always check for them)","volume":"9","author":"Osborne Jason W.","year":"2004","unstructured":"Jason W. Osborne and Amy Overbay. 2004. The power of outliers (and why researchers should always check for them). Practical Assessment, Research, and Evaluation 9, 6 (2004), 1\u20138.","journal-title":"Practical Assessment, Research, and Evaluation"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1177\/1609406919899220"},{"key":"e_1_3_2_56_2","volume-title":"Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference","author":"Pearl Judea","year":"2014","unstructured":"Judea Pearl. 2014. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Elsevier."},{"key":"e_1_3_2_57_2","volume-title":"EMELIOT Workshop at ISSSE","author":"Pepe Federica","year":"2023","unstructured":"Federica Pepe and Massimiliano Di Penta. 2023. Fairness, bias, and legal issues in pretrained models: An empirical study. In EMELIOT Workshop at ISSSE."},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13748-019-00194-y"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","unstructured":"Bogdan \u0160inik Domen Vake Jernej Vicic and Aleksandar To\u0161ic. 2024. Interactive tool for tracking open-source artificial intelligence progress on Hugging Face. In 27th International Multiconference Information Society (IS 2024) Data Mining and Data Warehouses (SiKDD). Jo\u017eef Stefan Institute Ljubljana Slovenia 1\u20134. DOI: 10.70314\/is.2024.sikdd.1","DOI":"10.70314\/is.2024.sikdd.1"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.5555\/2851535"},{"key":"e_1_3_2_61_2","first-page":"492","volume-title":"International Conference on Software Engineering","author":"Swanson E. Burton","year":"1976","unstructured":"E. Burton Swanson. 1976. The dimensions of maintenance. In International Conference on Software Engineering, 492\u2013497."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00033"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-015-9297-z"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.12.019"},{"key":"e_1_3_2_65_2","unstructured":"Zhou Yang Jieke Shi and David Lo. 2024. Ecosystem of large language models for code. arXiv:2405.16746. Retrieved from https:\/\/arxiv.org\/abs\/2405.16746"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3767157","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T14:44:50Z","timestamp":1781707490000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3767157"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,17]]},"references-count":64,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2026,7,31]]}},"alternative-id":["10.1145\/3767157"],"URL":"https:\/\/doi.org\/10.1145\/3767157","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,17]]},"assertion":[{"value":"2024-11-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-14","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}