{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T04:34:54Z","timestamp":1771043694763,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":61,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,6,29]],"date-time":"2020-06-29T00:00:00Z","timestamp":1593388800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,6,29]]},"DOI":"10.1145\/3379597.3387473","type":"proceedings-article","created":{"date-parts":[[2020,9,19]],"date-time":"2020-09-19T02:12:49Z","timestamp":1600481569000},"page":"431-442","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":64,"title":["The State of the ML-universe"],"prefix":"10.1145","author":[{"given":"Danielle","family":"Gonzalez","sequence":"first","affiliation":[{"name":"Rochester Institute of Technology, Rochester, NY, USA"}]},{"given":"Thomas","family":"Zimmermann","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, WA, USA"}]},{"given":"Nachiappan","family":"Nagappan","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, WA, USA"}]}],"member":"320","published-online":{"date-parts":[[2020,9,18]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2009.5070530"},{"key":"e_1_3_2_1_3_1","volume-title":"Requirements Engineering Challenges in Building AI-Based Complex Systems. In 2019 IEEE 27th International Requirements Engineering Conference Workshops (REW). IEEE","author":"Belani H.","year":"2019","unstructured":"H. Belani , M. Vukovic , and \u017d. Car. 2019 . Requirements Engineering Challenges in Building AI-Based Complex Systems. In 2019 IEEE 27th International Requirements Engineering Conference Workshops (REW). IEEE , New York, NY, 252--255. https:\/\/doi.org\/10.1109\/REW. 2019.00051 10.1109\/REW.2019.00051 H. Belani, M. Vukovic, and \u017d. Car. 2019. Requirements Engineering Challenges in Building AI-Based Complex Systems. In 2019 IEEE 27th International Requirements Engineering Conference Workshops (REW). IEEE, New York, NY, 252--255. https:\/\/doi.org\/10.1109\/REW.2019.00051"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3340571"},{"key":"e_1_3_2_1_5_1","volume-title":"NLTK: Natural Language Toolkit. https:\/\/github.com\/nltk\/nltk.","author":"Bird Steven","year":"2009","unstructured":"Steven Bird , Edward Loper , and Ewan Klein . 2009 . NLTK: Natural Language Toolkit. https:\/\/github.com\/nltk\/nltk. Steven Bird, Edward Loper, and Ewan Klein. 2009. NLTK: Natural Language Toolkit. https:\/\/github.com\/nltk\/nltk."},{"key":"e_1_3_2_1_6_1","volume-title":"Understanding the Factors That Impact the Popularity of GitHub Repositories. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE","author":"Borges H.","year":"2016","unstructured":"H. Borges , A. Hora , and M. T. Valente . 2016 . Understanding the Factors That Impact the Popularity of GitHub Repositories. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE , New York, NY, USA, 334--344. https:\/\/doi.org\/10.1109\/ICSME. 2016 .31 10.1109\/ICSME.2016.31 H. Borges, A. Hora, and M. T. Valente. 2016. Understanding the Factors That Impact the Popularity of GitHub Repositories. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, New York, NY, USA, 334--344. https:\/\/doi.org\/10.1109\/ICSME.2016.31"},{"key":"e_1_3_2_1_7_1","volume-title":"Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '08)","author":"Cataldo Marcelo","unstructured":"Marcelo Cataldo , James D. Herbsleb , and Kathleen M. Carley . 2008. Socio-Technical Congruence: A Framework for Assessing the Impact of Technical and Work Dependencies on Software Development Productivity . In Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '08) . Association for Computing Machinery, New York, NY, USA, 2--11. https:\/\/doi.org\/10.1145\/1414004.1414008 10.1145\/1414004.1414008 Marcelo Cataldo, James D. Herbsleb, and Kathleen M. Carley. 2008. Socio-Technical Congruence: A Framework for Assessing the Impact of Technical and Work Dependencies on Software Development Productivity. In Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '08). Association for Computing Machinery, New York, NY, USA, 2--11. https:\/\/doi.org\/10.1145\/1414004.1414008"},{"key":"e_1_3_2_1_8_1","volume-title":"Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work (CSCW '06)","author":"Cataldo Marcelo","year":"1808","unstructured":"Marcelo Cataldo , Patrick A. Wagstrom , James D. Herbsleb , and Kathleen M. Carley . 2006. Identification of Coordination Requirements: Implications for the Design of Collaboration and Awareness Tools . In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work (CSCW '06) . Association for Computing Machinery, New York, NY, USA, 353--362. https:\/\/doi.org\/10.1145\/1 1808 75.1180929 10.1145\/1180875.1180929 Marcelo Cataldo, Patrick A. Wagstrom, James D. Herbsleb, and Kathleen M. Carley. 2006. Identification of Coordination Requirements: Implications for the Design of Collaboration and Awareness Tools. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work (CSCW '06). Association for Computing Machinery, New York, NY, USA, 353--362. https:\/\/doi.org\/10.1145\/1180875.1180929"},{"key":"e_1_3_2_1_9_1","volume-title":"Cilib: Typesafe, purely functional Computational Intelligence. https:\/\/github.com\/cirg-up\/cilib.","author":"Community Cilib Development","year":"2009","unstructured":"Cilib Development Community . 2009 . Cilib: Typesafe, purely functional Computational Intelligence. https:\/\/github.com\/cirg-up\/cilib. Cilib Development Community. 2009. Cilib: Typesafe, purely functional Computational Intelligence. https:\/\/github.com\/cirg-up\/cilib."},{"key":"e_1_3_2_1_10_1","unstructured":"DS-Modules Development Community. 2017. Data Prediction and Law. https:\/\/github.com\/ds-modules\/LEGALST-190.  DS-Modules Development Community. 2017. Data Prediction and Law. https:\/\/github.com\/ds-modules\/LEGALST-190."},{"key":"e_1_3_2_1_11_1","volume-title":"Faceswap: Deepfakes Software For All. https:\/\/github.com\/deepfakes\/faceswap.","author":"Community Faceswap Development","year":"2017","unstructured":"Faceswap Development Community . 2017 . Faceswap: Deepfakes Software For All. https:\/\/github.com\/deepfakes\/faceswap. Faceswap Development Community. 2017. Faceswap: Deepfakes Software For All. https:\/\/github.com\/deepfakes\/faceswap."},{"key":"e_1_3_2_1_12_1","unstructured":"JuliaLang Development Community. 2011. The Julia Language: A fresh approach to technical computing. https:\/\/github.com\/JuliaLang\/julia.  JuliaLang Development Community. 2011. The Julia Language: A fresh approach to technical computing. https:\/\/github.com\/JuliaLang\/julia."},{"key":"e_1_3_2_1_13_1","unstructured":"Mage Development Community. 2012. XMage: Magic Another Game Engine. https:\/\/github.com\/magefree\/mage.  Mage Development Community. 2012. XMage: Magic Another Game Engine. https:\/\/github.com\/magefree\/mage."},{"key":"e_1_3_2_1_14_1","unstructured":"NEKit Development Community. 2016. A toolkit for Network Extension Framework. https:\/\/github.com\/zhuhaow\/NEKit.  NEKit Development Community. 2016. A toolkit for Network Extension Framework. https:\/\/github.com\/zhuhaow\/NEKit."},{"key":"e_1_3_2_1_15_1","unstructured":"OpenKore Development Community. 2016. A free\/open source client and automation tool for Ragnarok Online. https:\/\/github.com\/OpenKore\/openkore.  OpenKore Development Community. 2016. A free\/open source client and automation tool for Ragnarok Online. https:\/\/github.com\/OpenKore\/openkore."},{"key":"e_1_3_2_1_16_1","unstructured":"PyTorch Development Community. 2016. PyTorch:Tensors and Dynamic neural networks in Python with strong GPU acceleration. https:\/\/github.com\/pytorch\/pytorch.  PyTorch Development Community. 2016. PyTorch:Tensors and Dynamic neural networks in Python with strong GPU acceleration. https:\/\/github.com\/pytorch\/pytorch."},{"key":"e_1_3_2_1_17_1","unstructured":"SciKit-Learn Development Community. 2010. SciKit-Learn:Tensors and Dynamic neural networks in Python with strong GPU acceleration. https:\/\/github.com\/scikit-learn\/scikit-learn.  SciKit-Learn Development Community. 2010. SciKit-Learn:Tensors and Dynamic neural networks in Python with strong GPU acceleration. https:\/\/github.com\/scikit-learn\/scikit-learn."},{"key":"e_1_3_2_1_18_1","unstructured":"Tesseract Development Community. 2014. Tesseract Open Source OCR Engine. https:\/\/github.com\/tesseract-ocr\/tesseract.  Tesseract Development Community. 2014. Tesseract Open Source OCR Engine. https:\/\/github.com\/tesseract-ocr\/tesseract."},{"key":"e_1_3_2_1_19_1","volume-title":"Tensorflow: An Open Source Machine Learning Framework for Everyone. https:\/\/github.com\/tensorflow\/tensorflow.","author":"Community Tensorflow Development","year":"2015","unstructured":"Tensorflow Development Community . 2015 . Tensorflow: An Open Source Machine Learning Framework for Everyone. https:\/\/github.com\/tensorflow\/tensorflow. Tensorflow Development Community. 2015. Tensorflow: An Open Source Machine Learning Framework for Everyone. https:\/\/github.com\/tensorflow\/tensorflow."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2017.2682323"},{"key":"e_1_3_2_1_21_1","volume-title":"Understanding Development Process of Machine Learning Systems: Challenges and Solutions. In 2019 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE","author":"Nascimento E. d. S.","year":"2019","unstructured":"E. d. S. Nascimento , I. Ahmed , E. Oliveira , M. P. Palheta , I. Steinmacher , and T. Conte . 2019 . Understanding Development Process of Machine Learning Systems: Challenges and Solutions. In 2019 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE , New York, NY, 1--6. https:\/\/doi.org\/10.1109\/ESEM. 2019 .8870157 10.1109\/ESEM.2019.8870157 E. d. S. Nascimento, I. Ahmed, E. Oliveira, M. P. Palheta, I. Steinmacher, and T. Conte. 2019. Understanding Development Process of Machine Learning Systems: Challenges and Solutions. In 2019 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE, New York, NY, 1--6. https:\/\/doi.org\/10.1109\/ESEM.2019.8870157"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145396"},{"key":"e_1_3_2_1_23_1","unstructured":"Evans Data Corporation. 2019. Global Developer Population and Demographic Study. https:\/\/evansdata.com\/reports\/viewRelease.php?reportID=9.  Evans Data Corporation. 2019. Global Developer Population and Demographic Study. https:\/\/evansdata.com\/reports\/viewRelease.php?reportID=9."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1049\/ic.2011.0019"},{"key":"e_1_3_2_1_25_1","unstructured":"Shay Frendt. 2017. Introducing Topics. https:\/\/github.blog\/2017-01-31-introducing-topics\/  Shay Frendt. 2017. Introducing Topics. https:\/\/github.blog\/2017-01-31-introducing-topics\/"},{"key":"e_1_3_2_1_26_1","unstructured":"GitHub. 2019. The State of the Octoverse. https:\/\/octoverse.github.com\/.  GitHub. 2019. The State of the Octoverse. https:\/\/octoverse.github.com\/."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/2487085.2487132"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-C.2017.164"},{"key":"e_1_3_2_1_29_1","unstructured":"GitHub Help. 2019. About Collaborative Development Models. https:\/\/help.github.com\/en\/articles\/about-collaborative-development-models  GitHub Help. 2019. About Collaborative Development Models. https:\/\/help.github.com\/en\/articles\/about-collaborative-development-models"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985441.1985466"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"crossref","unstructured":"Nargiz Humbatova Gunel Jahangirova Gabriele Bavota Vincenzo Riccio Andrea Stocco and Paolo Tonella. 2019. Taxonomy of Real Faults in Deep Learning Systems. arXiv:cs.SE\/1910.11015  Nargiz Humbatova Gunel Jahangirova Gabriele Bavota Vincenzo Riccio Andrea Stocco and Paolo Tonella. 2019. Taxonomy of Real Faults in Deep Learning Systems. arXiv:cs.SE\/1910.11015","DOI":"10.1145\/3377811.3380395"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338955"},{"key":"e_1_3_2_1_33_1","volume-title":"Rangeet Pan, and Hridesh Rajan.","author":"Islam Md Johirul","year":"2019","unstructured":"Md Johirul Islam , Hoan Anh Nguyen , Rangeet Pan, and Hridesh Rajan. 2019 . What Do Developers Ask About ML Libraries? A Large-scale Study Using Stack Overflow . arXiv:cs.SE\/1906.11940 Md Johirul Islam, Hoan Anh Nguyen, Rangeet Pan, and Hridesh Rajan. 2019. What Do Developers Ask About ML Libraries? A Large-scale Study Using Stack Overflow. arXiv:cs.SE\/1906.11940"},{"key":"e_1_3_2_1_35_1","volume-title":"An in-depth study of the promises and perils of mining GitHub. Empirical Software Engineering 21, 5 (01","author":"Kalliamvakou Eirini","year":"2016","unstructured":"Eirini Kalliamvakou , Georgios Gousios , Kelly Blincoe , Leif Singer , Daniel M. German , and Daniela Damian . 2016. An in-depth study of the promises and perils of mining GitHub. Empirical Software Engineering 21, 5 (01 Oct 2016 ), 2035--2071. https:\/\/doi.org\/10.1007\/s10664-015-9393-5 10.1007\/s10664-015-9393-5 Eirini Kalliamvakou, Georgios Gousios, Kelly Blincoe, Leif Singer, Daniel M. German, and Daniela Damian. 2016. An in-depth study of the promises and perils of mining GitHub. Empirical Software Engineering 21, 5 (01 Oct 2016), 2035--2071. https:\/\/doi.org\/10.1007\/s10664-015-9393-5"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2754374"},{"key":"e_1_3_2_1_37_1","volume-title":"Helena Holmstr\u00f6m Olsson, and Ivica Crnkovic","author":"Lwakatare Lucy Ellen","year":"2019","unstructured":"Lucy Ellen Lwakatare , Aiswarya Raj , Jan Bosch , Helena Holmstr\u00f6m Olsson, and Ivica Crnkovic . 2019 . A Taxonomy of Software Engineering Challenges for Machine Learning Systems: An Empirical Investigation. In Agile Processes in Software Engineering and Extreme Programming, Philippe Kruchten, Steven Fraser, and Fran\u00e7ois Coallier (Eds.). Springer International Publishing , Cham, 227--243. Lucy Ellen Lwakatare, Aiswarya Raj, Jan Bosch, Helena Holmstr\u00f6m Olsson, and Ivica Crnkovic. 2019. A Taxonomy of Software Engineering Challenges for Machine Learning Systems: An Empirical Investigation. In Agile Processes in Software Engineering and Extreme Programming, Philippe Kruchten, Steven Fraser, and Fran\u00e7ois Coallier (Eds.). Springer International Publishing, Cham, 227--243."},{"key":"e_1_3_2_1_38_1","unstructured":"Microsoft. 2014. The foundational class libraries for. NET Core. https:\/\/github.com\/dotnet\/corefx.  Microsoft. 2014. The foundational class libraries for. NET Core. https:\/\/github.com\/dotnet\/corefx."},{"key":"e_1_3_2_1_39_1","unstructured":"Microsoft. 2015. Visual Studio Code. https:\/\/github.com\/microsoft\/vscode.  Microsoft. 2015. Visual Studio Code. https:\/\/github.com\/microsoft\/vscode."},{"key":"e_1_3_2_1_40_1","unstructured":"Joseph Misiti. 2015. Awesome Machine Learning: A curated list of awesome Machine Learning frameworks libraries and software. https:\/\/github.com\/josephmisiti\/awesome-machine-learning  Joseph Misiti. 2015. Awesome Machine Learning: A curated list of awesome Machine Learning frameworks libraries and software. https:\/\/github.com\/josephmisiti\/awesome-machine-learning"},{"key":"e_1_3_2_1_41_1","volume-title":"Curating GitHub for engineered software projects. Empirical Software Engineering 22, 6 (01","author":"Munaiah Nuthan","year":"2017","unstructured":"Nuthan Munaiah , Steven Kroh , Craig Cabrey , and Meiyappan Nagappan . 2017. Curating GitHub for engineered software projects. Empirical Software Engineering 22, 6 (01 Dec 2017 ), 3219--3253. https:\/\/doi.org\/10.1007\/s10664-017-9512-6 10.1007\/s10664-017-9512-6 Nuthan Munaiah, Steven Kroh, Craig Cabrey, and Meiyappan Nagappan. 2017. Curating GitHub for engineered software projects. Empirical Software Engineering 22, 6 (01 Dec 2017), 3219--3253. https:\/\/doi.org\/10.1007\/s10664-017-9512-6"},{"key":"#cr-split#-e_1_3_2_1_42_1.1","doi-asserted-by":"crossref","unstructured":"E. Murphy-Hill C. Jaspan C. Sadowski D. Shepherd M. Phillips C. Winter A. Knight E. Smith and M. Jorde. 2019. What Predicts Software Developers' Productivity? IEEE Transactions on Software Engineering null null (2019) 1--1. https:\/\/doi.org\/10.1109\/TSE.2019.2900308 10.1109\/TSE.2019.2900308","DOI":"10.1109\/TSE.2019.2900308"},{"key":"#cr-split#-e_1_3_2_1_42_1.2","doi-asserted-by":"crossref","unstructured":"E. Murphy-Hill C. Jaspan C. Sadowski D. Shepherd M. Phillips C. Winter A. Knight E. Smith and M. Jorde. 2019. What Predicts Software Developers' Productivity? IEEE Transactions on Software Engineering null null (2019) 1--1. https:\/\/doi.org\/10.1109\/TSE.2019.2900308","DOI":"10.1109\/TSE.2019.2900308"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2013.6606598"},{"key":"#cr-split#-e_1_3_2_1_44_1.1","doi-asserted-by":"crossref","unstructured":"A. Rastogi and N. Nagappan. 2016. Forking and the Sustainability of the Developer Community Participation - An Empirical Investigation on Outcomes and Reasons. In 2016 IEEE 23rd International Conference on Software Analysis Evolution and Reengineering (SANER) Vol. 1. IEEE Osaka Japan 102--111. https:\/\/doi.org\/10.1109\/SANER.2016.27 10.1109\/SANER.2016.27","DOI":"10.1109\/SANER.2016.27"},{"key":"#cr-split#-e_1_3_2_1_44_1.2","doi-asserted-by":"crossref","unstructured":"A. Rastogi and N. Nagappan. 2016. Forking and the Sustainability of the Developer Community Participation - An Empirical Investigation on Outcomes and Reasons. In 2016 IEEE 23rd International Conference on Software Analysis Evolution and Reengineering (SANER) Vol. 1. IEEE Osaka Japan 102--111. https:\/\/doi.org\/10.1109\/SANER.2016.27","DOI":"10.1109\/SANER.2016.27"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126905"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635922"},{"key":"e_1_3_2_1_47_1","volume-title":"Rethinking Productivity in Software Engineering","author":"Sadowski Caitlin","unstructured":"Caitlin Sadowski and Thomas Zimmermann . 2019. Rethinking Productivity in Software Engineering . Apress Open , New York, NY. Caitlin Sadowski and Thomas Zimmermann. 2019. Rethinking Productivity in Software Engineering. Apress Open, New York, NY."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3084226.3084287"},{"key":"#cr-split#-e_1_3_2_1_49_1.1","doi-asserted-by":"crossref","unstructured":"M. Storey T. Zimmermann C. Bird J. Czerwonka B. Murphy and E. Kalliamvakou. 2019. Towards a Theory of Software Developer Job Satisfaction and Perceived Productivity. IEEE Transactions on Software Engineering null null (2019) 1--1. https:\/\/doi.org\/10.1109\/TSE.2019.2944354 10.1109\/TSE.2019.2944354","DOI":"10.1109\/TSE.2019.2944354"},{"key":"#cr-split#-e_1_3_2_1_49_1.2","doi-asserted-by":"crossref","unstructured":"M. Storey T. Zimmermann C. Bird J. Czerwonka B. Murphy and E. Kalliamvakou. 2019. Towards a Theory of Software Developer Job Satisfaction and Perceived Productivity. IEEE Transactions on Software Engineering null null (2019) 1--1. https:\/\/doi.org\/10.1109\/TSE.2019.2944354","DOI":"10.1109\/TSE.2019.2944354"},{"key":"e_1_3_2_1_50_1","volume-title":"An Empirical Study on Real Bugs for Machine Learning Programs. In 2017 24th Asia-Pacific Software Engineering Conference (APSEC). IEEE Press","author":"Sun X.","year":"2017","unstructured":"X. Sun , T. Zhou , G. Li , J. Hu , H. Yang , and B. Li . 2017 . An Empirical Study on Real Bugs for Machine Learning Programs. In 2017 24th Asia-Pacific Software Engineering Conference (APSEC). IEEE Press , Piscataway, NJ, USA, 348--357. https:\/\/doi.org\/10.1109\/APSEC. 2017 .41 10.1109\/APSEC.2017.41 X. Sun, T. Zhou, G. Li, J. Hu, H. Yang, and B. Li. 2017. An Empirical Study on Real Bugs for Machine Learning Programs. In 2017 24th Asia-Pacific Software Engineering Conference (APSEC). IEEE Press, Piscataway, NJ, USA, 348--357. https:\/\/doi.org\/10.1109\/APSEC.2017.41"},{"key":"e_1_3_2_1_51_1","volume-title":"An Empirical Study of Bugs in Machine Learning Systems. In 2012 IEEE 23rd International Symposium on Software Reliability Engineering. IEEE","author":"Thung F.","year":"2012","unstructured":"F. Thung , S. Wang , D. Lo , and L. Jiang . 2012 . An Empirical Study of Bugs in Machine Learning Systems. In 2012 IEEE 23rd International Symposium on Software Reliability Engineering. IEEE , New York City, NY, 271--280. https:\/\/doi.org\/10.1109\/ISSRE. 2012 .22 10.1109\/ISSRE.2012.22 F. Thung, S. Wang, D. Lo, and L. Jiang. 2012. An Empirical Study of Bugs in Machine Learning Systems. In 2012 IEEE 23rd International Symposium on Software Reliability Engineering. IEEE, New York City, NY, 271--280. https:\/\/doi.org\/10.1109\/ISSRE.2012.22"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786850"},{"key":"e_1_3_2_1_53_1","unstructured":"Stefan Wagner and Melanie Ruhe. 2018. A Systematic Review of Productivity Factors in Software Development. arXiv:cs.SE\/1801.06475  Stefan Wagner and Melanie Ruhe. 2018. A Systematic Review of Productivity Factors in Software Development. arXiv:cs.SE\/1801.06475"},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2921343"},{"key":"e_1_3_2_1_55_1","volume-title":"Studying Software Engineering Patterns for Designing Machine Learning Systems. In 2019 10th International Workshop on Empirical Software Engineering in Practice (IWESEP). IEEE","author":"Washizaki H.","year":"2019","unstructured":"H. Washizaki , H. Uchida , F. Khomh , and Y. Gu\u00e9h\u00e9neuc . 2019 . Studying Software Engineering Patterns for Designing Machine Learning Systems. In 2019 10th International Workshop on Empirical Software Engineering in Practice (IWESEP). IEEE , New York City, NY, 49--55. https:\/\/doi.org\/10.1109\/IWESEP49350. 2019 .00017 10.1109\/IWESEP49350.2019.00017 H. Washizaki, H. Uchida, F. Khomh, and Y. Gu\u00e9h\u00e9neuc. 2019. Studying Software Engineering Patterns for Designing Machine Learning Systems. In 2019 10th International Workshop on Empirical Software Engineering in Practice (IWESEP). IEEE, New York City, NY, 49--55. https:\/\/doi.org\/10.1109\/IWESEP49350.2019.00017"},{"key":"e_1_3_2_1_56_1","volume-title":"A Study of Programming Languages and Their Bug Resolution Characteristics","author":"Zhang Jie","year":"2019","unstructured":"Jie Zhang , Feng Li , Dan Hao , Meng Wang , Hao Tang , Lu Zhang , and Mark Harman . 2019. A Study of Programming Languages and Their Bug Resolution Characteristics . IEEE Transactions on Software Engineering null, null ( 2019 ), 1--1. Jie Zhang, Feng Li, Dan Hao, Meng Wang, Hao Tang, Lu Zhang, and Mark Harman. 2019. A Study of Programming Languages and Their Bug Resolution Characteristics. IEEE Transactions on Software Engineering null, null (2019), 1--1."},{"key":"e_1_3_2_1_57_1","unstructured":"Jie M. Zhang Mark Harman Lei Ma and Yang Liu. 2019. Machine Learning Testing: Survey Landscapes and Horizons. arXiv:cs.LG\/1906.10742 To appear in IEEE Transactions on Software Engineering.  Jie M. Zhang Mark Harman Lei Ma and Yang Liu. 2019. Machine Learning Testing: Survey Landscapes and Horizons. arXiv:cs.LG\/1906.10742 To appear in IEEE Transactions on Software Engineering."},{"key":"e_1_3_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2019.00020"},{"key":"e_1_3_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213866"}],"event":{"name":"MSR '20: 17th International Conference on Mining Software Repositories","location":"Seoul Republic of Korea","acronym":"MSR '20","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering","IEEE CS","SIGAI ACM Special Interest Group on Artificial Intelligence","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 17th International Conference on Mining Software Repositories"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3379597.3387473","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3379597.3387473","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:20Z","timestamp":1750200080000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3379597.3387473"}},"subtitle":["10 Years of Artificial Intelligence & Machine Learning Software Development on GitHub"],"short-title":[],"issued":{"date-parts":[[2020,6,29]]},"references-count":61,"alternative-id":["10.1145\/3379597.3387473","10.1145\/3379597"],"URL":"https:\/\/doi.org\/10.1145\/3379597.3387473","relation":{},"subject":[],"published":{"date-parts":[[2020,6,29]]},"assertion":[{"value":"2020-09-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}