{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T21:29:32Z","timestamp":1780522172387,"version":"3.54.1"},"reference-count":91,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T00:00:00Z","timestamp":1700784000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"crossref","award":["RGPIN-2018-05960, RGPIN-2021-04000"],"award-info":[{"award-number":["RGPIN-2018-05960, RGPIN-2021-04000"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,1,31]]},"abstract":"<jats:p>\n            Modern software systems are designed to be deployed in different configured environments (e.g., permissions, virtual resources, network connections) and adapted at runtime to different situations (e.g., memory limits, enabling\/disabling features, database credentials). Such a configuration during the deployment and runtime of a software system is implemented via a set of configuration files, which together constitute what we refer to as a \u201cconfiguration system.\u201d Recent research efforts investigated the evolution and maintenance of configuration files. However, they merely focused on a limited part of the configuration system (e.g., specific infrastructure configuration files or Dockerfiles), and their results do not generalize to the whole configuration system. To cope with such a limitation, we aim to better capture and understand what files constitute a configuration system. To do so, we leverage an open card sort technique to qualitatively study 1,756 configuration files from OpenStack, a large and widely studied open source software ecosystem. Our investigation reveals the existence of nine types of configuration files, which cover the creation of the infrastructure on top of which OpenStack will be deployed, along with other types of configuration files used to customize OpenStack after its deployment. These configuration files are interconnected while being used at different deployment stages. For instance, we observe specific configuration files used during the deployment stage to create other configuration files that are used in the runtime stage. We also observe that identifying and classifying these types of files is not straightforward, as five out of the nine types can be written in similar programming languages (e.g., Python and Bash) as regular source code files. We also found that the same file extensions (e.g.,\n            <jats:monospace>Yaml<\/jats:monospace>\n            ) can be used for different configuration types, making it difficult to identify and classify configuration files. Thus, we first leverage a machine learning model to identify configuration from non-configuration files, which achieved a median area under the curve (AUC) of 0.91, a median Brier score of 0.12, a median precision of 0.86, and a median recall of 0.83. Thereafter, we leverage a multi-class classification model to classify configuration files based on the nine configuration types. Our multi-class classification model achieved a median weighted AUC of 0.92, a median Brier score of 0.04, a median weighted precision of 0.84, and a median weighted recall of 0.82. Our analysis also shows that with only 100 labeled configuration and non-configuration files, our model reached a median AUC higher than 0.69. Furthermore, our configuration model requires a minimum of 100 configuration files to reach a median weighted AUC higher than 0.75.\n          <\/jats:p>","DOI":"10.1145\/3607186","type":"journal-article","created":{"date-parts":[[2023,7,3]],"date-time":"2023-07-03T12:13:28Z","timestamp":1688386408000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["What Constitutes the Deployment and Runtime Configuration System? An Empirical Study on OpenStack Projects"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4808-5691","authenticated-orcid":false,"given":"Narjes","family":"Bessghaier","sequence":"first","affiliation":[{"name":"University of Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2724-0034","authenticated-orcid":false,"given":"Mohammed","family":"Sayagh","sequence":"additional","affiliation":[{"name":"University of Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4708-0362","authenticated-orcid":false,"given":"Ali","family":"Ouni","sequence":"additional","affiliation":[{"name":"University of Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6010-7561","authenticated-orcid":false,"given":"Mohamed Wiem","family":"Mkaouer","sequence":"additional","affiliation":[{"name":"Rochester Institute of Technology, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,11,24]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"P. Anderson. 2006. System Configuration. SAGE. Retrieved from https:\/\/2459d6dc103cb5933875-c0245c5c937c5dedcca3f1764ecc9b2f.ssl.cf2.rackcdn.com\/books\/14_sysconfig%0A.pdf"},{"key":"e_1_3_2_3_2","unstructured":"Mona Attariyan and Jason Flinn. 2010. Automating configuration troubleshooting with dynamic information flow analysis. In Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201910) . 237\u2013250."},{"key":"e_1_3_2_4_2","first-page":"422","volume-title":"Proceedings of the 2021 IEEE International Conference on Software Analysis, Evolution, and Reengineering (SANER\u201921)","author":"Barrak Amine","year":"2021","unstructured":"Amine Barrak, Ellis E. Eghan, and Bram Adams. 2021. On the co-evolution of ML pipelines and source code-empirical study of DVC projects. In Proceedings of the 2021 IEEE International Conference on Software Analysis, Evolution, and Reengineering (SANER\u201921). IEEE, Los Alamitos, CA, 422\u2013433."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786869"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0031-3203(96)00142-2"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2"},{"key":"e_1_3_2_8_2","volume-title":"Terraform: Up & Running: Writing Infrastructure as Code","author":"Brikman Yevgeniy","year":"2019","unstructured":"Yevgeniy Brikman. 2019. Terraform: Up & Running: Writing Infrastructure as Code. O\u2019Reilly Media."},{"issue":"5","key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"2780","DOI":"10.11591\/eei.v10i5.3157","article-title":"Performance comparison of TF-IDF and Word2Vec models for emotion text classification","volume":"10","author":"Cahyani Denis Eka","year":"2021","unstructured":"Denis Eka Cahyani and Irene Patasik. 2021. Performance comparison of TF-IDF and Word2Vec models for emotion text classification. Bulletin of Electrical Engineering and Informatics 10, 5 (2021), 2780\u20132788.","journal-title":"Bulletin of Electrical Engineering and Informatics"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409727"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2010.08.100"},{"key":"e_1_3_2_13_2","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1109\/MSR.2017.67","volume-title":"Proceedings of the 2017 IEEE\/ACM 14th International Conference on Mining Software Repositories (MSR\u201917)","author":"Cito J\u00fcrgen","year":"2017","unstructured":"J\u00fcrgen Cito, Gerald Schermann, John Erik Wittern, Philipp Leitner, Sali Zumberi, and Harald C. Gall. 2017. An empirical analysis of the Docker container ecosystem on GitHub. In Proceedings of the 2017 IEEE\/ACM 14th International Conference on Mining Software Repositories (MSR\u201917). IEEE, Los Alamitos, CA, 323\u2013333."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.114.3.494"},{"key":"e_1_3_2_15_2","volume-title":"Practical Nonparametric Statistics","author":"Conover William Jay","year":"1998","unstructured":"William Jay Conover. 1998. Practical Nonparametric Statistics, Vol. 350. John Wiley & Sons."},{"key":"e_1_3_2_16_2","doi-asserted-by":"crossref","first-page":"Article 110726,","DOI":"10.1016\/j.jss.2020.110726","article-title":"Towards a catalogue of software quality metrics for infrastructure code","author":"Palma Stefano Dalla","year":"2020","unstructured":"Stefano Dalla Palma, Dario Di Nucci, Fabio Palomba, and Damian Andrew Tamburri. 2020. Towards a catalogue of software quality metrics for infrastructure code. Journal of Systems and Software 170 (2020), Article 110726, 8 pages.","journal-title":"Journal of Systems and Software"},{"key":"e_1_3_2_17_2","article-title":"Within-project defect prediction of infrastructure-as-code using product and process metrics","author":"Palma Stefano Dalla","year":"2022","unstructured":"Stefano Dalla Palma, Dario Di Nucci, Fabio Palomba, and Damian Andrew Tamburri. 2022. Within-project defect prediction of infrastructure-as-code using product and process metrics. IEEE Transactions on Software Engineering 48, 6 (2022), 2086\u20132104.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_18_2","article-title":"TF-IDF vs word embeddings for morbidity identification in clinical notes: An initial study","author":"Dessi Danilo","year":"2021","unstructured":"Danilo Dessi, Rim Helaoui, Vivek Kumar, Diego Reforgiato Recupero, and Daniele Riboni. 2021. TF-IDF vs word embeddings for morbidity identification in clinical notes: An initial study. arXiv preprint arXiv:2105.09632 (2021).","journal-title":"arXiv preprint arXiv:2105.09632"},{"issue":"1","key":"e_1_3_2_19_2","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1158\/1078-0432.CCR-07-0443","article-title":"How large a training set is needed to develop a classifier for microarray data?","volume":"14","author":"Dobbin Kevin K.","year":"2008","unstructured":"Kevin K. Dobbin, Yingdong Zhao, and Richard M. Simon. 2008. How large a training set is needed to develop a classifier for microarray data? Clinical Cancer Research 14, 1 (2008), 108\u2013114.","journal-title":"Clinical Cancer Research"},{"key":"e_1_3_2_20_2","unstructured":"Facebook Engineering. 2021. Update About the October 4th Outage. Retrieved July 9 2023 from https:\/\/engineering.fb.com\/2021\/10\/04\/networking-traffic\/outage\/."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2011.01.017"},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1109\/ICSME.2019.00092","volume-title":"Proceedings of the 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME\u201919)","author":"Guerriero Michele","year":"2019","unstructured":"Michele Guerriero, Martin Garriga, Damian A. Tamburri, and Fabio Palomba. 2019. Adoption, support, and challenges of infrastructure-as-code: Insights from industry. In Proceedings of the 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME\u201919). IEEE, Los Alamitos, CA, 580\u2013589."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3305160.3305168"},{"key":"e_1_3_2_24_2","volume-title":"Ansible: Up and Running: Automating Configuration Management and Deployment the Easy Way","author":"Hochstein Lorin","year":"2017","unstructured":"Lorin Hochstein and Rene Moser. 2017. Ansible: Up and Running: Automating Configuration Management and Deployment the Easy Way. O\u2019Reilly Media."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/2110147.2110164"},{"issue":"6","key":"e_1_3_2_26_2","first-page":"1","article-title":"A study of how Docker Compose is used to compose multi-component systems","volume":"26","author":"Ibrahim Md. Hasan","year":"2021","unstructured":"Md. Hasan Ibrahim, Mohammed Sayagh, and Ahmed E. Hassan. 2021. A study of how Docker Compose is used to compose multi-component systems. Empirical Software Engineering 26, 6 (2021), 1\u201327.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_27_2","article-title":"The ScottKnott Clustering Algorithm","author":"Jelihovschi E. G.","year":"2014","unstructured":"E. G. Jelihovschi, Jos\u00e9 Cl\u00e1udio Faria, and Ivan Bezerra Allaman. 2014. The ScottKnott Clustering Algorithm. Universidade Estadual de Santa Cruz\u2013UESC, Ilheus, Bahia, Brazil.","journal-title":"Universidade Estadual de Santa Cruz\u2013UESC, Ilheus, Bahia, Brazil."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2015.12"},{"key":"e_1_3_2_29_2","article-title":"The impact of correlated metrics on defect models","author":"Jiarpakdee Jirayus","year":"2018","unstructured":"Jirayus Jiarpakdee, Chakkrit Tantithamthavorn, and Ahmed E. Hassan. 2018. The impact of correlated metrics on defect models. arXiv preprint arXiv:1801.10271 (2018).","journal-title":"arXiv preprint arXiv:1801.10271"},{"key":"e_1_3_2_30_2","doi-asserted-by":"crossref","DOI":"10.1109\/TSE.2019.2891758","article-title":"The impact of correlated metrics on the interpretation of defect models","author":"Jiarpakdee Jirayus","year":"2021","unstructured":"Jirayus Jiarpakdee, Chakkrit Tantithamthavorn, and Ahmed E. Hassan. 2021. The impact of correlated metrics on the interpretation of defect models. IEEE Transactions on Software Engineering 47, 2 (2021), 320\u2013331.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_31_2","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1109\/ISSREW.2016.30","volume-title":"Proceedings of the 2016 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW\u201916)","author":"Jiarpakdee Jirayus","year":"2016","unstructured":"Jirayus Jiarpakdee, Chakkrit Tantithamthavorn, Akinori Ihara, and Kenichi Matsumoto. 2016. A study of redundant metrics in defect prediction datasets. In Proceedings of the 2016 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW\u201916). IEEE, Los Alamitos, CA, 51\u201352."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09848-1"},{"key":"e_1_3_2_33_2","first-page":"151","volume-title":"Proceedings of the 29th ACM\/IEEE International Conference on Automated Software Engineering","author":"Jin Dongpu","year":"2014","unstructured":"Dongpu Jin, Myra B. Cohen, Xiao Qu, and Brian Robinson. 2014. PrefFinder: Getting the right preference in configurable software systems. In Proceedings of the 29th ACM\/IEEE International Conference on Automated Software Engineering. 151\u2013162."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/2591062.2591191"},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1007\/11691730_11","volume-title":"Proceedings of the International Workshop on Data Mining for Biomedical Applications","author":"Jin Xin","year":"2006","unstructured":"Xin Jin, Anbang Xu, Rongfang Bie, and Ping Guo. 2006. Machine learning techniques and chi-square feature selection for cancer classification using SAGE gene expression profiles. In Proceedings of the International Workshop on Data Mining for Biomedical Applications. 106\u2013115."},{"issue":"6","key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1109\/TPAMI.1983.4767459","article-title":"Predicting the required number of training samples","author":"Kalayeh H. M.","year":"1983","unstructured":"H. M. Kalayeh and David A. Landgrebe. 1983. Predicting the required number of training samples. IEEE Transactions on Pattern Analysis and Machine Intelligence6 (1983), 664\u2013667.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_37_2","volume-title":"Content Analysis: An Introduction to Its Methodology","author":"Krippendorff Klaus","year":"2018","unstructured":"Klaus Krippendorff. 2018. Content Analysis: An Introduction to Its Methodology. SAGE Publications."},{"key":"e_1_3_2_38_2","doi-asserted-by":"crossref","first-page":"106593","DOI":"10.1016\/j.infsof.2021.106593","article-title":"The do\u2019s and don\u2019ts of infrastructure code: A systematic grey literature review","author":"Kumara Indika","year":"2021","unstructured":"Indika Kumara, Mart\u00edn Garriga, Angel Urbano Romeu, Dario Di Nucci, Damian Andrew Tamburri, Willem-Jan van den Heuvel, and Fabio Palomba. 2021. The do\u2019s and don\u2019ts of infrastructure code: A systematic grey literature review. Information and Software Technology 137 (2021), 106593.","journal-title":"Information and Software Technology"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09840-9"},{"key":"e_1_3_2_40_2","article-title":"A qualitative study of the benefits and costs of logging from developers\u2019 perspectives","author":"Li Heng","year":"2021","unstructured":"Heng Li, Weiyi Shang, Bram Adams, Mohammed Sayagh, and Ahmed E. Hassan. 2021. A qualitative study of the benefits and costs of logging from developers\u2019 perspectives. IEEE Transactions on Software Engineering 47, 12 (2021), 2858\u20132873.","journal-title":"IEEE Transactions on Software Engineering"},{"issue":"4","key":"e_1_3_2_41_2","first-page":"673","article-title":"Weighted area under the receiver operating characteristic curve and its application to gene selection","volume":"59","author":"Li Jialiang","year":"2010","unstructured":"Jialiang Li and Jason P. Fine. 2010. Weighted area under the receiver operating characteristic curve and its application to gene selection. Journal of the Royal Statistical Society: Series C (Applied Statistics) 59, 4 (2010), 673\u2013692.","journal-title":"Journal of the Royal Statistical Society: Series C (Applied Statistics)"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.11613\/BM.2013.018"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9381-9"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.catena.2019.104421"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1089\/106652703321825928"},{"key":"e_1_3_2_46_2","first-page":"1","volume-title":"Proceedings of the 2017 13th International Conference on Network and Service Management (CNSM\u201917)","author":"Niwa Tomonobu","year":"2017","unstructured":"Tomonobu Niwa, Yuki Kasuya, and Takeshi Kitahara. 2017. Anomaly detection for OpenStack services with process-related topological analysis. In Proceedings of the 2017 13th International Conference on Network and Service Management (CNSM\u201917). IEEE, Los Alamitos, CA, 1\u20135."},{"key":"e_1_3_2_47_2","first-page":"240","volume-title":"Proceedings of the 2004 12th IEEE International Requirements Engineering Conference.","author":"Nurmuliani Nur","year":"2004","unstructured":"Nur Nurmuliani, Didar Zowghi, and Susan P. Williams. 2004. Using card sorting technique to classify requirements change. In Proceedings of the 2004 12th IEEE International Requirements Engineering Conference. IEEE, Los Alamitos, CA, 240\u2013248."},{"key":"e_1_3_2_48_2","unstructured":"OpenStack. 2010. Example of declaration configuration file used at run-time of OpenStack. GitHub . Retrieved January 14 2022 from https:\/\/github.com\/openstack\/nova\/blob\/master\/nova\/conf\/api.py."},{"key":"e_1_3_2_49_2","unstructured":"OpenStack. 2011. Example of infrastructure-setup configuration file used during deployment of OpenStack. GitHub . Retrieved March 8 2023 from https:\/\/github.com\/openstack\/puppet-nova\/blob\/master\/manifests\/init.pp."},{"key":"e_1_3_2_50_2","unstructured":"OpenStack. 2012. Example of reader configuration file used at run-time of OpenStack. GitHub . Retrieved January 14 2022 fromhttps:\/\/github.com\/openstack\/cinder\/blob\/master\/cinder\/api\/api_utils.py."},{"key":"e_1_3_2_51_2","unstructured":"OpenStack. 2013. Example of resource-operations configuration file used at run-time of OpenStack. GitHub . Retrieved January 14 2022 from https:\/\/github.com\/openstack\/aodh\/blob\/master\/devstack\/plugin.sh."},{"key":"e_1_3_2_52_2","unstructured":"OpenStack. 2014. Example of infrastructure-setup configuration file used during deployment of OpenStack. GitHub . Retrieved January 14 2022 from https:\/\/github.com\/openstack\/kolla-ansible\/blob\/master\/ansible\/roles\/keystone\/tasks\/config.yml."},{"key":"e_1_3_2_53_2","unstructured":"OpenStack. 2014. Example of infrastructure-template configuration file used during deployment of OpenStack. GitHub . Retrieved January 14 2022 from https:\/\/github.com\/openstack\/kolla-ansible\/blob\/master\/ansible\/roles\/keystone\/templates\/keystone.conf.j2."},{"key":"e_1_3_2_54_2","unstructured":"OpenStack. 2014. Example of infrastructure-variables configuration file used during deployment of OpenStack. GitHub . Retrieved January 14 2022 from https:\/\/github.com\/openstack\/kolla-ansible\/blob\/master\/ansible\/group_vars\/all.yml."},{"key":"e_1_3_2_55_2","unstructured":"OpenStack. 2019. Example of environment-access configuration file used at run-time of OpenStack. GitHub . Retrieved January 14 2022 from https:\/\/docs.openstack.org\/python-openstackclient\/pike\/configuration\/index.html."},{"key":"e_1_3_2_56_2","unstructured":"OpenStack. 2019. Example of infrastructure-creation configuration file used during deployment of OpenStack. GitHub . Retrieved January 14 2022 from https:\/\/github.com\/openstack\/loci\/blob\/master\/scripts\/setup_pip.sh."},{"key":"e_1_3_2_57_2","unstructured":"OpenStack. 2021. Example of external configuration file used during deployment of OpenStack. GitHub . Retrieved January 14 2022 fromhttps:\/\/docs.openstack.org\/nova\/queens\/configuration\/sample-config.html."},{"key":"e_1_3_2_58_2","unstructured":"GitHub. 2022. CongIdentification. Retrieved July 9 2023 from https:\/\/github.com\/stilab-ets\/CongIdentification."},{"key":"e_1_3_2_59_2","first-page":"1","volume-title":"Proceedings of the 2019 7th International Conference on Cyber and IT Service Management (CITSM\u201919)","volume":"7","year":"2019","unstructured":"Nurhayati, Armanda Eka Putra, Luh Kesuma Wardhani, and Busman. 2019. Chi-square feature selection effect on naive Bayes classifier algorithm performance for sentiment analysis document. In Proceedings of the 2019 7th International Conference on Cyber and IT Service Management (CITSM\u201919), Vol. 7. IEEE, Los Alamitos, CA, 1\u20137."},{"key":"e_1_3_2_60_2","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1109\/ICSE.2019.00033","volume-title":"Proceedings of the 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE\u201919)","author":"Rahman Akond","year":"2019","unstructured":"Akond Rahman, Chris Parnin, and Laurie Williams. 2019. The seven sins: Security smells in infrastructure as code scripts. In Proceedings of the 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE\u201919). IEEE, Los Alamitos, CA, 164\u2013175."},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/1555228.1555269"},{"key":"e_1_3_2_62_2","first-page":"020009","volume-title":"AIP Conference Proceedings","volume":"1891","author":"Raziff Abdul Rafiez Abdul","year":"2017","unstructured":"Abdul Rafiez Abdul Raziff, Md. Nasir Sulaiman, Norwati Mustapha, and Thinagaran Perumal. 2017. Single classifier, OvO, OvA and RCC multiclass classification method in handheld based smartphone gait identification. In AIP Conference Proceedings, Vol. 1891. AIP Publishing LLC, 020009."},{"key":"e_1_3_2_63_2","first-page":"31","volume-title":"Proceedings of the 2015 IEEE 15th International Working Conference on Source Code Analysis and Manipulation (SCAM \u201915)","author":"Sayagh Mohammed","year":"2015","unstructured":"Mohammed Sayagh and Bram Adams. 2015. Multi-layer software configuration: Empirical study on WordPress. In Proceedings of the 2015 IEEE 15th International Working Conference on Source Code Analysis and Manipulation (SCAM \u201915). IEEE, Los Alamitos, CA, 31\u201340."},{"key":"e_1_3_2_64_2","article-title":"ConfigMiner: Identifying the appropriate configuration options for config-related user questions by mining online forums","author":"Sayagh Mohammed","year":"2021","unstructured":"Mohammed Sayagh and Ahmed E. Hassan. 2021. ConfigMiner: Identifying the appropriate configuration options for config-related user questions by mining online forums. IEEE Transactions on Software Engineering 47 (2021), 2907\u20132918.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_65_2","first-page":"255","volume-title":"Proceedings of the 2017 IEEE\/ACM 39th International Conference on Software Engineering (ICSE\u201917)","author":"Sayagh Mohammed","year":"2017","unstructured":"Mohammed Sayagh, Noureddine Kerzazi, and Bram Adams. 2017. On cross-stack configuration errors. In Proceedings of the 2017 IEEE\/ACM 39th International Conference on Software Engineering (ICSE\u201917). IEEE, Los Alamitos, CA, 255\u2013265."},{"issue":"6","key":"e_1_3_2_66_2","doi-asserted-by":"crossref","first-page":"646","DOI":"10.1109\/TSE.2018.2867847","article-title":"Software configuration engineering in practice interviews, survey, and systematic literature review","volume":"46","author":"Sayagh Mohammed","year":"2018","unstructured":"Mohammed Sayagh, Noureddine Kerzazi, Bram Adams, and Fabio Petrillo. 2018. Software configuration engineering in practice interviews, survey, and systematic literature review. IEEE Transactions on Software Engineering 46, 6 (2018), 646\u2013673.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09790-x"},{"key":"e_1_3_2_68_2","first-page":"220","volume-title":"Proceedings of the 2018 11th International Conference on the Quality of Information and Communications Technology (QUATIC\u201918)","author":"Schwarz Julian","year":"2018","unstructured":"Julian Schwarz, Andreas Steffens, and Horst Lichter. 2018. Code smells in infrastructure as code. In Proceedings of the 2018 11th International Conference on the Quality of Information and Communications Technology (QUATIC\u201918). IEEE, Los Alamitos, CA, 220\u2013228."},{"key":"e_1_3_2_69_2","unstructured":"Maria Korolov. 2018. Cloud security configuration errors put data at risk; new tools can help. CSO . Retrieved July 9 2023 from https:\/\/www.csoonline.com\/article\/564399\/cloud-security-configuration-errors-put-data-at-risk-new-tools-can-help.html."},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/2908080.2908083"},{"key":"e_1_3_2_71_2","first-page":"189","volume-title":"Proceedings of the 2016 IEEE\/ACM 13th Working Conference on Mining Software Repositories (MSR\u201916)","author":"Sharma Tushar","year":"2016","unstructured":"Tushar Sharma, Marios Fragkoulis, and Diomidis Spinellis. 2016. Does your configuration code smell? In Proceedings of the 2016 IEEE\/ACM 13th Working Conference on Mining Software Repositories (MSR\u201916). IEEE, Los Alamitos, CA, 189\u2013200."},{"key":"e_1_3_2_72_2","volume-title":"Card Sorting: Designing Usable Categories","author":"Spencer Donna","year":"2009","unstructured":"Donna Spencer. 2009. Card Sorting: Designing Usable Categories. Rosenfeld Media."},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/3183519.3183547"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2876537"},{"key":"e_1_3_2_75_2","doi-asserted-by":"crossref","first-page":"812","DOI":"10.1109\/ICSE.2015.93","volume-title":"Proceedings of the 2015 IEEE\/ACM 37th IEEE International Conference on Software Engineering","volume":"1","author":"Tantithamthavorn Chakkrit","year":"2015","unstructured":"Chakkrit Tantithamthavorn, Shane McIntosh, Ahmed E. Hassan, Akinori Ihara, and Kenichi Matsumoto. 2015. The impact of mislabelling on the performance and interpretation of defect prediction models. In Proceedings of the 2015 IEEE\/ACM 37th IEEE International Conference on Software Engineering, Vol. 1. IEEE, Los Alamitos, CA, 812\u2013823."},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884857"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2553030"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2584050"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2794977"},{"key":"e_1_3_2_80_2","volume-title":"Learning Chef: A Guide to Configuration Management and Automation","author":"Taylor Mischa","year":"2014","unstructured":"Mischa Taylor and Seth Vargo. 2014. Learning Chef: A Guide to Configuration Management and Automation. O\u2019Reilly Media."},{"key":"e_1_3_2_81_2","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1109\/TSA.2015.14","volume-title":"Proceedings of the 2015 2nd International Conference on Trustworthy Systems and Their Applications","author":"Tong Jia","year":"2015","unstructured":"Jia Tong, Li Ying, Yuan Xiaoyong, Tang Hongyan, and Wu Zhonghai. 2015. Characterizing and predicting bug assignment in OpenStack. In Proceedings of the 2015 2nd International Conference on Trustworthy Systems and Their Applications. IEEE, Los Alamitos, CA, 16\u201323."},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.5555\/1989619"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409709"},{"key":"e_1_3_2_84_2","first-page":"245","volume-title":"Proceedings of the 6th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201904)","author":"Wang Helen J.","year":"2004","unstructured":"Helen J. Wang, John C. Platt, Yu Chen, Ruyun Zhang, and Yi-Min Wang. 2004. Automatic misconfiguration troubleshooting with PeerPressure. In Proceedings of the 6th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201904). 245\u2013257."},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2003.12.009"},{"key":"e_1_3_2_86_2","article-title":"Predicting defective lines using a model-agnostic technique","author":"Wattanakriengkrai Supatsara","year":"2020","unstructured":"Supatsara Wattanakriengkrai, Patanamon Thongtanunam, Chakkrit Tantithamthavorn, Hideaki Hata, and Kenichi Matsumoto. 2020. Predicting defective lines using a model-agnostic technique. arXiv preprint arXiv:2009.03612 (2020).","journal-title":"arXiv preprint arXiv:2009.03612"},{"key":"e_1_3_2_87_2","first-page":"150","volume-title":"Proceedings of the 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE\u201916)","author":"Wen Wei","year":"2016","unstructured":"Wei Wen, Tingting Yu, and Jane Huffman Hayes. 2016. CoLUA: Automatically predicting configuration bug reports and extracting configuration options. In Proceedings of the 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE\u201916). IEEE, Los Alamitos, CA, 150\u2013161."},{"key":"e_1_3_2_88_2","first-page":"206","volume-title":"Proceedings of the 2015 Asia-Pacific Software Engineering Conference (APSEC\u201915)","author":"Xu Bowen","year":"2015","unstructured":"Bowen Xu, David Lo, Xin Xia, Ashish Sureka, and Shanping Li. 2015. EFSPredictor: Predicting configuration bugs with ensemble feature selection. In Proceedings of the 2015 Asia-Pacific Software Engineering Conference (APSEC\u201915). IEEE, Los Alamitos, CA, 206\u2013213."},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786852"},{"issue":"7","key":"e_1_3_2_90_2","doi-asserted-by":"crossref","first-page":"1377","DOI":"10.1587\/transinf.2015EDL8049","article-title":"Software maintenance evaluation of agile software development method based on OpenStack","volume":"98","author":"Yamato Yoji","year":"2015","unstructured":"Yoji Yamato, Shinichiro Katsuragi, Shinji Nagao, and Norihiro Miura. 2015. Software maintenance evaluation of agile software development method based on OpenStack. IEICE Transactions on Information and Systems 98, 7 (2015), 1377\u20131380.","journal-title":"IEICE Transactions on Information and Systems"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2599161"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-804206-9.00027-1"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3607186","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3607186","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:34Z","timestamp":1750178254000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3607186"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,24]]},"references-count":91,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1,31]]}},"alternative-id":["10.1145\/3607186"],"URL":"https:\/\/doi.org\/10.1145\/3607186","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,24]]},"assertion":[{"value":"2022-11-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-07","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}