{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T09:03:58Z","timestamp":1774602238829,"version":"3.50.1"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2017,7,31]],"date-time":"2017-07-31T00:00:00Z","timestamp":1501459200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2017,7,31]]},"abstract":"<jats:p>Variability-sensitive verification pursues effective analysis of the exponentially many variants of a program family. Several variability-aware techniques have been proposed, but researchers still lack examples of concrete bugs induced by variability, occurring in real large-scale systems. A collection of real world bugs is needed to evaluate tool implementations of variability-sensitive analyses by testing them on real bugs. We present a qualitative study of 98 diverse variability bugs (i.e., bugs that occur in some variants and not in others) collected from bug-fixing commits in the Linux, Apache, BusyBox, and Marlin repositories. We analyze each of the bugs, and record the results in a database. For each bug, we create a self-contained simplified version and a simplified patch, in order to help researchers who are not experts on these subject studies to understand them, so that they can use these bugs for evaluation of their tools. In addition, we provide single-function versions of the bugs, which are useful for evaluating intra-procedural analyses. A web-based user interface for the database allows to conveniently browse and visualize the collection of bugs. Our study provides insights into the nature and occurrence of variability bugs in four highly-configurable systems implemented in C\/C++, and shows in what ways variability hinders comprehension and the uncovering of software bugs.<\/jats:p>","DOI":"10.1145\/3149119","type":"journal-article","created":{"date-parts":[[2018,1,12]],"date-time":"2018-01-12T13:49:50Z","timestamp":1515764990000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":54,"title":["Variability Bugs in Highly Configurable Systems"],"prefix":"10.1145","volume":"26","author":[{"given":"Iago","family":"Abal","sequence":"first","affiliation":[{"name":"IT University of Copenhagen, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jean","family":"Melo","sequence":"additional","affiliation":[{"name":"IT University of Copenhagen, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"\u015etefan","family":"St\u0103nciulescu","sequence":"additional","affiliation":[{"name":"IT University of Copenhagen, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Claus","family":"Brabrand","sequence":"additional","affiliation":[{"name":"IT University of Copenhagen, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"M\u00e1rcio","family":"Ribeiro","sequence":"additional","affiliation":[{"name":"Federal University of Alagoas, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrzej","family":"W\u0105sowski","sequence":"additional","affiliation":[{"name":"IT University of Copenhagen, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,1,12]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2642937.2642990"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2866614.2866626"},{"key":"e_1_2_1_3_1","volume-title":"Feature-Oriented Software Product Lines","author":"Apel Sven","unstructured":"Sven Apel , Don Batory , Christian K\u00e4stner , and Gunter Saake . 2013. Feature-Oriented Software Product Lines . New York : Springer-Verlag . Sven Apel, Don Batory, Christian K\u00e4stner, and Gunter Saake. 2013. Feature-Oriented Software Product Lines. New York: Springer-Verlag."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-010-0066-8"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2011.6100075"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2010.01.001"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2430502.2430513"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2013.34"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491956.2491976"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/2486788.2486805"},{"key":"e_1_2_1_11_1","unstructured":"D. P. Bovet and M. Cesati. 2005. Understanding the Linux Kernel. O\u2019Reilly Media Sebastopol CA.   D. P. Bovet and M. Cesati. 2005. Understanding the Linux Kernel. O\u2019Reilly Media Sebastopol CA."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/2554488.2554491"},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the International Conference on Software Testing Analysis 8 Review.","author":"Burr K.","unstructured":"K. Burr and W. Young . 1998. Combinatorial test tech-niques: Table-based automation, test generation and code coverage . In Proceedings of the International Conference on Software Testing Analysis 8 Review. K. Burr and W. Young. 1998. Combinatorial test tech-niques: Table-based automation, test generation and code coverage. In Proceedings of the International Conference on Software Testing Analysis 8 Review."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-024X(200006)30:7%3C775::AID-SPE309%3E3.0.CO;2-H"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1389-1286(02)00352-3"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985838"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806850"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.605761"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1173706.1173738"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/302405.302640"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-005-3861-2"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/780822.781149"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1158288"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/249069.231389"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-013-9208-0"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-68863-1_8"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1944892.1944895"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1251535.1251537"},{"key":"e_1_2_1_29_1","volume-title":"Effective analysis of C programs by rewriting variability. CoRR abs\/1701.08114","author":"Iosif-Lazar Alexandru Florin","year":"2017","unstructured":"Alexandru Florin Iosif-Lazar , Jean Melo , Aleksandar S. Dimovski , Claus Brabrand , and Andrzej Wasowski . 2017. Effective analysis of C programs by rewriting variability. CoRR abs\/1701.08114 ( 2017 ). http:\/\/arxiv.org\/abs\/1701.08114. Alexandru Florin Iosif-Lazar, Jean Melo, Aleksandar S. Dimovski, Claus Brabrand, and Andrzej Wasowski. 2017. Effective analysis of C programs by rewriting variability. CoRR abs\/1701.08114 (2017). http:\/\/arxiv.org\/abs\/1701.08114."},{"key":"e_1_2_1_30_1","doi-asserted-by":"crossref","unstructured":"Kyo Kang Sholom Cohen James Hess William Nowak and Spencer Peterson. 1990. Feature-Oriented Domain Analysis (FODA) Feasibility Study. Tech.\u00a0Rep. CMU\/SEI-90-TR-21. CMU-SEI.  Kyo Kang Sholom Cohen James Hess William Nowak and Spencer Peterson. 1990. Feature-Oriented Domain Analysis (FODA) Feasibility Study. Tech.\u00a0Rep. CMU\/SEI-90-TR-21. CMU-SEI.","DOI":"10.21236\/ADA235785"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2008.36"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1368088.1368131"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1868688.1868693"},{"key":"e_1_2_1_35_1","series-title":"Lecture Notes in Computer Science","volume-title":"1st International Conference on Runtime Verification (RV)","author":"Peter Kim Chang Hwan","unstructured":"Chang Hwan Peter Kim , Eric Bodden , Don Batory , and Sarfraz Khurshid . 2010. Reducing configurations to monitor in a software product line . In 1st International Conference on Runtime Verification (RV) , Lecture Notes in Computer Science , Vol. 6418 . Springer , Berlin . Chang Hwan Peter Kim, Eric Bodden, Don Batory, and Sarfraz Khurshid. 2010. Reducing configurations to monitor in a software product line. In 1st International Conference on Runtime Verification (RV), Lecture Notes in Computer Science, Vol. 6418. Springer, Berlin."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2009.16"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806819"},{"key":"e_1_2_1_38_1","unstructured":"R. Love. 2010. Linux Kernel Development. Pearson Education.   R. Love. 2010. Linux Kernel Development. Pearson Education."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884793"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2517208.2517221"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884831"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2017.34"},{"key":"e_1_2_1_43_1","volume-title":"13th International Conference on Software Product Lines, SPLC","author":"Mendon\u00e7a Marc\u00edlio","year":"2009","unstructured":"Marc\u00edlio Mendon\u00e7a , Andrzej Wasowski , and Krzysztof Czarnecki . 2009. SAT-based analysis of feature models is easy . In 13th International Conference on Software Product Lines, SPLC 2009 , San Francisco, California, USA , August 24-28, 2009, Proceedings (ACM International Conference Proceeding Series), Dirk Muthig and John D. McGregor (eds.), Vol. 446 . ACM , 231--240. Marc\u00edlio Mendon\u00e7a, Andrzej Wasowski, and Krzysztof Czarnecki. 2009. SAT-based analysis of feature models is easy. In 13th International Conference on Software Product Lines, SPLC 2009, San Francisco, California, USA, August 24-28, 2009, Proceedings (ACM International Conference Proceeding Series), Dirk Muthig and John D. McGregor (eds.), Vol. 446. ACM, 231--240."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568283"},{"key":"e_1_2_1_45_1","volume-title":"Thomas Zimmermann","author":"Nadi Sarah","unstructured":"Sarah Nadi , Christian Dietrich , Reinhard Tartler , Richard C. Holt , and Daniel Lohmann . 2013. Linux variability anomalies: What causes them and how do they get fixed? In MSR , Thomas Zimmermann , Massimiliano Di Penta, and Sunghun Kim (eds.). IEEE\/ACM. Sarah Nadi, Christian Dietrich, Reinhard Tartler, Richard C. Holt, and Daniel Lohmann. 2013. Linux variability anomalies: What causes them and how do they get fixed? In MSR, Thomas Zimmermann, Massimiliano Di Penta, and Sunghun Kim (eds.). IEEE\/ACM."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1217935.1217942"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1961295.1950401"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9364-x"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2008.45"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-35873-9_17"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2015.7332461"},{"key":"e_1_2_1_52_1","volume-title":"Finding and burying Configuration Defects in Linux with the undertaker. Retrieved","author":"Tartler Reinhard","year":"2017","unstructured":"Reinhard Tartler . 2011. Finding and burying Configuration Defects in Linux with the undertaker. Retrieved November 8, 2017 from http:\/\/www4.informatik.uni-erlangen.de\/Publications\/2011\/plumbers-presentation-tartler.pdf. Reinhard Tartler. 2011. Finding and burying Configuration Defects in Linux with the undertaker. Retrieved November 8, 2017 from http:\/\/www4.informatik.uni-erlangen.de\/Publications\/2011\/plumbers-presentation-tartler.pdf."},{"key":"e_1_2_1_53_1","volume-title":"2014 USENIX Annual Technical Conference, USENIX ATC\u201914","author":"Tartler Reinhard","year":"2014","unstructured":"Reinhard Tartler , Christian Dietrich , Julio Sincero , Wolfgang Schr\u00f6der-Preikschat , and Daniel Lohmann . 2014 . Static analysis of variability in system software: The 90,000 #ifdefs issue . In 2014 USENIX Annual Technical Conference, USENIX ATC\u201914 , Philadelphia, PA, USA , June 19-20, 2014, Garth Gibson and Nickolai Zeldovich (eds.). USENIX Association, 421--432. https:\/\/www.usenix.org\/conference\/atc14\/technical-sessions\/presentation\/tartler. Reinhard Tartler, Christian Dietrich, Julio Sincero, Wolfgang Schr\u00f6der-Preikschat, and Daniel Lohmann. 2014. Static analysis of variability in system software: The 90,000 #ifdefs issue. In 2014 USENIX Annual Technical Conference, USENIX ATC\u201914, Philadelphia, PA, USA, June 19-20, 2014, Garth Gibson and Nickolai Zeldovich (eds.). USENIX Association, 421--432. https:\/\/www.usenix.org\/conference\/atc14\/technical-sessions\/presentation\/tartler."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2094091.2094095"},{"key":"e_1_2_1_55_1","volume-title":"IEEE Standard Glossary of Software Engineering Terminology","author":"The Institute of Electrical and Electronics Engineers. 1990.","unstructured":"The Institute of Electrical and Electronics Engineers. 1990. IEEE Standard Glossary of Software Engineering Terminology . IEEE Standard . DOI:http:\/\/ieeexplore.ieee.org\/document\/159342. The Institute of Electrical and Electronics Engineers. 1990. IEEE Standard Glossary of Software Engineering Terminology. IEEE Standard. DOI:http:\/\/ieeexplore.ieee.org\/document\/159342."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2580950"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.5555\/2337223.2337269"},{"key":"e_1_2_1_58_1","volume-title":"A first step towards automated detection of buffer overrun vulnerabilities","author":"Wagner David","unstructured":"David Wagner , Jeffrey S. Foster , Eric A. Brewer , and Alexander Aiken . 2000. A first step towards automated detection of buffer overrun vulnerabilities . In NDSS. The Internet Society . David Wagner, Jeffrey S. Foster, Eric A. Brewer, and Alexander Aiken. 2000. A first step towards automated detection of buffer overrun vulnerabilities. In NDSS. The Internet Society."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043556.2043572"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3149119","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3149119","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T21:37:02Z","timestamp":1750282622000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3149119"}},"subtitle":["A Qualitative Analysis"],"short-title":[],"issued":{"date-parts":[[2017,7,31]]},"references-count":58,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2017,7,31]]}},"alternative-id":["10.1145\/3149119"],"URL":"https:\/\/doi.org\/10.1145\/3149119","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,7,31]]},"assertion":[{"value":"2016-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-01-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}