{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T10:24:01Z","timestamp":1779359041295,"version":"3.51.4"},"reference-count":59,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2014,3,1]],"date-time":"2014-03-01T00:00:00Z","timestamp":1393632000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2014,3]]},"abstract":"<jats:p>\n            The nature of the requirements analysis problem, based as it is on uncertain and often inaccurate estimates of costs and effort, makes sensitivity analysis important. Sensitivity analysis allows the decision maker to identify those requirements and budgets that are particularly sensitive to misestimation. However, finding scalable sensitivity analysis techniques is not easy because the underlying optimization problem is NP-hard. This article introduces an approach to sensitivity analysis based on exact optimization. We implemented this approach as a tool, O\n            <jats:sc>ATSAC<\/jats:sc>\n            , which allowed us to experimentally evaluate the scalability and applicability of Requirements Sensitivity Analysis (RSA). Our results show that O\n            <jats:sc>ATSAC<\/jats:sc>\n            scales sufficiently well for practical applications in Requirements Sensitivity Analysis. We also show how the sensitivity analysis can yield insights into difficult and otherwise obscure interactions between budgets, requirements costs, and estimate inaccuracies using a real-world case study.\n          <\/jats:p>","DOI":"10.1145\/2537853","type":"journal-article","created":{"date-parts":[[2014,4,8]],"date-time":"2014-04-08T12:24:20Z","timestamp":1396959860000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":31,"title":["Exact scalable sensitivity analysis for the next release problem"],"prefix":"10.1145","volume":"23","author":[{"given":"Mark","family":"Harman","sequence":"first","affiliation":[{"name":"University College London"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jens","family":"Krinke","sequence":"additional","affiliation":[{"name":"University College London"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Inmaculada","family":"Medina-Bulo","sequence":"additional","affiliation":[{"name":"University of C\u00e1diz"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Francisco","family":"Palomo-Lozano","sequence":"additional","affiliation":[{"name":"University of C\u00e1diz"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jian","family":"Ren","sequence":"additional","affiliation":[{"name":"University College London"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shin","family":"Yoo","sequence":"additional","affiliation":[{"name":"University College London"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2014,4,4]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2006.12.020"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/IWSPM.2006.5"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10732-006-5224-6"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.5555\/1018439.1021904"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2005.79"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(01)00194-X"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2006.56"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcss.2004.04.004"},{"key":"e_1_2_1_9_1","volume-title":"MIP: Theory and practice\u2014closing the gap. Syst. Model. Optim.: Methods, Theory, Appl. 1","author":"Bixby R. E.","year":"2000","unstructured":"R. E. Bixby , M. Fenelon , and Z. Gu . 2000 . MIP: Theory and practice\u2014closing the gap. Syst. Model. Optim.: Methods, Theory, Appl. 1 , 1960, 1--31. R. E. Bixby, M. Fenelon, and Z. Gu. 2000. MIP: Theory and practice\u2014closing the gap. Syst. Model. Optim.: Methods, Theory, Appl. 1, 1960, 1--31."},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the 9th International Advanced Science and Technology. 247--252","author":"Chao C.","year":"1993","unstructured":"C. Chao , J. Komada , Q. Liu , M. Muteja , Y. Alsalqan , and C. Chang 1993 . An application of genetic algorithms to software project management . In Proceedings of the 9th International Advanced Science and Technology. 247--252 . C. Chao, J. Komada, Q. Liu, M. Muteja, Y. Alsalqan, and C. Chang 1993. An application of genetic algorithms to software project management. In Proceedings of the 9th International Advanced Science and Technology. 247--252."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/FOSE.2007.17"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2012.07.041"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/SSBSE.2010.25"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/REV.2006.2"},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the 14th Annual International Symposium of International Council on Systems Engineering (INCOSE'04)","author":"Feather M. S.","unstructured":"M. S. Feather , J. D. Kiper , and S. Kalafat . 2004. Combining heuristic search, visualization and data mining for exploration of system design space . In Proceedings of the 14th Annual International Symposium of International Council on Systems Engineering (INCOSE'04) . M. S. Feather, J. D. Kiper, and S. Kalafat. 2004. Combining heuristic search, visualization and data mining for exploration of system design space. In Proceedings of the 14th Annual International Symposium of International Council on Systems Engineering (INCOSE'04)."},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the 10th IEEE International Conference on Requirements Engineering (RE'02)","author":"Feather M. S.","unstructured":"M. S. Feather and T. Menzies . 2002. Converging on the optimal attainment of requirements . In Proceedings of the 10th IEEE International Conference on Requirements Engineering (RE'02) . IEEE, 263--270. M. S. Feather and T. Menzies. 2002. Converging on the optimal attainment of requirements. In Proceedings of the 10th IEEE International Conference on Requirements Engineering (RE'02). IEEE, 263--270."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2008.61"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2008.61"},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the 3rd International Symposium on Search Based Software Engineering (SSBSE'11)","author":"Freitas F. G.","unstructured":"F. G. Freitas and J. T. Souza . 2011. Ten years of search based software engineering: A bibliometric analysis . In Proceedings of the 3rd International Symposium on Search Based Software Engineering (SSBSE'11) . 18--32. F. G. Freitas and J. T. Souza. 2011. Ten years of search based software engineering: A bibliometric analysis. In Proceedings of the 3rd International Symposium on Search Based Software Engineering (SSBSE'11). 18--32."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1111\/0272-4332.00039"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/322077.322090"},{"key":"e_1_2_1_22_1","unstructured":"M. R. Garey and D. S. Johnson. 1979. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman.   M. R. Garey and D. S. Johnson. 1979. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2003.07.002"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00547132"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/FOSE.2007.29"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2372251.2372253"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1569901.1570126"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2379776.2379787"},{"key":"e_1_2_1_29_1","doi-asserted-by":"crossref","unstructured":"M.\n      Harman P.\n      Mcminn J.\n      Souza and \n      S.\n      Yoo\n  . \n  2012\n  c. Search based software engineering: Techniques taxonomy tutorial. In Empirical Software Engineering and Verification: (LASER'09--10) B. Meyer and M. Nordio Eds. Lecture Notes in Computer Science vol. \n  7007 Springer 1--59.   M. Harman P. Mcminn J. Souza and S. Yoo. 2012c. Search based software engineering: Techniques taxonomy tutorial. In Empirical Software Engineering and Verification: (LASER'09--10) B. Meyer and M. Nordio Eds. Lecture Notes in Computer Science vol. 7007 Springer 1--59.","DOI":"10.1007\/978-3-642-25231-0_1"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ress.2005.11.017"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1370788.1370807"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.v20:5"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(97)00053-0"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4684-2001-2_9"},{"key":"e_1_2_1_35_1","first-page":"942","article-title":"A sensitivity analysis of cross-country growth regressions","volume":"82","author":"Levine R.","year":"1992","unstructured":"R. Levine and D. Renelt . 1992 . A sensitivity analysis of cross-country growth regressions . Amer. Econ. Rev. 82 , 4, 942 -- 963 . R. Levine and D. Renelt. 1992. A sensitivity analysis of cross-country growth regressions. Amer. Econ. Rev. 82, 4, 942--963.","journal-title":"Amer. Econ. Rev."},{"key":"e_1_2_1_36_1","series-title":"Lecture Notes in Computer Science","volume-title":"Requirements Engineering: Foundation for Software Quality (RefsQ'07)","author":"Li C.","year":"2007","unstructured":"C. Li , M. van den Akker , S. Brinkkemper , and G. Diepen . 2007 . Integrated requirement selection and scheduling for the release planning of a software product. In Proceedings of the 13th International Working Conference on Requirements Engineering: Foundation for Software Quality (RefsQ'07) . Lecture Notes in Computer Science , vol. 4542 . Springer , 93--108. C. Li, M. van den Akker, S. Brinkkemper, and G. Diepen. 2007. Integrated requirement selection and scheduling for the release planning of a software product. In Proceedings of the 13th International Working Conference on Requirements Engineering: Foundation for Software Quality (RefsQ'07). Lecture Notes in Computer Science, vol. 4542. Springer, 93--108."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.15.9.494"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.2514\/2.2416"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2008.80"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cor.2004.03.002"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/RTAS.2005.10"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1083217.1083229"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218194003001378"},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the International Symposium on Empirical Software Engineering (ISESE'03)","author":"Ruhe G.","unstructured":"G. Ruhe and D. Greer . 2003. Quantitative studies in software release planning under risk and resource constraints . In Proceedings of the International Symposium on Empirical Software Engineering (ISESE'03) . IEEE, 262--270. G. Ruhe and D. Greer. 2003. Quantitative studies in software release planning under risk and resource constraints. In Proceedings of the International Symposium on Empirical Software Engineering (ISESE'03). IEEE, 262--270."},{"key":"e_1_2_1_45_1","first-page":"1","article-title":"Hybrid intelligence in software release planning","volume":"1","author":"Ruhe G.","year":"2004","unstructured":"G. Ruhe and A. Ngo-The . 2004 . Hybrid intelligence in software release planning . Int. J. Hybrid Intell. Syst. 1 , 1 -- 2 , 99--110. G. Ruhe and A. Ngo-The. 2004. Hybrid intelligence in software release planning. Int. J. Hybrid Intell. Syst. 1, 1--2, 99--110.","journal-title":"Int. J. Hybrid Intell. Syst."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1287624.1287641"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1287624.1287641"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.envsoft.2010.04.012"},{"key":"e_1_2_1_49_1","doi-asserted-by":"crossref","unstructured":"A. Saltelli M. Ratto T. Andres F. Campolongo J. Cariboni D. Gatelli M. Saisana and S. Tarantola. 2008. Global Sensitivity Analysis: The Primer. John Wiley & Sons Sussex.  A. Saltelli M. Ratto T. Andres F. Campolongo J. Cariboni D. Gatelli M. Saisana and S. Tarantola. 2008. Global Sensitivity Analysis: The Primer. John Wiley & Sons Sussex.","DOI":"10.1002\/9780470725184"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1214\/ss\/1009213004"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.atmosenv.2003.08.019"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/FOSE.2007.23"},{"key":"e_1_2_1_53_1","unstructured":"M. van den Akker S. Brinkkemper G. Diepen and J. Versendaal. 2004. Flexible release composition using integer linear programming. Tech. rep. UU-CS-2004-063 Institute of Information and Computing Sciences Utrecht University.  M. van den Akker S. Brinkkemper G. Diepen and J. Versendaal. 2004. Flexible release composition using integer linear programming. Tech. rep. UU-CS-2004-063 Institute of Information and Computing Sciences Utrecht University."},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the 11th International Workshop on Requirements Engineering for Software Quality (RefsQ'05)","author":"van den Akker M.","unstructured":"M. van den Akker , S. Brinkkemper , G. Diepen , and J. Versendaal . 2005. Flexible release planning using integer linear programming . In Proceedings of the 11th International Workshop on Requirements Engineering for Software Quality (RefsQ'05) . Essener Informatik Beitrage, 247--262. M. van den Akker, S. Brinkkemper, G. Diepen, and J. Versendaal. 2005. Flexible release planning using integer linear programming. In Proceedings of the 11th International Workshop on Requirements Engineering for Software Quality (RefsQ'05). Essener Informatik Beitrage, 247--262."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2007.15"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/PROMISE.2007.7"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-69062-7_8"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276958.1277179"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-005-4251-0"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2537853","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2537853","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T08:18:14Z","timestamp":1750234694000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2537853"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,3]]},"references-count":59,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2014,3]]}},"alternative-id":["10.1145\/2537853"],"URL":"https:\/\/doi.org\/10.1145\/2537853","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,3]]},"assertion":[{"value":"2012-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-04-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}