{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T11:48:58Z","timestamp":1771588138444,"version":"3.50.1"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2008,10,31]],"date-time":"2008-10-31T00:00:00Z","timestamp":1225411200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGSOFT Softw. Eng. Notes"],"published-print":{"date-parts":[[2008,10,31]]},"abstract":"<jats:p>Software inspections, reviews, and walkthroughs have become a standard process component in many software development domains. Maturity level 3 of the CMM-I requires establishment of peer reviews [12] and substantial sustained improvements in quality and productivity have been reported as a result of using reviews ([16], [21], [22], [27]). The NSF Impact project identifies the degree to which these industrial success cases have been instigated and improved by research in software engineering.<\/jats:p>\n          <jats:p>This research identifies that there is widespread adoption of inspections, reviews or walkthroughs but that companies do not generally exploit their full potential. However there exist sustained industrial success cases with respect to the wide-spread and measurably successful application of them. It also identifies research in software engineering that can be credibly documented as having influenced the industrial success cases. Credible documentation may exist in the form of publications or documented reports by witnesses. Due to the semi-formal nature of inspections, reviews, and walkthroughs, a specific focus is given to empirical research results as motivators for adoption. Through the examination of one detailed case study, it is shown that software engineering research has had a significant impact on practice and that the impact can be traced in this case from research to that practice. The case study chosen provides evidence of both success and failure regarding sustained application in practice.<\/jats:p>\n          <jats:p>Thus the analysis of historic impact chains of research reveals a clear impact of software engineering research on sustained industrial success for inspections, reviews and walkthroughs. More importantly, in impact chains where the empirical results have not been established, we conclude that success has not been achieved or has not been sustained.<\/jats:p>\n          <jats:p>The paper closes with (1) lessons learned for creating the sustained use and impact of semi-formal software engineering processes, (2) a request for researchers and practitioners to further consider how their work can improve the effectiveness of research and practice, and (3) a request to contribute additional success cases and impact factors to the authors database for future enhancements of this paper.<\/jats:p>","DOI":"10.1145\/1449603.1449609","type":"journal-article","created":{"date-parts":[[2008,12,4]],"date-time":"2008-12-04T17:19:26Z","timestamp":1228411166000},"page":"26-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":26,"title":["Impact of research on practice in the field of inspections, reviews and walkthroughs"],"prefix":"10.1145","volume":"33","author":[{"given":"Dieter","family":"Rombach","sequence":"first","affiliation":[{"name":"University of Kaiserslautern, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marcus","family":"Ciolkowski","sequence":"additional","affiliation":[{"name":"University of Kaiserslautern, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ross","family":"Jeffery","sequence":"additional","affiliation":[{"name":"University of Kaiserslautern, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Oliver","family":"Laitenberger","sequence":"additional","affiliation":[{"name":"University of Kaiserslautern, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Frank","family":"McGarry","sequence":"additional","affiliation":[{"name":"University of Kaiserslautern, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Forrest","family":"Shull","sequence":"additional","affiliation":[{"name":"University of Kaiserslautern, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2008,10,31]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.243"},{"key":"e_1_2_1_2_1","volume-title":"Extreme Programming Explained: Embrace Change","author":"Beck","year":"1999","unstructured":"Beck : Extreme Programming Explained: Embrace Change . Addison Wesley 1999 . Beck: Extreme Programming Explained: Embrace Change. Addison Wesley 1999."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(97)00065-4"},{"key":"e_1_2_1_4_1","first-page":"528","volume-title":"Goal Question Metric Paradigm,\" in Encyclopedia of Software Engineering","author":"Basili V.R.","year":"1994","unstructured":"Basili , V.R. , Caldiera , G. , and Rombach , H.D. , \" Goal Question Metric Paradigm,\" in Encyclopedia of Software Engineering , vol. 1 , J.J. Marciniak , Ed ., 1994 , pp. 528 -- 532 . Basili, V.R., Caldiera, G., and Rombach, H.D., \"Goal Question Metric Paradigm,\" in Encyclopedia of Software Engineering, vol. 1, J.J. Marciniak, Ed., 1994, pp. 528--532."},{"issue":"1","key":"e_1_2_1_5_1","first-page":"133164","article-title":"1996","volume":"2","author":"Basili V.","unstructured":"Basili , V. , Green , S. , Laitenberger , O. , Lanubile , F. , Shull , F. , Sorumgard , S. , and Zelkowitz , M ., 1996 . The Empirical Investigation of Perspective-based Reading. Journal of Empirical Software Engineering , 2 ( 1 ): 133164 . Basili, V., Green, S., Laitenberger, O., Lanubile, F., Shull, F., Sorumgard, S., and Zelkowitz, M., 1996. The Empirical Investigation of Perspective-based Reading. Journal of Empirical Software Engineering, 2(1):133164.","journal-title":"Journal of Empirical Software Engineering"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/581339.581351"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1987.232881"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.962984"},{"key":"e_1_2_1_9_1","first-page":"1","volume-title":"Proc. 2nd Int'l Software Quality Week Europe, Software Research Inc.","author":"Briand","year":"1998","unstructured":"L. Briand et al., Quality Assurance Technologies for the EURO Conversion--Industrial Experience at Allianz Life Assurance , Proc. 2nd Int'l Software Quality Week Europe, Software Research Inc. , 1998 , pp. 1 -- 23 . L. Briand et al., Quality Assurance Technologies for the EURO Conversion--Industrial Experience at Allianz Life Assurance, Proc. 2nd Int'l Software Quality Week Europe, Software Research Inc., 1998, pp. 1--23."},{"key":"e_1_2_1_10_1","first-page":"104","volume-title":"LNCS","author":"Ciolkowski M.","year":"2003","unstructured":"Ciolkowski , M. , Shull , F. , and Biffl , S .: Practical Experiences in the Design and Conduct of Surveys in Empirical Software Engineering , ESERNET Method and Experience, R. Conradi, ed., LNCS , Springer-Verlag , 2003 , Volume 2765 \/ 2003, August 2003, pp. 104 -- 128 . Ciolkowski, M., Shull, F., and Biffl, S.: Practical Experiences in the Design and Conduct of Surveys in Empirical Software Engineering, ESERNET Method and Experience, R. Conradi, ed., LNCS, Springer-Verlag, 2003, Volume 2765 \/ 2003, August 2003, pp. 104--128."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2003.1241366"},{"key":"e_1_2_1_12_1","unstructured":"CMMI Product Team Software Engineering Institute Capability Maturity Model\u00ae Integration (CMMISM) Version 1.1 CMU\/SEI-2002-TR-012.  CMMI Product Team Software Engineering Institute Capability Maturity Model\u00ae Integration (CMMISM) Version 1.1 CMU\/SEI-2002-TR-012."},{"key":"e_1_2_1_13_1","volume-title":"Out of the crisis","author":"Deming W.","year":"1982","unstructured":"Deming , W. : Out of the crisis , SPC Press , 1982 . Deming, W.: Out of the crisis, SPC Press, 1982."},{"key":"e_1_2_1_14_1","volume-title":"Edison-Wesley","author":"Endres A.","year":"2003","unstructured":"Endres , A. , Rombach , H.D. , A Handbook of Software and Systems Engineering - Empirical Observations, Laws and Theories , Edison-Wesley , 2003 . Endres, A., Rombach, H.D., A Handbook of Software and Systems Engineering - Empirical Observations, Laws and Theories, Edison-Wesley, 2003."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1101815.1101817"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1147\/sj.153.0182"},{"key":"e_1_2_1_17_1","volume-title":"Software Inspection","author":"Gilb T.","year":"1993","unstructured":"Gilb , T. and Graham , D. , Software Inspection , Addison-Wesley Publishing Company , 1993 . Gilb, T. and Graham, D., Software Inspection, Addison-Wesley Publishing Company, 1993."},{"key":"e_1_2_1_19_1","volume-title":"Standard for Software Reviews","author":"Std","year":"1998","unstructured":"IEEE Std . 1028-1997 ( 1998 ): Standard for Software Reviews , The Institute of Electrical and Electronics Engineering, Inc. ISBN 1-55937-987-1 IEEE Std. 1028-1997 (1998): Standard for Software Reviews, The Institute of Electrical and Electronics Engineering, Inc. ISBN 1-55937-987-1"},{"key":"e_1_2_1_20_1","unstructured":"Kohli Robert O. and Radice R.A. Low Level Design Inspection Specification TR 21.629 IBM System Communications Division Kingston NY 12401.  Kohli Robert O. and Radice R.A. Low Level Design Inspection Specification TR 21.629 IBM System Communications Division Kingston NY 12401."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(99)00073-4"},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the 6th International Software Metrics Symposium.","author":"Laitenberger O.","unstructured":"Laitenberger , O. , Leszak , M. , Stoll , D. , El Emam , K. , Quantitative Modeling of Software Reviews in an Industrial Setting , in Proceedings of the 6th International Software Metrics Symposium. Laitenberger, O., Leszak, M., Stoll, D., El Emam, K., Quantitative Modeling of Software Reviews in an Industrial Setting, in Proceedings of the 6th International Software Metrics Symposium."},{"key":"e_1_2_1_23_1","volume-title":"VISEK technical report 011.02","author":"Laitenberger S.","year":"2002","unstructured":"O. Laitenberger , S. Vegas , and M. Ciolkowski , The State of the Practice of Review and Review Technologies in Germany , VISEK technical report 011.02 , 2002 . O. Laitenberger, S. Vegas, and M. Ciolkowski, The State of the Practice of Review and Review Technologies in Germany, VISEK technical report 011.02, 2002."},{"key":"e_1_2_1_24_1","volume-title":"Structured Programming: Theory and Practice","author":"Linger R.C.","year":"1979","unstructured":"Linger , R.C. , Mills , H.D. , and Witt , B.I. , 1979 . Structured Programming: Theory and Practice . Addison-Wesley Publishing Company . Linger, R.C., Mills, H.D., and Witt, B.I., 1979. Structured Programming: Theory and Practice. Addison-Wesley Publishing Company."},{"key":"e_1_2_1_25_1","volume-title":"Repeatable Software Engineering Experiments for Comparing Defect-Detection Techniques","author":"Lott C.M.","year":"1996","unstructured":"Lott , C.M. and Rombach , H.D. , \" Repeatable Software Engineering Experiments for Comparing Defect-Detection Techniques ,\" 1996 . Lott, C.M. and Rombach, H.D., \"Repeatable Software Engineering Experiments for Comparing Defect-Detection Techniques,\" 1996."},{"key":"e_1_2_1_26_1","first-page":"10841090","volume-title":"Encyclopedia of Software Engineering","author":"Marciniak J.J.","unstructured":"Marciniak , J.J. , 1994. Reviews and Audits . In: Marciniak, J.J., editor, Encyclopedia of Software Engineering , volume 2 , pages 10841090 . John Wiley and Sons. Marciniak, J.J., 1994. Reviews and Audits. In: Marciniak, J.J., editor, Encyclopedia of Software Engineering, volume 2, pages 10841090. John Wiley and Sons."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2002.1049397"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.948"},{"key":"e_1_2_1_29_1","volume-title":"American Programmer, Pages 31--37","author":"Mills H","year":"1991","unstructured":"Mills , H : \" Cleanroom Engineering \", American Programmer, Pages 31--37 , May 1991 . Mills, H: \"Cleanroom Engineering\", American Programmer, Pages 31--37, May 1991."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/359588.359602"},{"key":"e_1_2_1_31_1","unstructured":"National Institute of Standards and Technology The Economic Impact of Inadequate Infrastructure for Software Testing http:\/\/www.nist.gov\/public_affairs\/releases\/n02-10.htm May 2002  National Institute of Standards and Technology The Economic Impact of Inadequate Infrastructure for Software Testing http:\/\/www.nist.gov\/public_affairs\/releases\/n02-10.htm May 2002"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/357474.355060"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1101815.1101818"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1987.233525"},{"key":"e_1_2_1_35_1","volume-title":"The Goal\/Question\/Metric Method. A Practical Guide for Quality Improvement of Software Development","author":"Solingen R.","year":"1999","unstructured":"Solingen , R. van, Berghout , E. , The Goal\/Question\/Metric Method. A Practical Guide for Quality Improvement of Software Development , London : McGraw-Hill , 1999 , ISBN: 0-07-709553-7. Solingen, R. van, Berghout, E., The Goal\/Question\/Metric Method. A Practical Guide for Quality Improvement of Software Development, London: McGraw-Hill, 1999, ISBN: 0-07-709553-7."},{"key":"e_1_2_1_36_1","volume-title":"Software Inspection Process","author":"Strauss S.H.","unstructured":"Strauss , S.H. and Ebenau , R.G .: Software Inspection Process . McGraw Hill Systems Design & amp; Implementation Series. Strauss, S.H. and Ebenau, R.G.: Software Inspection Process. McGraw Hill Systems Design &amp; Implementation Series."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/167049.167070"},{"issue":"1","key":"e_1_2_1_38_1","first-page":"6872","article-title":"1984. Reviews, Walkthroughs, and Inspections","volume":"12","author":"Weinberg G.M.","unstructured":"Weinberg , G.M. and Freedman , D.P ., 1984. Reviews, Walkthroughs, and Inspections . IEEE Transactions on Software Engineering , 12 ( 1 ): 6872 . Weinberg, G.M. and Freedman, D.P., 1984. Reviews, Walkthroughs, and Inspections. IEEE Transactions on Software Engineering, 12(1):6872.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_2_1_39_1","volume-title":"Peer Reviews in Software--A Practical Guide","author":"Wiegers K.","year":"2002","unstructured":"Wiegers , K. : Peer Reviews in Software--A Practical Guide , Addison-Wesley , 2002 . Wiegers, K.: Peer Reviews in Software--A Practical Guide, Addison-Wesley, 2002."},{"key":"e_1_2_1_40_1","volume-title":"Structured Walkthroughs","author":"Yourdon E.","year":"1989","unstructured":"Yourdon , E. : Structured Walkthroughs , N.Y. : Prentice Hall , 4 th edition, 1989 . Yourdon, E.: Structured Walkthroughs, N.Y.: Prentice Hall, 4th edition, 1989.","edition":"4"}],"container-title":["ACM SIGSOFT Software Engineering Notes"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1449603.1449609","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1449603.1449609","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T20:22:28Z","timestamp":1750278148000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1449603.1449609"}},"subtitle":["learning from successful industrial uses"],"short-title":[],"issued":{"date-parts":[[2008,10,31]]},"references-count":39,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2008,10,31]]}},"alternative-id":["10.1145\/1449603.1449609"],"URL":"https:\/\/doi.org\/10.1145\/1449603.1449609","relation":{},"ISSN":["0163-5948"],"issn-type":[{"value":"0163-5948","type":"print"}],"subject":[],"published":{"date-parts":[[2008,10,31]]},"assertion":[{"value":"2008-10-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}