{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,1]],"date-time":"2026-02-01T20:10:35Z","timestamp":1769976635919,"version":"3.49.0"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2013,10,1]],"date-time":"2013-10-01T00:00:00Z","timestamp":1380585600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/D052785\/1 and EP\/J017515\/1"],"award-info":[{"award-number":["EP\/D052785\/1 and EP\/J017515\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2013,10]]},"abstract":"<jats:p>Ensembles of learning machines are promising for software effort estimation (SEE), but need to be tailored for this task to have their potential exploited. A key issue when creating ensembles is to produce diverse and accurate base models. Depending on how differently different performance measures behave for SEE, they could be used as a natural way of creating SEE ensembles. We propose to view SEE model creation as a multiobjective learning problem. A multiobjective evolutionary algorithm (MOEA) is used to better understand the tradeoff among different performance measures by creating SEE models through the simultaneous optimisation of these measures. We show that the performance measures behave very differently, presenting sometimes even opposite trends. They are then used as a source of diversity for creating SEE ensembles. A good tradeoff among different measures can be obtained by using an ensemble of MOEA solutions. This ensemble performs similarly or better than a model that does not consider these measures explicitly. Besides, MOEA is also flexible, allowing emphasis of a particular measure if desired. In conclusion, MOEA can be used to better understand the relationship among performance measures and has shown to be very effective in creating SEE models.<\/jats:p>","DOI":"10.1145\/2522920.2522928","type":"journal-article","created":{"date-parts":[[2013,10,17]],"date-time":"2013-10-17T12:23:34Z","timestamp":1382012614000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":75,"title":["Software effort estimation as a multiobjective learning problem"],"prefix":"10.1145","volume":"22","author":[{"given":"Leandro L.","family":"Minku","sequence":"first","affiliation":[{"name":"The University of Birmingham, UK"}]},{"given":"Xin","family":"Yao","sequence":"additional","affiliation":[{"name":"The University of Birmingham, UK"}]}],"member":"320","published-online":{"date-parts":[[2013,10,22]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/505482.505491"},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of ISCIS'07","author":"Baskeles B."},{"key":"e_1_2_1_3_1","unstructured":"Bishop C. M. 2005. Neural Networks for Pattern Recognition. Oxford University Press UK.  Bishop C. M. 2005. Neural Networks for Pattern Recognition. Oxford University Press UK."},{"key":"e_1_2_1_4_1","unstructured":"Boehm B. 1981. Software Engineering Economics. Prentice-Hall Englewood Cliffs NJ.   Boehm B. 1981. Software Engineering Economics. Prentice-Hall Englewood Cliffs NJ."},{"key":"e_1_2_1_5_1","unstructured":"Boehm B. Abts C. Brown A. W. Chulani S. Clark B. K. Horowitz E. Madachy R. Reifer D. J. and Steece B. 2000. Software Cost Estimation with COCOMO II. Prentice-Hall Englewood Cliffs NJ.   Boehm B. Abts C. Brown A. W. Chulani S. Clark B. K. Horowitz E. Madachy R. Reifer D. J. and Steece B. 2000. Software Cost Estimation with COCOMO II. Prentice-Hall Englewood Cliffs NJ."},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of IJCNN'07","author":"Braga P. L."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1018054314350"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2004.04.004"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of METRICS'03","author":"Cartwright M. H."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10852-005-9020-3"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2009.2034144"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.799958"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.112.1.155"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/4235.996017"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2011.55"},{"key":"e_1_2_1_16_1","first-page":"130","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","author":"Dem\u0161ar J.","year":"2006","journal-title":"J. Mach. Learn. Rese."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.879821"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(00)00137-3"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80122-4"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2003.1245300"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/13487689.13487693"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1656274.1656278"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/1018439.1021893"},{"key":"e_1_2_1_24_1","unstructured":"Hartigan J. A. 1975. Clustering Algorithms. John Wiley & Sons New York.   Hartigan J. A. 1975. Clustering Algorithms. John Wiley & Sons New York."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(02)00128-3"},{"key":"e_1_2_1_26_1","unstructured":"ISBSG. 2011. The International Software Benchmarking Standards Group. http:\/\/www.isbsg.org.  ISBSG. 2011. The International Software Benchmarking Standards Group. http:\/\/www.isbsg.org."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.3"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.78"},{"key":"e_1_2_1_29_1","doi-asserted-by":"crossref","unstructured":"Khare V. Yao X. and \n      \n      \n      Deb K\n      \n  \n  . \n  2003\n  . Performance scaling of multi-objective evolutionary algorithms. In Proceedings of the 2nd International Conference on Evolutionary Multi-Criterion Optimization (EMO'03) C. M. Fonseca P. J. Fleming E. Zitzler K. Deb and L. Thiele Eds. Lecture Notes in Computer Science vol. \n  2632\n  . \n  Springer-Verlag 376--390.   Khare V. Yao X. and Deb K. 2003. Performance scaling of multi-objective evolutionary algorithms. In Proceedings of the 2nd International Conference on Evolutionary Multi-Criterion Optimization (EMO'03) C. M. Fonseca P. J. Fleming E. Zitzler K. Deb and L. Thiele Eds. Lecture Notes in Computer Science vol. 2632. Springer-Verlag 376--390.","DOI":"10.1007\/3-540-36970-8_27"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of ISSRE'07","author":"Kocaguneli E."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2011.111"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2009.05.001"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022859003006"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the Congress of Evolutionary Computation (CEC). 2144--2151","author":"Legg S."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(99)00073-8"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/3477.809027"},{"key":"e_1_2_1_37_1","doi-asserted-by":"crossref","unstructured":"Lukasiewycz M. Gla M. Reimann F. and Helwig S. 2011. Opt4j: The meta-heuristic optimisation framework for java. http:\/\/opt4j.sourceforge.net.  Lukasiewycz M. Gla M. Reimann F. and Helwig S. 2011. Opt4j: The meta-heuristic optimisation framework for java. http:\/\/opt4j.sourceforge.net.","DOI":"10.1145\/2001576.2001808"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2006.114"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-011-9193-5"},{"key":"e_1_2_1_40_1","first-page":"193","article-title":"Genetic algorithms, tournament selection, and the effects of noise","volume":"9","author":"Miller B. L.","year":"1995","journal-title":"Complex Syst."},{"key":"e_1_2_1_41_1","unstructured":"Minku L. L. 2011. Machine learning for software effort estimation. The 13th CREST Open Workshop Future Internet Testing (FITTEST) & Search Based Software Engineering (SBSE) http:\/\/crest.cs.ucl.ac.uk\/cow\/13\/slides\/presentation_leandro.pdf http:\/\/crest.cs.ucl.ac.uk\/cow\/13\/videos\/M2U00270Minku.mp4.  Minku L. L. 2011. Machine learning for software effort estimation. The 13th CREST Open Workshop Future Internet Testing (FITTEST) & Search Based Software Engineering (SBSE) http:\/\/crest.cs.ucl.ac.uk\/cow\/13\/slides\/presentation_leandro.pdf http:\/\/crest.cs.ucl.ac.uk\/cow\/13\/videos\/M2U00270Minku.mp4."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2020390.2020399"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2012.09.012"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/1062455.1062516"},{"key":"e_1_2_1_45_1","unstructured":"Montgomery D. C. 2004. Design and Analysis of Experiments 6th Ed. John Wiley and Sons.   Montgomery D. C. 2004. Design and Analysis of Experiments 6th Ed. John Wiley and Sons."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.26"},{"key":"e_1_2_1_47_1","volume-title":"Proceedings of the International Conference on Computational Intelligence and Security (CIS'06)","volume":"1","author":"Praditwong K."},{"key":"e_1_2_1_48_1","unstructured":"Rosenthal R. 1994. The Handbook of Research Synthesis. Vol. 236 Sage New York.  Rosenthal R. 1994. The Handbook of Research Synthesis. Vol. 236 Sage New York."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1370788.1370796"},{"key":"e_1_2_1_50_1","volume-title":"Proceedings of the ICCCAS & WESINO EXPO.","volume":"2","author":"Shan Y."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.637387"},{"key":"e_1_2_1_52_1","unstructured":"Shirabad J. S. and Menzies T. 2005. The PROMISE Repository of Software Engineering Databases. School of Information Technology and Engineering University of Ottawa Canada http:\/\/promise. site.uottawa.ca\/SERepository.  Shirabad J. S. and Menzies T. 2005. The PROMISE Repository of Software Engineering Databases. School of Information Technology and Engineering University of Ottawa Canada http:\/\/promise. site.uottawa.ca\/SERepository."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1162\/evco.1994.2.3.221"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.345828"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134331"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/1571629.1571630"},{"key":"e_1_2_1_57_1","volume-title":"Proceedings of IJCNN'07","author":"Tronto I. F. B.","year":"2007"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2010.2057310"},{"key":"e_1_2_1_59_1","first-page":"87","article-title":"Using artificial neural networks and function points to estimate 4GL software development effort","volume":"1","author":"Wittig G. E.","year":"1994","journal-title":"Austral. J. Info. Syst."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(97)00004-9"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.asr.2007.07.020"},{"key":"e_1_2_1_62_1","volume-title":"Proceedings of the Conference on Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems (EUROGEN'02)","author":"Zitzler E."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2522920.2522928","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2522920.2522928","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T07:34:53Z","timestamp":1750232093000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2522920.2522928"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,10]]},"references-count":62,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2013,10]]}},"alternative-id":["10.1145\/2522920.2522928"],"URL":"https:\/\/doi.org\/10.1145\/2522920.2522928","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,10]]},"assertion":[{"value":"2011-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-12-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-10-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}