{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T19:42:19Z","timestamp":1774640539181,"version":"3.50.1"},"reference-count":26,"publisher":"World Scientific Pub Co Pte Ltd","issue":"09","funder":[{"DOI":"10.13039\/501100000923","name":"Australian Research Council","doi-asserted-by":"crossref","award":["LP160101691"],"award-info":[{"award-number":["LP160101691"]}],"id":[{"id":"10.13039\/501100000923","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61872167"],"award-info":[{"award-number":["61872167"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Soft. Eng. Knowl. Eng."],"published-print":{"date-parts":[[2021,9]]},"abstract":"<jats:p> This paper presents an automated, domain-independent, metamorphic testing platform called MTKeras. In this paper, we report on an investigation demonstrating the effectiveness and usability of MTKeras through five case studies in the four domains of image classification, sentiment analysis, search engines and database management systems. We also report on the effectiveness of combining metamorphic relation (input) patterns in individual metamorphic relations, enhancing the failure-finding abilities of the individual relations. The results of our experiments support combining patterns, and the use of MTKeras. The research reported in this paper shows the applicability of metamorphic relation patterns, and introduces a practical tool for the research community. <\/jats:p>","DOI":"10.1142\/s021819402150039x","type":"journal-article","created":{"date-parts":[[2021,10,3]],"date-time":"2021-10-03T11:39:21Z","timestamp":1633261161000},"page":"1235-1249","source":"Crossref","is-referenced-by-count":3,"title":["MTKeras: An Automated Metamorphic Testing Platform"],"prefix":"10.1142","volume":"31","author":[{"given":"Yelin","family":"Liu","sequence":"first","affiliation":[{"name":"School of Computing and Information Technology, University of Wollongong, Wollongong, NSW 2522, Australia"}]},{"given":"Zhi Quan","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Computing and Information Technology, University of Wollongong, Wollongong, NSW 2522, Australia"}]},{"given":"Tsong Yueh","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Software Engineering, Swinburne University of Technology, Hawthorn, VIC 3122, Australia"}]},{"given":"Yang","family":"Liu","sequence":"additional","affiliation":[{"name":"Booking.com B.V., Herengracht 597, 1017 CE Amsterdam, Netherlands"}]},{"given":"Dave","family":"Towey","sequence":"additional","affiliation":[{"name":"School of Computer Science, University of Nottingham Ningbo China, Zhejiang 315100, P. R. China"}]}],"member":"219","published-online":{"date-parts":[[2021,10,3]]},"reference":[{"issue":"1","key":"S021819402150039XBIB001","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0950-5849(02)00129-5","volume":"45","author":"Chen T. Y.","year":"2003","journal-title":"Inf. Softw. Technol."},{"key":"S021819402150039XBIB002","author":"Zhang J. M.","journal-title":"IEEE Trans. Softw. Eng."},{"key":"S021819402150039XBIB004","first-page":"1140","volume-title":"Proc. 28th ACM Joint Meeting on European Software Engineering Conf. Symp. Foundations of Software Engineering","author":"Rigger M.","year":"2020"},{"issue":"3","key":"S021819402150039XBIB005","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1109\/MS.2018.2875968","volume":"37","author":"Segura S.","year":"2020","journal-title":"IEEE Softw."},{"key":"S021819402150039XBIB006","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1145\/3180155.3180220","volume-title":"Proc. 40th Int. Conf. Software Engineering","author":"Tian Y.","year":"2018"},{"issue":"3","key":"S021819402150039XBIB007","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1145\/3241979","volume":"62","author":"Zhou Z. Q.","year":"2019","journal-title":"Commun. ACM"},{"issue":"10","key":"S021819402150039XBIB008","doi-asserted-by":"crossref","first-page":"1120","DOI":"10.1109\/TSE.2018.2876433","volume":"46","author":"Zhou Z. Q.","year":"2020","journal-title":"IEEE Trans. Softw. Eng."},{"issue":"6","key":"S021819402150039XBIB009","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1109\/MC.2016.176","volume":"49","author":"Chen T. Y.","year":"2016","journal-title":"Computer"},{"issue":"11","key":"S021819402150039XBIB010","doi-asserted-by":"crossref","first-page":"1083","DOI":"10.1109\/TSE.2017.2764464","volume":"44","author":"Segura S.","year":"2018","journal-title":"IEEE Trans. Softw. Eng."},{"key":"S021819402150039XBIB011","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1109\/MET.2017.6","volume-title":"2017 IEEE\/ACM 2nd Int. Workshop on Metamorphic Testing","author":"Lindvall M.","year":"2017"},{"key":"S021819402150039XBIB012","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1007\/978-1-4842-2766-4_7","volume-title":"Deep Learning with Python: A Hands-on Introduction","author":"Ketkar N.","year":"2017"},{"key":"S021819402150039XBIB013","first-page":"265","volume-title":"12th USENIX Symp. Operating Systems Design and Implementation","author":"Abadi M.","year":"2016"},{"issue":"3","key":"S021819402150039XBIB014","first-page":"1","volume":"4","author":"Bergstra J.","year":"2010","journal-title":"Proc. Python Sci. Comput. Conf."},{"key":"S021819402150039XBIB015","first-page":"59","volume-title":"Proc. Twelfth Int. Conf. Quality Software","author":"Liu H.","year":"2012"},{"key":"S021819402150039XBIB016","author":"Qiu K.","journal-title":"IEEE Trans. Softw. Eng."},{"key":"S021819402150039XBIB017","first-page":"96","volume-title":"2018 25th Australasian Software Engineering Conf.","author":"Sun L.","year":"2018"},{"key":"S021819402150039XBIB018","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1109\/MET.2019.00011","volume-title":"2019 IEEE\/ACM 4th Int. Workshop on Metamorphic Testing","author":"Wu C.","year":"2019"},{"issue":"4","key":"S021819402150039XBIB020","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1002\/stvr.437","volume":"22","author":"Zhou Z. Q.","year":"2012","journal-title":"Softw. Test. Verification Reliab."},{"key":"S021819402150039XBIB021","doi-asserted-by":"publisher","DOI":"10.1145\/3143561"},{"key":"S021819402150039XBIB022","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"S021819402150039XBIB023","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1145\/3387940.3392694","volume-title":"Proc. IEEE\/ACM 42nd Int. Conf. Software Engineering Workshops","author":"Liu Y.","year":"2020"},{"key":"S021819402150039XBIB024","first-page":"7","volume-title":"2020 IEEE Int. Conf. Artificial Intelligence Testing","author":"Arcaini P.","year":"2020"},{"key":"S021819402150039XBIB025","doi-asserted-by":"crossref","first-page":"4790","DOI":"10.1364\/AO.29.004790","volume":"29","author":"Zhang W.","year":"1990","journal-title":"Appl. Opt."},{"key":"S021819402150039XBIB026","first-page":"2267","volume-title":"Proc. AAAI Conf. Artificial Intelligence","volume":"29","author":"Lai S.","year":"2015"},{"key":"S021819402150039XBIB028","first-page":"850","volume-title":"IET Conf. Proc.","author":"Gers F. A.","year":"1999"},{"issue":"10","key":"S021819402150039XBIB030","doi-asserted-by":"crossref","first-page":"2702","DOI":"10.1109\/TPAMI.2019.2926463","volume":"42","author":"Huang X.","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["International Journal of Software Engineering and Knowledge Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S021819402150039X","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,10,3]],"date-time":"2021-10-03T11:40:26Z","timestamp":1633261226000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/abs\/10.1142\/S021819402150039X"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9]]},"references-count":26,"journal-issue":{"issue":"09","published-print":{"date-parts":[[2021,9]]}},"alternative-id":["10.1142\/S021819402150039X"],"URL":"https:\/\/doi.org\/10.1142\/s021819402150039x","relation":{},"ISSN":["0218-1940","1793-6403"],"issn-type":[{"value":"0218-1940","type":"print"},{"value":"1793-6403","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9]]}}}