{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,11]],"date-time":"2026-07-11T21:19:42Z","timestamp":1783804782776,"version":"3.55.0"},"reference-count":66,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2019,1,9]],"date-time":"2019-01-09T00:00:00Z","timestamp":1546992000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100010663","name":"European Research Council","doi-asserted-by":"publisher","award":["694277"],"award-info":[{"award-number":["694277"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001866","name":"Fonds National de la Recherche Luxembourg","doi-asserted-by":"publisher","award":["FNR\/P10\/03 and FNR-11601446"],"award-info":[{"award-number":["FNR\/P10\/03 and FNR-11601446"]}],"id":[{"id":"10.13039\/501100001866","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2019,1,31]]},"abstract":"<jats:p>Domain models are a useful vehicle for making the interpretation and elaboration of natural-language requirements more precise. Advances in natural-language processing (NLP) have made it possible to automatically extract from requirements most of the information that is relevant to domain model construction. However, alongside the relevant information, NLP extracts from requirements a significant amount of information that is superfluous (not relevant to the domain model). Our objective in this article is to develop automated assistance for filtering the superfluous information extracted by NLP during domain model extraction. To this end, we devise an active-learning-based approach that iteratively learns from analysts\u2019 feedback over the relevance and superfluousness of the extracted domain model elements and uses this feedback to provide recommendations for filtering superfluous elements. We empirically evaluate our approach over three industrial case studies. Our results indicate that, once trained, our approach automatically detects an average of \u2248 45% of the superfluous elements with a precision of \u2248 96%. Since precision is very high, the automatic recommendations made by our approach are trustworthy. Consequently, analysts can dispose of a considerable fraction \u2013 nearly half \u2013 of the superfluous elements with minimal manual work. The results are particularly promising, as they should be considered in light of the non-negligible subjectivity that is inherently tied to the notion of relevance.<\/jats:p>","DOI":"10.1145\/3293454","type":"journal-article","created":{"date-parts":[[2019,1,7]],"date-time":"2019-01-07T13:42:28Z","timestamp":1546868548000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":32,"title":["An Active Learning Approach for Improving the Accuracy of Automated Domain Model Extraction"],"prefix":"10.1145","volume":"28","author":[{"given":"Chetan","family":"Arora","sequence":"first","affiliation":[{"name":"SnT Centre for Security, Reliability and Trust, University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mehrdad","family":"Sabetzadeh","sequence":"additional","affiliation":[{"name":"SnT Centre for Security, Reliability and Trust, University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shiva","family":"Nejati","sequence":"additional","affiliation":[{"name":"SnT Centre for Security, Reliability and Trust, University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lionel","family":"Briand","sequence":"additional","affiliation":[{"name":"SnT Centre for Security, Reliability and Trust, University of Luxembourg, Luxembourg"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,1,9]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Workshop on Semantic Search at International World Wide Web Conference (WWW\u201909)","author":"Akbik Alan","year":"2009","unstructured":"Alan Akbik and Juergen Bro\u00df . 2009 . Wanderlust: Extracting semantic relations from natural language text using dependency grammar patterns . In Workshop on Semantic Search at International World Wide Web Conference (WWW\u201909) . ACM, Madrid, Spain, 5--16. Alan Akbik and Juergen Bro\u00df. 2009. Wanderlust: Extracting semantic relations from natural language text using dependency grammar patterns. In Workshop on Semantic Search at International World Wide Web Conference (WWW\u201909). ACM, Madrid, Spain, 5--16."},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL\u201915)","author":"Angeli Gabor","unstructured":"Gabor Angeli , Melvin Johnson Premkumar , and Christopher D. Manning . 2015. Leveraging linguistic structure for open domain information extraction . In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL\u201915) . ACL, Beijing, China, 344--354. Gabor Angeli, Melvin Johnson Premkumar, and Christopher D. Manning. 2015. Leveraging linguistic structure for open domain information extraction. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL\u201915). ACL, Beijing, China, 344--354."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2976767.2976769"},{"key":"e_1_2_1_4_1","volume-title":"Silverstein","author":"Auerbach Carl","year":"2003","unstructured":"Carl Auerbach and Louise B . Silverstein . 2003 . Qualitative Data : An Introduction to Coding and Analysis. NYU Press , New York, NY. Carl Auerbach and Louise B. Silverstein. 2003. Qualitative Data: An Introduction to Coding and Analysis. NYU Press, New York, NY."},{"key":"e_1_2_1_5_1","first-page":"281","article-title":"Random search for hyper-parameter optimization","author":"Bergstra James","year":"2012","unstructured":"James Bergstra and Yoshua Bengio . 2012 . Random search for hyper-parameter optimization . Journal of Machine Learning Research 13 , Feb (2012), 281 -- 305 . James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of Machine Learning Research 13, Feb (2012), 281--305.","journal-title":"Journal of Machine Learning Research 13"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2907942"},{"key":"e_1_2_1_7_1","volume-title":"Classification and Regression Trees. Wadsworth and Brooks","author":"Breiman Leo","unstructured":"Leo Breiman , Jerome Friedman , Richard Olshen , and Charles Stone . 1984. Classification and Regression Trees. Wadsworth and Brooks , London, UK . Leo Breiman, Jerome Friedman, Richard Olshen, and Charles Stone. 1984. Classification and Regression Trees. Wadsworth and Brooks, London, UK."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2009.10.010"},{"key":"e_1_2_1_9_1","volume-title":"Sampling","author":"Chen Jianfeng","year":"2018","unstructured":"Jianfeng Chen , Vivek Nair , Rahul Krishna , and Tim Menzies . 2018. \u201c Sampling \u201d as a baseline optimizer for search-based software engineering. IEEE Transactions on Software Engineering ( 2018 ). (in press). https:\/\/ieeexplore.ieee.org\/document\/8249828. Jianfeng Chen, Vivek Nair, Rahul Krishna, and Tim Menzies. 2018. \u201cSampling\u201d as a baseline optimizer for search-based software engineering. IEEE Transactions on Software Engineering (2018). (in press). https:\/\/ieeexplore.ieee.org\/document\/8249828."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806825"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-007-0045-1"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2600428.2609601"},{"key":"e_1_2_1_13_1","volume-title":"6th Conference on Email and Anti-Spam (CEAS\u201909)","author":"DeBarr Dave","year":"2009","unstructured":"Dave DeBarr and Harry Wechsler . 2009 . Spam detection using clustering, random forests, and active learning . In 6th Conference on Email and Anti-Spam (CEAS\u201909) . Citeseer, Mountain View, CA, 1--6. Dave DeBarr and Harry Wechsler. 2009. Spam detection using clustering, random forests, and active learning. In 6th Conference on Email and Anti-Spam (CEAS\u201909). Citeseer, Mountain View, CA, 1--6."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2011.02.025"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/2145432.2145596"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0031619"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2016.04.017"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-011-0134-z"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.9"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.5555\/944919.944968"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/AIRE.2014.6894850"},{"key":"e_1_2_1_22_1","volume-title":"Natural Language Processing for Online Applications: Text Retrieval, Extraction and Categorization","author":"Jackson Peter","unstructured":"Peter Jackson and Isabelle Moulinier . 2007. Natural Language Processing for Online Applications: Text Retrieval, Extraction and Categorization . Vol. 5 . John Benjamins Publishing , Amsterdam, Netherlands . Peter Jackson and Isabelle Moulinier. 2007. Natural Language Processing for Online Applications: Text Retrieval, Extraction and Categorization. Vol. 5. John Benjamins Publishing, Amsterdam, Netherlands."},{"key":"e_1_2_1_23_1","volume-title":"Spotting and Discovering Terms Through Natural Language Processing","author":"Jacquemin Christian","unstructured":"Christian Jacquemin . 2001. Spotting and Discovering Terms Through Natural Language Processing . MIT Press , Cambridge, MA . Christian Jacquemin. 2001. Spotting and Discovering Terms Through Natural Language Processing. MIT Press, Cambridge, MA."},{"key":"e_1_2_1_24_1","first-page":"31","article-title":"Representational learning with extreme learning machine for big data","volume":"28","author":"Chamara Kasun Liyanaarachchi Lekamalage","year":"2013","unstructured":"Liyanaarachchi Lekamalage Chamara Kasun , Hongming Zhou , Guang-Bin Huang , and Chi Man Vong . 2013 . Representational learning with extreme learning machine for big data . IEEE Intelligent Systems 28 , 6 (2013), 31 -- 34 . Liyanaarachchi Lekamalage Chamara Kasun, Hongming Zhou, Guang-Bin Huang, and Chi Man Vong. 2013. Representational learning with extreme learning machine for big data. IEEE Intelligent Systems 28, 6 (2013), 31--34.","journal-title":"IEEE Intelligent Systems"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/360402.360406"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2004.08.004"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2017.82"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-018-0293-2"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.2307\/2529786"},{"key":"e_1_2_1_30_1","volume-title":"Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development","author":"Larman Craig","unstructured":"Craig Larman . 2004. Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development ( 3 rd ed.). Prentice Hall , Upper Saddle River, NJ. Craig Larman. 2004. Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development (3rd ed.). Prentice Hall, Upper Saddle River, NJ.","edition":"3"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the 17th ACM International Conference on Research and Development in Information Retrieval (SIGIR\u201994)","author":"David","unstructured":"David D. Lewis and William A. Gale. 1994. A sequential algorithm for training text classifiers . In Proceedings of the 17th ACM International Conference on Research and Development in Information Retrieval (SIGIR\u201994) . Springer, Dublin, Ireland, 3--12. David D. Lewis and William A. Gale. 1994. A sequential algorithm for training text classifiers. In Proceedings of the 17th ACM International Conference on Research and Development in Information Retrieval (SIGIR\u201994). Springer, Dublin, Ireland, 3--12."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.268955"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.114"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-017-0270-1"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-003-0179-8"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-016-0251-9"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-016-0252-8"},{"key":"e_1_2_1_38_1","volume-title":"Manning and Hinrich Sch\u00fctze","author":"Christopher","year":"1999","unstructured":"Christopher D. Manning and Hinrich Sch\u00fctze . 1999 . Foundations of Statistical Natural Language Processing. MIT Press , Cambridge, MA. Christopher D. Manning and Hinrich Sch\u00fctze. 1999. Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, MA."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-5010"},{"key":"e_1_2_1_40_1","volume-title":"Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications","author":"Miner Gary","unstructured":"Gary Miner , John Elder , and Thomas Hill . 2012. Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications . Academic Press , Cambridge, MA . Gary Miner, John Elder, and Thomas Hill. 2012. Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications. Academic Press, Cambridge, MA."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2014.06.005"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocu041"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2009.191"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2012.52"},{"key":"e_1_2_1_45_1","volume-title":"Requirements Engineering \u2014Fundamentals, Principles, and Techniques","author":"Pohl Klaus","unstructured":"Klaus Pohl . 2010. Requirements Engineering \u2014Fundamentals, Principles, and Techniques . Springer , Heidelberg, Germany . Klaus Pohl. 2010. Requirements Engineering \u2014Fundamentals, Principles, and Techniques. Springer, Heidelberg, Germany."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-54045-0_19"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-014-0156-0"},{"key":"e_1_2_1_48_1","volume-title":"Recommendation Systems in Software Engineering","author":"Robillard Martin P.","unstructured":"Martin P. Robillard , Walid Maalej , Robert J. Walker , and Thomas Zimmermann . 2014. Recommendation Systems in Software Engineering . Springer Science 8 Business, New York, NY. Martin P. Robillard, Walid Maalej, Robert J. Walker, and Thomas Zimmermann. 2014. Recommendation Systems in Software Engineering. Springer Science 8 Business, New York, NY."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.13"},{"key":"e_1_2_1_50_1","volume-title":"The Coding Manual for Qualitative Researchers","author":"Salda\u00f1a Johnny","unstructured":"Johnny Salda\u00f1a . 2015. The Coding Manual for Qualitative Researchers ( 3 rd ed.). Sage Publications , London, UK . Johnny Salda\u00f1a. 2015. The Coding Manual for Qualitative Researchers (3rd ed.). Sage Publications, London, UK.","edition":"3"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-38824-8_1"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.5555\/3019233"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.5555\/1613715.1613855"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0016973"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2970276.2970323"},{"key":"e_1_2_1_56_1","volume-title":"Practical Model-based Testing: A Tools Approach. Morgan Kaufmann","author":"Utting Mark","unstructured":"Mark Utting and Bruno Legeard . 2010. Practical Model-based Testing: A Tools Approach. Morgan Kaufmann , Burlington, MA . Mark Utting and Bruno Legeard. 2010. Practical Model-based Testing: A Tools Approach. Morgan Kaufmann, Burlington, MA."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2013.08.036"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/2771783.2771812"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2018.00-53"},{"key":"e_1_2_1_60_1","volume-title":"Multi Search \u2014 Weka package for parameter optimization. Retrieved","author":"Weka","year":"2018","unstructured":"Weka MultiSearch 2017. Multi Search \u2014 Weka package for parameter optimization. Retrieved March 2018 . https:\/\/github.com\/fracpete\/multisearch-weka-package\/. Weka MultiSearch 2017. Multi Search \u2014 Weka package for parameter optimization. Retrieved March 2018. https:\/\/github.com\/fracpete\/multisearch-weka-package\/."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/REW.2016.021"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-77243-1_4"},{"key":"e_1_2_1_63_1","volume-title":"Pal","author":"Witten Ian H.","year":"2016","unstructured":"Ian H. Witten , Eibe Frank , Mark A. Hall , and Christopher J . Pal . 2016 . Data Mining : Practical Machine Learning Tools and Techniques. Morgan Kaufmann , Burlington, MA. Ian H. Witten, Eibe Frank, Mark A. Hall, and Christopher J. Pal. 2016. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Burlington, MA."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/1858996.1859007"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9587-0"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-010-0111-y"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3293454","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3293454","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:02:01Z","timestamp":1750208521000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3293454"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,9]]},"references-count":66,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,1,31]]}},"alternative-id":["10.1145\/3293454"],"URL":"https:\/\/doi.org\/10.1145\/3293454","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,9]]},"assertion":[{"value":"2017-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-01-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}