{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T13:54:39Z","timestamp":1780581279680,"version":"3.54.1"},"reference-count":56,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2021,4,21]],"date-time":"2021-04-21T00:00:00Z","timestamp":1618963200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61772468"],"award-info":[{"award-number":["61772468"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004731","name":"Natural Science Foundation of Zhejiang Province","doi-asserted-by":"publisher","award":["Y18E050014"],"award-info":[{"award-number":["Y18E050014"]}],"id":[{"id":"10.13039\/501100004731","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Mobile application (app) reviews are feedback about experiences, requirements, and issues raised after users have used the app. The iteration of an app is driven by bug reports and user requirements analyzed and extracted from app reviews, which is a problem that app designers and developers are committed to solving. However, a great number of app reviews vary in quality and reliability. It is a difficult and time-consuming challenge to analyze app reviews using manual methods. To address this, a novel approach is proposed as an automated method to predict high priority user requests with fourteen extracted features. A semi-automated approach is applied to annotate requirements with high or low priority with the help of app changelogs. Reviews from six apps were retrieved from the Apple App Store to evaluate the feasibility of the approach and interpret the principles. The performance comparison results of the approach greatly exceed the IDEA method, with an average precision of 75.4% and recall of 70.4%. Our approach can be applied to specific app development to assist app developers in quickly locating user requirements and implement app maintenance and evolution.<\/jats:p>","DOI":"10.3390\/info12050177","type":"journal-article","created":{"date-parts":[[2021,4,21]],"date-time":"2021-04-21T12:47:07Z","timestamp":1619009227000},"page":"177","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["A Phrase-Level User Requests Mining Approach in Mobile Application Reviews: Concept, Framework, and Operation"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1783-4760","authenticated-orcid":false,"given":"Cheng","family":"Yang","sequence":"first","affiliation":[{"name":"College of Creativity and Art Design, Zhejiang University City College, Hangzhou 310015, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2301-9199","authenticated-orcid":false,"given":"Lingang","family":"Wu","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8864-0106","authenticated-orcid":false,"given":"Chunyang","family":"Yu","sequence":"additional","affiliation":[{"name":"College of Creativity and Art Design, Zhejiang University City College, Hangzhou 310015, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7390-6597","authenticated-orcid":false,"given":"Yuliang","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1016\/j.jss.2016.11.027","article-title":"A systematic literature review: Opinion mining studies from mobile app store user reviews","volume":"125","author":"Abran","year":"2017","journal-title":"J. Syst. Softw."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.jss.2017.06.071","article-title":"A statistical analysis approach to predict user\u2019s changing requirements for software service evolution","volume":"132","author":"Xie","year":"2017","journal-title":"J. Syst. Softw."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1016\/j.jss.2018.08.028","article-title":"Software engineering process models for mobile app development: A systematic literature review","volume":"145","author":"Jabangwe","year":"2018","journal-title":"J. Syst. Softw."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1016\/j.jss.2019.06.095","article-title":"An empirical study of configuration changes and adoption in Android apps","volume":"156","author":"Jha","year":"2019","journal-title":"J. Syst. Softw."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.jss.2017.11.043","article-title":"Crowdsourcing user reviews to support the evolution of mobile apps","volume":"137","author":"Palomba","year":"2018","journal-title":"J. Syst. Softw."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1964","DOI":"10.1007\/s10664-019-09684-y","article-title":"Towards prioritizing user-related issue reports of mobile applications","volume":"24","author":"Noei","year":"2019","journal-title":"Empir. Softw. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Pagano, D., and Maalej, W. (2013, January 15\u201319). User feedback in the Appstore: An empirical study. Proceedings of the 2013 21st IEEE International Requirements Engineering Conference (RE), Rio de Janeiro, Brazil.","DOI":"10.1109\/RE.2013.6636712"},{"key":"ref_8","unstructured":"Gao, C., Zeng, J., Lyu, M.R., and King, I. (June, January 27). Online app review analysis for identifying emerging issues. Proceedings of the 40th International Conference on Software Engineering, Gothenburg, Sweden."},{"key":"ref_9","unstructured":"Chen, N., Lin, J., Hoi, S.C., Xiao, X., and Zhang, B. (June, January 31). AR-miner: Mining informative reviews for developers from mobile app marketplace. Proceedings of the 36th International Conference on Software Engineering, Hyderabad, India."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.jss.2017.12.028","article-title":"Automatically classifying user requests in crowdsourcing requirements engineering","volume":"138","author":"Li","year":"2018","journal-title":"J. Syst. Softw."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Suprayogi, E., Budi, I., and Mahendra, R. (2018, January 27\u201328). Information extraction for mobile application user review. Proceedings of the 2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS), Yogyakarta, Indonesia.","DOI":"10.1109\/ICACSIS.2018.8618164"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Nayebi, M., and Ruhe, G. (2017, January 4\u20138). Optimized functionality for super mobile apps. Proceedings of the 2017 IEEE 25th International Requirements Engineering Conference (RE), Lisbon, Portugal.","DOI":"10.1109\/RE.2017.72"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Gu, X., and Kim, S. (2015, January 9\u201313). What parts of your apps are loved by users?. Proceedings of the 2015 30th IEEE\/ACM International Conference on Automated Software Engineering (ASE), Lincoln, NE, USA.","DOI":"10.1109\/ASE.2015.57"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1109\/TSE.2017.2759112","article-title":"Listening to the crowd for the release planning of mobile Apps","volume":"45","author":"Scalabrino","year":"2019","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Panichella, S., Di Sorbo, A., Guzman, E., Visaggio, C.A., Canfora, G., and Gall, H.C. (October, January 29). How can I improve my App? Classifying user reviews for software maintenance and evolution. Proceedings of the 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME), Bremen, Germany.","DOI":"10.1109\/ICSM.2015.7332474"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1007\/s00766-016-0251-9","article-title":"On the automatic classification of App reviews","volume":"21","author":"Maalej","year":"2016","journal-title":"Requir. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1007\/s10664-015-9375-7","article-title":"Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews","volume":"21","author":"McIlroy","year":"2016","journal-title":"Empir. Softw. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Guzman, E., El-Haliby, M., and Bruegge, B. (2015, January 9\u201313). Ensemble methods for App review classification: An approach for software evolution. Proceedings of the 2015 30th IEEE\/ACM International Conference on Automated Software Engineering (ASE), Lincoln, NE, USA.","DOI":"10.1109\/ASE.2015.88"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3659","DOI":"10.1007\/s10664-019-09716-7","article-title":"Mining non-functional requirements from App Store reviews","volume":"24","author":"Jha","year":"2019","journal-title":"Empir. Softw. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1109\/MIS.2013.30","article-title":"New avenues in opinion mining and sentiment analysis","volume":"28","author":"Cambria","year":"2013","journal-title":"IEEE Intell. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ranjan, S., and Mishra, S. (2020, January 1\u20133). Comparative sentiment analysis of App reviews. Proceedings of the 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kharagpur, India.","DOI":"10.1109\/ICCCNT49239.2020.9225348"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1109\/TSE.2018.2810895","article-title":"Asymmetric release planning: Compromising satisfaction against dissatisfaction","volume":"45","author":"Nayebi","year":"2019","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Jo, Y., and Oh, A.H. (2011, January 9\u201312). Aspect and sentiment unification model for online review analysis. Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, Hong Kong, China.","DOI":"10.1145\/1935826.1935932"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Guzman, E., and Maalej, W. (2014, January 25\u201329). How do users like this feature? a fine grained sentiment analysis of app reviews. Proceedings of the 2014 IEEE 22nd International Requirements Engineering Conference (RE), Karlskrona, Sweden.","DOI":"10.1109\/RE.2014.6912257"},{"key":"ref_25","unstructured":"Shuyo, N. (2021, April 19). Language Detection Library for JAVA. Available online: https:\/\/github.com\/shuyo\/language-detection."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Palomba, F., Salza, P., Ciurumelea, A., Panichella, S., Gall, H., Ferrucci, F., and De Lucia, A. (2017, January 20\u201328). Recommending and localizing change requests for mobile apps based on user reviews. Proceedings of the 2017 IEEE\/ACM 39th International Conference on Software Engineering (ICSE), Buenos Aires, Argentina.","DOI":"10.1109\/ICSE.2017.18"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sarro, F., Al-Subaihin, A.A., Harman, M., Jia, Y., Martin, W., and Zhang, Y. (2015, January 24\u201328). Feature lifecycles as they spread, migrate, remain, and die in app stores. Proceedings of the 2015 IEEE 23rd International Requirements Engineering Conference (RE), Ottawa, ON, Canada.","DOI":"10.1109\/RE.2015.7320410"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.dss.2017.01.006","article-title":"Whose online reviews to trust? Understanding reviewer trustworthiness and its impact on business","volume":"96","author":"Banerjee","year":"2017","journal-title":"Decis. Support Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.infsof.2018.12.002","article-title":"Software feature refinement prioritization based on online user review mining","volume":"108","author":"Zhang","year":"2019","journal-title":"Inf. Softw. Technol."},{"key":"ref_30","unstructured":"Manning, C., and Schutze, H. (1999). Foundations of Statistical Natural Language Processing, MIT Press."},{"key":"ref_31","unstructured":"Bird, S., Klein, E., and Loper, E. (2009). Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit, O\u2019Reilly Media, Inc."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1075\/ijcl.11.4.04che","article-title":"From n-gram to skipgram to concgram","volume":"11","author":"Cheng","year":"2006","journal-title":"Int. J. Corpus Linguist."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1080\/10864415.2016.1087823","article-title":"What in consumer reviews affects the sales of mobile apps: A multifacet sentiment analysis approach","volume":"20","author":"Liang","year":"2015","journal-title":"Int. J. Electron. Commer."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"5142","DOI":"10.1080\/00207543.2015.1066519","article-title":"Predicting consumer product demands via Big Data: the roles of online promotional marketing and online reviews","volume":"55","author":"Chong","year":"2017","journal-title":"Int. J. Prod. Res."},{"key":"ref_35","unstructured":"Bouma, G. (2009, January 30). Normalized (pointwise) mutual information in collocation extraction. Proceedings of the Biennial GSCL Conference, Potsdam, Germany."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1376815.1376819","article-title":"Semantic text similarity using corpus-based word similarity and string similarity","volume":"2","author":"Islam","year":"2008","journal-title":"ACM Trans. Knowl. Discov. Data"},{"key":"ref_37","unstructured":"Rehurek, R., and Sojka, P. (2010, January 22). Software framework for topic modelling with large corpora. Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, Valletta, Malta."},{"key":"ref_38","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean, J. (2013, January 5\u201310). Distributed representations ofwords and phrases and their compositionality. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_39","unstructured":"Mikolov, T., Yih, W.T., and Zweig, G. (2013, January 9\u201314). Linguistic regularities in continuous space word representations. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, GA, USA."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1007730.1007733","article-title":"Special issue on learning from imbalanced data sets","volume":"6","author":"Chawla","year":"2004","journal-title":"ACM SIGKDD Explor. Newsl."},{"key":"ref_41","first-page":"2825","article-title":"Scikit-learn: Machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Maalej, W., and Nabil, H. (2015, January 24\u201328). Bug report, feature request, or simply praise? on automatically classifying app reviews. Proceedings of the 2015 IEEE 23rd International Requirements Engineering Conference (RE), Ottawa, ON, Canada.","DOI":"10.1109\/RE.2015.7320414"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Wang, C., Zhang, F., Liang, P., Daneva, M., and van Sinderen, M. (2018, January 11\u201312). Can app changelogs improve requirements classification from app reviews? an exploratory study. Proceedings of the 12th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement, Oulu, Finland.","DOI":"10.1145\/3239235.3267428"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"3316","DOI":"10.1007\/s10664-019-09706-9","article-title":"Towards understanding and detecting fake reviews in app stores","volume":"24","author":"Martens","year":"2019","journal-title":"Empir. Softw. Eng."},{"key":"ref_45","unstructured":"Baeza-Yates, R., and Ribeiro-Neto, B. (1999). Modern Information Retrieval, ACM Press."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Carreno, L.V.G., and Winbladh, K. (2013, January 18\u201326). Analysis of user comments: an approach for software requirements evolution. Proceedings of the 2013 35th International Conference on Software Engineering (ICSE), San Francisco, CA, USA.","DOI":"10.1109\/ICSE.2013.6606604"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An introduction to ROC analysis","volume":"27","author":"Fawcett","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_48","unstructured":"Ho, T.K. (1995, January 14\u201316). Random decision forests. Proceedings of the 3rd International Conference on Document Analysis and Recognition, Montreal, QC, Canada."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"3021","DOI":"10.21105\/joss.03021","article-title":"Seaborn: statistical data visualization","volume":"6","author":"Waskom","year":"2021","journal-title":"J. Open Source Softw."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1214\/aoms\/1177730491","article-title":"On a test of whether one of two random variables is stochastically larger than the other","volume":"18","author":"Mann","year":"1947","journal-title":"Ann. Math. Stat."},{"key":"ref_51","first-page":"1157","article-title":"An introduction to variable and feature selection","volume":"3","author":"Guyon","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1023\/A:1012487302797","article-title":"Gene selection for cancer classification using support vector machines","volume":"46","author":"Guyon","year":"2002","journal-title":"Mach. Learn."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Chen, X.W., and Jeong, J.C. (2007, January 13\u201315). Enhanced recursive feature elimination. Proceedings of the 6th International Conference on Machine Learning and Applications, ICMLA 2007, Cincinnati, OH, USA.","DOI":"10.1109\/ICMLA.2007.35"},{"key":"ref_54","first-page":"281","article-title":"Random search for hyper-parameter optimization","volume":"13","author":"Bergstra","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Hinton, G.E. (2012). A practical guide to training restricted Boltzmann machines. Neural Networks: Tricks of the Trade, Springer.","DOI":"10.1007\/978-3-642-35289-8_32"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Bernard, S., Heutte, L., and Adam, S. (2009, January 10\u201312). Influence of hyperparameters on random forest accuracy. Proceedings of the International Workshop on Multiple Classifier Systems, Reykjavik, Iceland.","DOI":"10.1007\/978-3-642-02326-2_18"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/5\/177\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:50:43Z","timestamp":1760161843000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/5\/177"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,21]]},"references-count":56,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2021,5]]}},"alternative-id":["info12050177"],"URL":"https:\/\/doi.org\/10.3390\/info12050177","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,21]]}}}