{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T18:45:00Z","timestamp":1761677100977,"version":"3.40.5"},"reference-count":29,"publisher":"SAGE Publications","issue":"8","license":[{"start":{"date-parts":[[2022,8,1]],"date-time":"2022-08-01T00:00:00Z","timestamp":1659312000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["International Journal of Distributed Sensor Networks"],"published-print":{"date-parts":[[2022,8]]},"abstract":"<jats:p> The explosive growth and rapid version iteration of various mobile applications have brought enormous workloads to mobile application testing. Robotic testing methods can efficiently handle repetitive testing tasks, which can compensate for the accuracy of manual testing and improve the efficiency of testing work. Vision-based robotic testing identifies the types of test actions by analyzing expert test videos and generates expert imitation test cases. The mobile application expert imitation testing method uses machine learning algorithms to analyze the behavior of experts imitating test videos, generates test cases with high reliability and reusability, and drives robots to execute test cases. However, the difficulty of estimating multi-dimensional gestures in 2D images leads to complex algorithm steps, including tracking, detection, and recognition of dynamic gestures. Hence, this article focuses on the analysis and recognition of test actions in mobile application robot testing. Combined with the improved YOLOv5 algorithm and the ResNet-152 algorithm, a visual modeling method of mobile application test action based on machine vision is proposed. The precise localization of the hand is accomplished by injecting dynamic anchors, attention mechanism, and the weighted boxes fusion in the YOLOv5 algorithm. The improved algorithm recognition accuracy increased from 82.6% to 94.8%. By introducing the pyramid context awareness mechanism into the ResNet-152 algorithm, the accuracy of test action classification is improved. The accuracy of the test action classification was improved from 72.57% to 76.84%. Experiments show that this method can reduce the probability of multiple detections and missed detection of test actions, and improve the accuracy of test action recognition. <\/jats:p>","DOI":"10.1177\/15501329221115375","type":"journal-article","created":{"date-parts":[[2022,8,4]],"date-time":"2022-08-04T11:16:18Z","timestamp":1659611778000},"page":"155013292211153","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":8,"title":["Machine vision-based testing action recognition method for robotic testing of mobile application"],"prefix":"10.1177","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3695-8328","authenticated-orcid":false,"given":"Tao","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Software, Northwestern Polytechnical University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhengqi","family":"Su","sequence":"additional","affiliation":[{"name":"School of Software, Northwestern Polytechnical University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jing","family":"Cheng","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Xi\u2019an Technological University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Feng","family":"Xue","sequence":"additional","affiliation":[{"name":"School of Software, Northwestern Polytechnical University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shengyu","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Software, Northwestern Polytechnical University, Xi\u2019an, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2022,8,4]]},"reference":[{"first-page":"293","volume-title":"Proceedings of the 2019 9th international conference on advanced computer information technologies (ACIT)","author":"Yatskiv S","key":"bibr1-15501329221115375"},{"volume-title":"Proceedings of the 2019 IEEE international conference on smart cloud","author":"Ma YW","key":"bibr2-15501329221115375"},{"volume-title":"Proceedings of the 2019 IEEE 4th advanced information technology, electronic and automation control conference (IAEAC)","author":"Maalla A","key":"bibr3-15501329221115375"},{"volume-title":"Proceedings of the 2020 10th international conference on advanced computer information technologies (ACIT)","author":"Yatskiv N","key":"bibr4-15501329221115375"},{"first-page":"334","volume-title":"Proceedings of the 2016 IEEE international conference on electron devices and solid-state circuits (EDSSC)","author":"Abhishek KS","key":"bibr5-15501329221115375"},{"volume-title":"Proceedings of the 20th international conference on human-computer interaction with mobile devices and services","author":"Krieter P","key":"bibr6-15501329221115375"},{"key":"bibr7-15501329221115375","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2020.2987044"},{"key":"bibr8-15501329221115375","first-page":"1084602","volume":"2022","author":"Xue F","year":"2022","journal-title":"Mob Inform Syst"},{"key":"bibr9-15501329221115375","doi-asserted-by":"publisher","DOI":"10.1109\/TFUZZ.2022.3170646"},{"key":"bibr10-15501329221115375","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2021.3108237"},{"key":"bibr11-15501329221115375","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2016.2617464"},{"first-page":"1","volume-title":"Proceedings of the 2019 international conference on electronics, information, and communication (ICEIC)","author":"Park S","key":"bibr12-15501329221115375"},{"first-page":"3742","volume-title":"Proceedings of the 2019 IEEE\/CVF international conference on computer vision workshop (ICCVW)","author":"Roig C","key":"bibr13-15501329221115375"},{"key":"bibr14-15501329221115375","doi-asserted-by":"publisher","DOI":"10.3390\/s22072513"},{"first-page":"1","volume-title":"Proceedings of the 2020 IEEE international symposium on systems engineering (ISSE)","author":"M\u00fcezzino\u011flu T","key":"bibr15-15501329221115375"},{"key":"bibr16-15501329221115375","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-13-3185-5_6"},{"first-page":"784","volume-title":"Proceedings of the 2018 IEEE international conference on robotics and biomimetics (ROBIO)","author":"Zhu C","key":"bibr17-15501329221115375"},{"key":"bibr18-15501329221115375","unstructured":"Liu C, Szir\u00e1nyi T. Gesture recognition for UAV-based rescue operation based on deep learning, 2021, pp.180\u2013187, https:\/\/pdfs.semanticscholar.org\/b38c\/115443b12d8ab3f65ecf948fccb52bef7399.pdf?_ga=2.48293408.702853629.1658460100-1792531526.1619760732"},{"key":"bibr19-15501329221115375","unstructured":"Human interface guidelines, 2022, https:\/\/developer.apple.com\/design\/human-interface-guidelines\/guidelines\/overview\/"},{"key":"bibr20-15501329221115375","unstructured":"Material design, 2022, https:\/\/material.io\/design\/interaction\/gestures.html"},{"first-page":"508","volume-title":"Proceedings of the 2021 8th international conference on dependable systems and their applications (DSA)","author":"Zhang C","key":"bibr21-15501329221115375"},{"first-page":"233","volume-title":"Proceedings of the 2021 international conference on intelligent computing, automation and applications (ICAA)","author":"Cheng J","key":"bibr22-15501329221115375"},{"first-page":"238","volume-title":"Proceedings of the 2016 31st IEEE\/ACM international conference on automated software engineering (ASE)","author":"Baek YM","key":"bibr23-15501329221115375"},{"first-page":"269","volume-title":"Proceedings of the 2019 IEEE\/ACM 41st international conference on software engineering (ICSE)","author":"Gu T","key":"bibr24-15501329221115375"},{"first-page":"1729","volume-title":"Proceedings of the 2020 international conference on computational science and computational intelligence (CSCI)","author":"Frister D","key":"bibr25-15501329221115375"},{"key":"bibr26-15501329221115375","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.49"},{"first-page":"801","volume-title":"Proceedings of the 2021 33rd Chinese control and decision conference (CCDC)","author":"Ling L","key":"bibr27-15501329221115375"},{"key":"bibr28-15501329221115375","doi-asserted-by":"crossref","unstructured":"Menghi C, Tsigkanos C, Pelliccione P, et al. Specification patterns for robotic missions, 2019, https:\/\/arxiv.org\/pdf\/1901.02077.pdf","DOI":"10.1109\/ICSE-Companion.2019.00048"},{"first-page":"571","volume-title":"Proceedings of the 38th international conference on software engineering (ICSE)","author":"Qin Z","key":"bibr29-15501329221115375"}],"container-title":["International Journal of Distributed Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/15501329221115375","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/15501329221115375","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/15501329221115375","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,4]],"date-time":"2022-08-04T11:16:40Z","timestamp":1659611800000},"score":1,"resource":{"primary":{"URL":"http:\/\/journals.sagepub.com\/doi\/10.1177\/15501329221115375"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8]]},"references-count":29,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2022,8]]}},"alternative-id":["10.1177\/15501329221115375"],"URL":"https:\/\/doi.org\/10.1177\/15501329221115375","relation":{},"ISSN":["1550-1329","1550-1477"],"issn-type":[{"type":"print","value":"1550-1329"},{"type":"electronic","value":"1550-1477"}],"subject":[],"published":{"date-parts":[[2022,8]]}}}