{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,22]],"date-time":"2025-12-22T08:15:31Z","timestamp":1766391331596,"version":"3.48.0"},"reference-count":57,"publisher":"World Scientific Pub Co Pte Ltd","issue":"02","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61902015"],"award-info":[{"award-number":["61902015"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61872026"],"award-info":[{"award-number":["61872026"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62077003"],"award-info":[{"award-number":["62077003"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Soft. Eng. Knowl. Eng."],"published-print":{"date-parts":[[2026,2]]},"abstract":"<jats:p>Continuous Integration (CI) is a fundamental practice in modern software development. It enables early fault detection through regression testing, where Test Case Prioritization (TCP) plays a crucial role in improving detection efficiency. Reinforcement Learning-based TCP (RL-TCP) has shown promise in CI scenarios, but its performance often fluctuates due to CI\u2019s dynamic nature. Existing solutions address this by assigning additional rewards or periodically retraining agents. However, these methods either risk over-adjusting strategies due to excessive reliance on additional rewards or fail to respond promptly to changes due to fixed retraining intervals. We propose a novel Environment Adaptation Agent-based RL-TCP method (EAA) that addresses these challenges through a dual mechanism. EAA detects significant environmental changes by analyzing fluctuations in prioritization effectiveness. When a change is detected, it assigns targeted rewards to test cases. EAA also refines the agent\u2019s gradient update so that environmental dynamics are better incorporated into retraining. This enables agents to swiftly adapt while retaining learned prioritization knowledge. Evaluations on 12 real-world industrial datasets show that EAA improves the NAPFD metric by 4.7\u201324.79% and reduces the average TTF by 35.85\u201350.37 positions compared to state-of-the-art RL-TCP methods. Additionally, EAA significantly reduces occurrences of NAPFD equal to zero, effectively mitigating prioritization instability.<\/jats:p>","DOI":"10.1142\/s0218194025500792","type":"journal-article","created":{"date-parts":[[2025,10,14]],"date-time":"2025-10-14T01:09:50Z","timestamp":1760404190000},"page":"311-341","source":"Crossref","is-referenced-by-count":0,"title":["An Environment Adaptation Agent of Reinforcement Learning in Continuous Integration Test Case Prioritization"],"prefix":"10.1142","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3938-7033","authenticated-orcid":false,"given":"Zheng","family":"Li","sequence":"first","affiliation":[{"name":"College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100020, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-5187-4853","authenticated-orcid":false,"given":"Jiping","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100020, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-2030-2228","authenticated-orcid":false,"given":"Shunqing","family":"Xu","sequence":"additional","affiliation":[{"name":"College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100020, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5884-2089","authenticated-orcid":false,"given":"Hengyuan","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100020, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1754-3039","authenticated-orcid":false,"given":"Yong","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100020, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2025,11,11]]},"reference":[{"key":"S0218194025500792BIB001","doi-asserted-by":"publisher","DOI":"10.1109\/Agile.2008.31"},{"key":"S0218194025500792BIB002","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-016-6112-3"},{"key":"S0218194025500792BIB003","doi-asserted-by":"publisher","DOI":"10.1023\/B:SQJO.0000034708.84524.22"},{"key":"S0218194025500792BIB004","doi-asserted-by":"publisher","DOI":"10.1145\/3092703.3092709"},{"key":"S0218194025500792BIB005","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.01.012"},{"volume-title":"Reinforcement Learning: An Introduction","year":"2018","author":"Sutton R. S.","key":"S0218194025500792BIB006"},{"key":"S0218194025500792BIB007","doi-asserted-by":"publisher","DOI":"10.1109\/QRS57517.2022.00088"},{"key":"S0218194025500792BIB008","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-024-11062-0"},{"key":"S0218194025500792BIB009","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.1997.630875"},{"key":"S0218194025500792BIB010","doi-asserted-by":"publisher","DOI":"10.1016\/j.jksuci.2018.09.005"},{"key":"S0218194025500792BIB011","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.1999.792604"},{"key":"S0218194025500792BIB012","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2013.91"},{"key":"S0218194025500792BIB013","doi-asserted-by":"publisher","DOI":"10.1109\/APSIPAASC47483.2019.9023052"},{"key":"S0218194025500792BIB014","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635910"},{"key":"S0218194025500792BIB015","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.38"},{"key":"S0218194025500792BIB016","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-39742-4_10"},{"key":"S0218194025500792BIB017","doi-asserted-by":"publisher","DOI":"10.1109\/TETCI.2017.2699228"},{"key":"S0218194025500792BIB018","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2018.09.055"},{"key":"S0218194025500792BIB019","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110787"},{"key":"S0218194025500792BIB020","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106268"},{"key":"S0218194025500792BIB021","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-10066-6"},{"key":"S0218194025500792BIB022","doi-asserted-by":"publisher","DOI":"10.1109\/SERA57763.2023.10197719"},{"key":"S0218194025500792BIB023","doi-asserted-by":"publisher","DOI":"10.1109\/ICMLA.2016.0065"},{"key":"S0218194025500792BIB024","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME52107.2021.00053"},{"key":"S0218194025500792BIB025","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2023.107339"},{"key":"S0218194025500792BIB026","doi-asserted-by":"publisher","DOI":"10.1109\/SANER-C62648.2024.00030"},{"key":"S0218194025500792BIB027","doi-asserted-by":"publisher","DOI":"10.1109\/ICIEM48762.2020.9160225"},{"key":"S0218194025500792BIB028","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA51224.2020.00023"},{"key":"S0218194025500792BIB029","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3070549"},{"key":"S0218194025500792BIB030","doi-asserted-by":"publisher","DOI":"10.1109\/IEMCON.2019.8936202"},{"key":"S0218194025500792BIB031","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3184842"},{"key":"S0218194025500792BIB032","doi-asserted-by":"publisher","DOI":"10.1109\/ICST62969.2025.10989002"},{"key":"S0218194025500792BIB033","unstructured":"H. Mhalla and M. A. Saied, Detecting continuous integration skip: A reinforcement learning-based approach, preprint, 2024, arXiv:2405.09657."},{"key":"S0218194025500792BIB034","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1548"},{"key":"S0218194025500792BIB035","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2023.110815"},{"key":"S0218194025500792BIB036","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2009.05.011"},{"key":"S0218194025500792BIB037","doi-asserted-by":"publisher","DOI":"10.1145\/3379177.3388903"},{"key":"S0218194025500792BIB038","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC57700.2023.00118"},{"issue":"5","key":"S0218194025500792BIB039","first-page":"1438","volume":"30","author":"He L.","year":"2019","journal-title":"J. Softw."},{"key":"S0218194025500792BIB040","doi-asserted-by":"publisher","DOI":"10.1145\/3361242.3361258"},{"key":"S0218194025500792BIB041","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380369"},{"key":"S0218194025500792BIB042","unstructured":"Z. Wang, V. Bapst, N. Heess, V. Mnih, R. Munos, K. Kavukcuoglu and N. De Freitas, Sample efficient actor-critic with experience replay, preprint, 2016, arXiv:1611.01224."},{"key":"S0218194025500792BIB043","unstructured":"J. Schulman, F. Wolski, P. Dhariwal, A. Radford and O. Klimov, Proximal policy optimization algorithms, preprint, 2017, arXiv:1707.06347."},{"key":"S0218194025500792BIB044","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2024.107654"},{"key":"S0218194025500792BIB045","doi-asserted-by":"publisher","DOI":"10.3390\/app15042243"},{"key":"S0218194025500792BIB046","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW58534.2023.00023"},{"key":"S0218194025500792BIB047","unstructured":"J. R. Romero, Automated machine learning for test case prioritization, Available at SSRN 4517474."},{"key":"S0218194025500792BIB048","doi-asserted-by":"publisher","DOI":"10.1145\/3644032.3644467"},{"key":"S0218194025500792BIB049","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2024.3367329"},{"key":"S0218194025500792BIB050","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"S0218194025500792BIB051","first-page":"6467","volume":"30","author":"Lopez-Paz D.","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0218194025500792BIB052","unstructured":"A. Chaudhry, M. Ranzato, M. Rohrbach and M. Elhoseiny, Efficient lifelong learning with a-gem, preprint, 2018, arXiv:1812.00420."},{"key":"S0218194025500792BIB053","unstructured":"G. Saha, I. Garg and K. Roy, Gradient projection memory for continual learning, preprint, 2021, arXiv:2103.09762."},{"key":"S0218194025500792BIB054","unstructured":"S. Lin, L. Yang, D. Fan and J. Zhang, Trgp: Trust region gradient projection for continual learning, preprint, 2022, arXiv:2202.02931."},{"volume-title":"12th Int. Conf. Learning Representations","year":"2024","author":"Qiao J.","key":"S0218194025500792BIB055"},{"key":"S0218194025500792BIB056","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.07.002"},{"key":"S0218194025500792BIB057","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2007.4362638"}],"container-title":["International Journal of Software Engineering and Knowledge Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0218194025500792","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,22]],"date-time":"2025-12-22T08:06:58Z","timestamp":1766390818000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S0218194025500792"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,11]]},"references-count":57,"journal-issue":{"issue":"02","published-print":{"date-parts":[[2026,2]]}},"alternative-id":["10.1142\/S0218194025500792"],"URL":"https:\/\/doi.org\/10.1142\/s0218194025500792","relation":{},"ISSN":["0218-1940","1793-6403"],"issn-type":[{"type":"print","value":"0218-1940"},{"type":"electronic","value":"1793-6403"}],"subject":[],"published":{"date-parts":[[2025,11,11]]}}}