{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T02:15:31Z","timestamp":1775873731726,"version":"3.50.1"},"reference-count":136,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA2","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2025,10,9]]},"abstract":"<jats:p>Runtime verification (RV) monitors program executions for conformance with formal specifications (specs). This paper concerns Monitoring-Oriented Programming (MOP), the only RV approach shown to scale to thousands of open-source GitHub projects when simultaneously monitoring passing unit tests against dozens of specs. Explicitly storing traces\u2014sequences of spec-related program events\u2014can make it easier to debug spec violations or to monitor tests against hyperproperties, which requires reasoning about sets of traces. But, most online MOP algorithms are implicit trace, i.e. they work event by event to avoid the time and space costs of storing traces. Yet, TraceMOP, the only explicit-trace online MOP algorithm, is often too slow and often fails.<\/jats:p>\n          <jats:p>We propose LazyMOP, a faster explicit-trace online MOP algorithm for RV of tests that is enabled by three simple optimizations. First, whereas all existing online MOP algorithms eagerly monitor all events as they occur, LazyMOP lazily stores only unique traces at runtime and monitors them just before the test run ends. Lazy monitoring is inspired by a recent finding: 99.87% of traces during RV of tests are duplicates. Second, to speed up trace storage, LazyMOP encodes events and their locations as integers, and amortizes the cost of looking up locations across events. Lastly, LazyMOP only synchronizes accesses to its trace store after detecting multi-threading, unlike TraceMOP\u2019s eager and wasteful synchronization of all accesses.<\/jats:p>\n          <jats:p>\n            On 179 Java open-source projects, LazyMOP is up to 4.9x faster and uses 4.8x less memory than TraceMOP, finding the same traces (modulo test non-determinism) and violations. We show LazyMOP\u2019s usefulness in the context of software evolution, where tests are re-run after each code change. LazyMOP\n            <jats:sup>\n              <jats:italic toggle=\"yes\">e<\/jats:italic>\n            <\/jats:sup>\n            optimizes LazyMOP in this context by generating fewer duplicate traces. Using unique traces from one code version, LazyMOP\n            <jats:sup>\n              <jats:italic toggle=\"yes\">e<\/jats:italic>\n            <\/jats:sup>\n            finds all pairs of method \ud835\udc5a and spec \ud835\udc60, where all traces for \ud835\udc60 in \ud835\udc5a are identical. Then, in a future version, LazyMOP\n            <jats:sup>\n              <jats:italic toggle=\"yes\">e<\/jats:italic>\n            <\/jats:sup>\n            generates and monitors only one trace of \ud835\udc60 in \ud835\udc5a. LazyMOP\n            <jats:sup>\n              <jats:italic toggle=\"yes\">e<\/jats:italic>\n            <\/jats:sup>\n            is up to 3.9x faster than LazyMOP and it speeds up two recent techniques that speed up RV during evolution by up to 4.6x with no loss in violations.\n          <\/jats:p>","DOI":"10.1145\/3763183","type":"journal-article","created":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T08:49:50Z","timestamp":1759999790000},"page":"3696-3725","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Faster Explicit-Trace Monitoring-Oriented Programming for Runtime Verification of Software Tests"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-4553-7474","authenticated-orcid":false,"given":"Kevin","family":"Guan","sequence":"first","affiliation":[{"name":"Cornell University, Ithaca, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1323-8769","authenticated-orcid":false,"given":"Marcelo","family":"d'Amorim","sequence":"additional","affiliation":[{"name":"North Carolina State University, Raleigh, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5631-4816","authenticated-orcid":false,"given":"Owolabi","family":"Legunsen","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,10,9]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Luca Aceto Antonis Achilleos Elli Anastasiadi Adrian Francalanza Daniele Gorla and Jana Wagemaker. 2024. Centralized vs decentralized monitors for hyperproperties. arXiv preprint arXiv:2405.12882.","DOI":"10.1145\/3767738"},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Abdulrahman Alshammari Paul Ammann Michael Hilton and Jonathan Bell. 2024. 230 439 Test Failures Later: An Empirical Evaluation of Flaky Failure Classifiers. In ICST.","DOI":"10.1109\/ICST60714.2024.00031"},{"key":"e_1_2_1_3_1","unstructured":"2025. ARTCAT: Autonomic Response To Cyber-Attack. https:\/\/grammatech.github.io\/prj\/artcat"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tcs.2004.11.007"},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Cyrille Artho Doron Drusinksy Allen Goldberg Klaus Havelund Mike Lowry Corina Pasareanu Grigore Ro\u015fu and Willem Visser. 2003. Experiments with test case generation and runtime analysis. In Abstract State Machines.","DOI":"10.1007\/3-540-36498-6_5"},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","unstructured":"Piramanayagam Arumuga Nainar Ting Chen Jake Rosin and Ben Liblit. 2007. Statistical debugging using compound Boolean predicates. In ISSTA.","DOI":"10.1145\/1273463.1273467"},{"key":"e_1_2_1_7_1","unstructured":"2025. ASM. http:\/\/asm.ow2.org\/"},{"key":"e_1_2_1_8_1","unstructured":"2025. Sampling CPU and HEAP profiler for Java. https:\/\/github.com\/async-profiler\/async-profiler"},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Thomas Ball. 1998. On the Limit of Control Flow Analysis for Regression Test Selection. In ISSTA.","DOI":"10.1145\/271771.271802"},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Howard Barringer Allen Goldberg Klaus Havelund and Koushik Sen. 2004. Rule-Based Runtime Verification. In VMCAI.","DOI":"10.1007\/978-3-540-24622-0_5"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1093\/logcom\/exn076"},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","unstructured":"David Basin Felix Klaedtke and Eugen Z\u0103linescu. 2017. Runtime verification of temporal properties over out-of-order data streams. In CAV.","DOI":"10.1007\/978-3-319-63387-9_18"},{"key":"e_1_2_1_13_1","volume-title":"The MonPoly Monitoring Tool.. RV-CuBES, 3","author":"Basin David A","year":"2017","unstructured":"David A Basin, Felix Klaedtke, and Eugen Zalinescu. 2017. The MonPoly Monitoring Tool.. RV-CuBES, 3 (2017)."},{"key":"e_1_2_1_14_1","volume-title":"Efficient decentralized LTL monitoring framework using tableau technique. TECS, 18, 5s","author":"Bataineh Omar","year":"2019","unstructured":"Omar Bataineh, David S Rosenblum, and Mark Reynolds. 2019. Efficient decentralized LTL monitoring framework using tableau technique. TECS, 18, 5s (2019)."},{"key":"e_1_2_1_15_1","unstructured":"Jon Bell Owolabi Legunsen Michael Hilton Lamyaa Eloussi Tifany Yung and Darko Marinov. 2018. DeFlaker: Automatically Detecting Flaky Tests. In ICSE."},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Raven Beutner Bernd Finkbeiner Hadar Frenkel and Niklas Metzger. 2024. Monitoring second-order hyperproperties. arXiv preprint arXiv:2404.09652.","DOI":"10.1007\/978-3-031-37703-7_15"},{"key":"e_1_2_1_17_1","volume-title":"Rosenblum","author":"Bible John","year":"2001","unstructured":"John Bible, Gregg Rothermel, and David S. Rosenblum. 2001. A Comparative Study of Coarse- and Fine-grained Safe Regression Test-selection Techniques. TOSEM, 10, 2 (2001)."},{"key":"e_1_2_1_18_1","volume-title":"Naeem","author":"Bodden Eric","year":"2007","unstructured":"Eric Bodden, Laurie Hendren, Patrick Lam, Ond\u0159ej Lhot\u00e1k, and Nomair A. Naeem. 2007. Collaborative Runtime Verification with Tracematches. In RV."},{"key":"e_1_2_1_19_1","doi-asserted-by":"crossref","unstructured":"Eric Bodden Patrick Lam and Laurie Hendren. 2008. Finding Programming Errors Earlier by Evaluating Runtime Monitors Ahead-of-time. In FSE.","DOI":"10.1145\/1453101.1453109"},{"key":"e_1_2_1_20_1","volume-title":"Time-triggered runtime verification. 43, 1","author":"Bonakdarpour Borzoo","year":"2013","unstructured":"Borzoo Bonakdarpour, Samaneh Navabpour, and Sebastian Fischmeister. 2013. Time-triggered runtime verification. 43, 1 (2013)."},{"key":"e_1_2_1_21_1","volume-title":"Automating Regression Test Selection Based on UML Designs. IST, 51, 1","author":"Briand Lionel","year":"2009","unstructured":"Lionel Briand, Yvan Labiche, and Siyuan He. 2009. Automating Regression Test Selection Based on UML Designs. IST, 51, 1 (2009)."},{"key":"e_1_2_1_22_1","unstructured":"2024. Known bug in eMOP. https:\/\/github.com\/SoftEngResearch\/emop\/issues\/97"},{"key":"e_1_2_1_23_1","doi-asserted-by":"crossref","unstructured":"Ian Cassar Adrian Francalanza Luca Aceto and Anna Ing\u00f3lfsd\u00f3ttir. 2017. A survey of runtime monitoring instrumentation techniques. arXiv preprint arXiv:1708.07229.","DOI":"10.4204\/EPTCS.254.2"},{"key":"e_1_2_1_24_1","doi-asserted-by":"crossref","unstructured":"Marek Chalupa and Thomas A Henzinger. 2023. Monitoring hyperproperties with prefix transducers. In RV.","DOI":"10.1007\/978-3-031-44267-4_9"},{"key":"e_1_2_1_25_1","doi-asserted-by":"crossref","unstructured":"Feng Chen Marcelo d\u2019Amorim and Grigore Ro\u015fu. 2004. A Formal Monitoring-Based Framework for Software Development and Analysis. In ICFEM.","DOI":"10.1007\/978-3-540-30482-1_31"},{"key":"e_1_2_1_26_1","volume-title":"Dongyun Jin, and Grigore Ro\u015fu.","author":"Chen Feng","year":"2009","unstructured":"Feng Chen, Patrick O\u2019Neil Meredith, Dongyun Jin, and Grigore Ro\u015fu. 2009. Efficient formalism-independent monitoring of parametric properties. In ASE."},{"key":"e_1_2_1_27_1","volume-title":"Dongyun Jin, and Grigore Ro\u015fu.","author":"Chen Feng","year":"2009","unstructured":"Feng Chen, Patrick O\u2019Neil Meredith, Dongyun Jin, and Grigore Ro\u015fu. 2009. Efficient Formalism-Independent Monitoring of Parametric Properties. Computer Science Dept., UIUC."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1297027.1297069"},{"key":"e_1_2_1_29_1","doi-asserted-by":"crossref","unstructured":"Feng Chen and Grigore Ro\u015fu. 2003. Towards Monitoring-Oriented Programming: A paradigm combining specification and implementation. In RV.","DOI":"10.1016\/S1571-0661(04)81045-4"},{"key":"e_1_2_1_30_1","doi-asserted-by":"crossref","unstructured":"Feng Chen and Grigore Ro\u015fu. 2008. Parametric trace slicing and monitoring. Computer Science Dept. UIUC.","DOI":"10.1007\/978-3-642-00768-2_23"},{"key":"e_1_2_1_31_1","doi-asserted-by":"crossref","unstructured":"Feng Chen and Grigore Ro\u015fu. 2009. Parametric trace slicing and monitoring. In TACAS.","DOI":"10.1007\/978-3-642-00768-2_23"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.3233\/JCS-2009-0393"},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Christian Colombo Gordon J Pace and Patrick Abela. 2009. Offline runtime verification with real-time properties: A case study.","DOI":"10.1007\/978-3-642-04368-0_10"},{"key":"e_1_2_1_34_1","doi-asserted-by":"crossref","unstructured":"Marcelo d\u2019Amorim and Klaus Havelund. 2005. Event-based runtime verification of Java programs. In WODA.","DOI":"10.1145\/1083246.1083249"},{"key":"e_1_2_1_35_1","unstructured":"Ben d\u2019Angelo Sriram Sankaranarayanan C\u00e9sar S\u00e1nchez Will Robinson Bernd Finkbeiner Henny B Sipma Sandeep Mehrotra and Zohar Manna. 2005. LOLA: runtime monitoring of synchronous systems. In Temporal Representation and Reasoning."},{"key":"e_1_2_1_36_1","doi-asserted-by":"crossref","unstructured":"Normann Decker Jannis Harder Torben Scheffel Malte Schmitz and Daniel Thoma. 2016. Runtime Monitoring with Union-Find Structures. In TACAS.","DOI":"10.1007\/978-3-662-49674-9_54"},{"key":"e_1_2_1_37_1","doi-asserted-by":"crossref","unstructured":"Normann Decker Martin Leucker and Daniel Thoma. 2013. jUnit RV\u2013adding runtime verification to jUnit. In FM.","DOI":"10.1007\/978-3-642-38088-4_34"},{"key":"e_1_2_1_38_1","doi-asserted-by":"crossref","unstructured":"Matthew B. Dwyer Rahul Purandare and Suzette Person. 2010. Runtime Verification in Context: Can Optimizing Error Detection Improve Fault Diagnosis? In RV.","DOI":"10.1007\/978-3-642-16612-9_4"},{"key":"e_1_2_1_39_1","volume-title":"A Systematic Review on Regression Test Selection Techniques. IST, 52, 1","author":"Engstr\u00f6m Emelie","year":"2010","unstructured":"Emelie Engstr\u00f6m, Per Runeson, and Mats Skoglund. 2010. A Systematic Review on Regression Test Selection Techniques. IST, 52, 1 (2010)."},{"key":"e_1_2_1_40_1","doi-asserted-by":"crossref","unstructured":"Emelie Engstr\u00f6m Mats Skoglund and Per Runeson. 2008. Empirical evaluations of regression test selection techniques: A systematic review. In ESEM.","DOI":"10.1145\/1414004.1414011"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/SECPRI.2000.848461"},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Bernd Finkbeiner Christopher Hahn Marvin Stenger and Leander Tentrup. 2018. RVHyper: A runtime verification tool for temporal hyperproperties. In TACAS.","DOI":"10.1007\/978-3-319-89963-3_11"},{"key":"e_1_2_1_43_1","volume-title":"Monitoring hyperproperties. FMSD, 54, 3","author":"Finkbeiner Bernd","year":"2019","unstructured":"Bernd Finkbeiner, Christopher Hahn, Marvin Stenger, and Leander Tentrup. 2019. Monitoring hyperproperties. FMSD, 54, 3 (2019)."},{"key":"e_1_2_1_44_1","doi-asserted-by":"crossref","unstructured":"Vojt\u011bch Forejt Marta Kwiatkowska David Parker Hongyang Qu and Mateusz Ujma. 2012. Incremental Runtime Verification of Probabilistic Systems. In RV.","DOI":"10.1007\/978-3-642-35632-2_30"},{"key":"e_1_2_1_45_1","doi-asserted-by":"crossref","unstructured":"Mark Gabel and Zhendong Su. 2012. Testing Mined Specifications. In FSE.","DOI":"10.1145\/2393596.2393598"},{"key":"e_1_2_1_46_1","volume-title":"Ekstazi: Lightweight Test Selection. In ICSE Demo.","author":"Gligoric Milos","year":"2015","unstructured":"Milos Gligoric, Lamyaa Eloussi, and Darko Marinov. 2015. Ekstazi: Lightweight Test Selection. In ICSE Demo."},{"key":"e_1_2_1_47_1","doi-asserted-by":"crossref","unstructured":"Milos Gligoric Lamyaa Eloussi and Darko Marinov. 2015. Practical regression test selection with dynamic file dependencies. In ISSTA.","DOI":"10.1145\/2771783.2771784"},{"key":"e_1_2_1_48_1","doi-asserted-by":"crossref","unstructured":"Milos Gligoric Rupak Majumdar Rohan Sharma Lamyaa Eloussi and Darko Marinov. 2014. Regression Test Selection for Distributed Software Histories. In CAV.","DOI":"10.1007\/978-3-319-08867-9_19"},{"key":"e_1_2_1_49_1","doi-asserted-by":"crossref","unstructured":"Milos Gligoric Stas Negara Owolabi Legunsen and Darko Marinov. 2014. An empirical evaluation and comparison of manual and automated test selection. In ASE.","DOI":"10.1145\/2642937.2643019"},{"key":"e_1_2_1_50_1","volume-title":"Seyed Armin Vakil Ghahani, and Manos Kapritsos","author":"Goldweber Eli","year":"2024","unstructured":"Eli Goldweber, Weixin Yu, Seyed Armin Vakil Ghahani, and Manos Kapritsos. 2024. IronSpec: Increasing the Reliability of Formal Specifications. In OSDI."},{"key":"e_1_2_1_51_1","doi-asserted-by":"crossref","unstructured":"Kevin Guan and Owolabi Legunsen. 2024. An In-depth Study of Runtime Verification Overheads during Software Testing. In ISSTA.","DOI":"10.1145\/3650212.3680400"},{"key":"e_1_2_1_52_1","doi-asserted-by":"crossref","unstructured":"Kevin Guan and Owolabi Legunsen. 2025. Instrumentation-Driven Evolution-Aware Runtime Verification. In ICSE.","DOI":"10.1109\/ICSE55347.2025.00099"},{"key":"e_1_2_1_53_1","doi-asserted-by":"crossref","unstructured":"Kevin Guan and Owolabi Legunsen. 2025. TraceMOP: An Explicit-Trace Runtime Verification Tool for Java. In FSE Demo.","DOI":"10.1145\/3696630.3728613"},{"key":"e_1_2_1_54_1","doi-asserted-by":"crossref","unstructured":"Alex Gyori Ben Lambeth August Shi Owolabi Legunsen and Darko Marinov. 2016. NonDex: A tool for detecting and debugging wrong assumptions on Java API specifications. In FSE Demo.","DOI":"10.1145\/2950290.2983932"},{"key":"e_1_2_1_55_1","doi-asserted-by":"crossref","unstructured":"Alex Gyori Owolabi Legunsen Farah Hariri and Darko Marinov. 2018. Evaluating regression test selection opportunities in a very large open-source ecosystem. In ISSRE.","DOI":"10.1109\/ISSRE.2018.00022"},{"key":"e_1_2_1_56_1","unstructured":"Christopher Hahn. 2019. Algorithms for monitoring hyperproperties. In RV."},{"key":"e_1_2_1_57_1","unstructured":"Christopher Hahn Marvin Stenger and Leander Tentrup. 2019. Constraint-based monitoring of hyperproperties. In TACAS."},{"key":"e_1_2_1_58_1","doi-asserted-by":"crossref","unstructured":"Mary Jean Harrold James A. Jones Tongyu Li Donglin Liang Alessandro Orso Maikel Pennings Saurabh Sinha S. Alexander Spoon and Ashish Gujarathi. 2001. Regression Test Selection for Java Software. In OOPSLA.","DOI":"10.1145\/504282.504305"},{"key":"e_1_2_1_59_1","doi-asserted-by":"crossref","unstructured":"Negar Hashemi Amjed Tahir Shawn Rasheed August Shi and Rachel Blagojevic. 2025. Detecting and evaluating order-dependent flaky tests in JavaScript. In ICST.","DOI":"10.1109\/ICST62969.2025.10988962"},{"key":"e_1_2_1_60_1","doi-asserted-by":"crossref","unstructured":"Matthias Hauswirth and Trishul M Chilimbi. 2004. Low-overhead memory leak detection using adaptive statistical profiling. In ASPLOS.","DOI":"10.1145\/1024393.1024412"},{"key":"e_1_2_1_61_1","doi-asserted-by":"crossref","unstructured":"Klaus Havelund Doron Peled and Dogan Ulus. 2017. First order temporal logic monitoring with BDDs. In FMCAD.","DOI":"10.23919\/FMCAD.2017.8102249"},{"key":"e_1_2_1_62_1","unstructured":"Klaus Havelund and Grigore Ro\u015fu. 2001. Monitoring Programs Using Rewriting. In ASE."},{"key":"e_1_2_1_63_1","unstructured":"Hsi-Ming Ho Jo\u00ebl Ouaknine and James Worrell. 2014. Online monitoring of metric temporal logic. In RV."},{"key":"e_1_2_1_64_1","unstructured":"2009. JaCoCo Java Code Coverage Library. http:\/\/www.eclemma.org\/jacoco\/"},{"key":"e_1_2_1_65_1","unstructured":"2025. The For-Each Loop. https:\/\/docs.oracle.com\/javase\/8\/docs\/technotes\/guides\/language\/foreach.html"},{"key":"e_1_2_1_66_1","unstructured":"2025. JavaParser - Home. https:\/\/javaparser.org"},{"key":"e_1_2_1_67_1","doi-asserted-by":"crossref","unstructured":"Omar Javed and Walter Binder. 2018. Large-Scale Evaluation of the Efficiency of Runtime-Verification Tools in the Wild. In APSEC.","DOI":"10.1109\/APSEC.2018.00091"},{"key":"e_1_2_1_68_1","volume-title":"Dennis Griffith, and Grigore Ro\u015fu.","author":"Jin Dongyun","year":"2011","unstructured":"Dongyun Jin, Patrick O\u2019Neil Meredith, Dennis Griffith, and Grigore Ro\u015fu. 2011. Garbage Collection for Monitoring Parametric Properties. In PLDI."},{"key":"e_1_2_1_69_1","volume-title":"Choonghwan Lee, and Grigore Ro\u015fu.","author":"Jin Dongyun","year":"2012","unstructured":"Dongyun Jin, Patrick O\u2019Neil Meredith, Choonghwan Lee, and Grigore Ro\u015fu. 2012. JavaMOP: Efficient Parametric Runtime Monitoring Framework. In ICSE Demo."},{"key":"e_1_2_1_70_1","volume-title":"Jones and Mary Jean Harrold","author":"James","year":"2003","unstructured":"James A. Jones and Mary Jean Harrold. 2003. Test-Suite Reduction and Prioritization for Modified Condition\/Decision Coverage. TSE, 29, 3 (2003)."},{"key":"e_1_2_1_71_1","doi-asserted-by":"crossref","unstructured":"Hannes Kallwies Martin Leucker Malte Schmitz Albert Schulz Daniel Thoma and Alexander Weiss. 2022. TeSSLa\u2013an ecosystem for runtime verification. In RV.","DOI":"10.1007\/978-3-031-17196-3_20"},{"key":"e_1_2_1_72_1","volume-title":"Limiting Transitive Closure for Static Regression Test Selection approaches. Master\u2019s thesis","author":"Karlsson Henrik","unstructured":"Henrik Karlsson. 2019. Limiting Transitive Closure for Static Regression Test Selection approaches. Master\u2019s thesis. KTH Royal Institute of Technology. Sweden."},{"key":"e_1_2_1_73_1","doi-asserted-by":"crossref","unstructured":"Gregor Kiczales Erik Hilsdale Jim Hugunin Mik Kersten Jeffrey Palm and William G Griswold. 2001. An overview of AspectJ. In ECOOP.","DOI":"10.1007\/3-540-45337-7_18"},{"key":"e_1_2_1_74_1","unstructured":"Moonjoo Kim Mahesh Viswanathan Hanene Ben-Abdallah Sampath Kannan Insup Lee and Oleg Sokolsky. 1999. Formally specified monitoring of temporal properties. In ECRTS."},{"key":"e_1_2_1_75_1","doi-asserted-by":"crossref","unstructured":"Wing Lam Reed Oei August Shi Darko Marinov and Tao Xie. 2019. iDFlakies: A framework for detecting and partially classifying flaky tests. In ICST.","DOI":"10.1109\/ICST.2019.00038"},{"key":"e_1_2_1_76_1","volume-title":"Patrick O\u2019Neil Meredith, and Grigore Ro\u015fu","author":"Lee Choonghwan","year":"2012","unstructured":"Choonghwan Lee, Dongyun Jin, Patrick O\u2019Neil Meredith, and Grigore Ro\u015fu. 2012. Towards Categorizing and Formalizing the JDK API. Computer Science Dept., UIUC."},{"key":"e_1_2_1_77_1","article-title":"How Effective are Existing Java API Specifications for Finding Bugs During Runtime Verification","volume":"26","author":"Legunsen Owolabi","year":"2019","unstructured":"Owolabi Legunsen, Nader Al Awar, Xinyue Xu, Wajih Ul Hassan, Grigore Ro\u015fu, and Darko Marinov. 2019. How Effective are Existing Java API Specifications for Finding Bugs During Runtime Verification? ASE Journal, 26, 4 (2019).","journal-title":"ASE Journal"},{"key":"e_1_2_1_78_1","doi-asserted-by":"crossref","unstructured":"Owolabi Legunsen Farah Hariri August Shi Yafeng Lu Lingming Zhang and Darko Marinov. 2016. An Extensive Study of Static Regression Test Selection in Modern Software Evolution. In FSE.","DOI":"10.1145\/2950290.2950361"},{"key":"e_1_2_1_79_1","volume-title":"Xinyue Xu, Grigore Ro\u015fu, and Darko Marinov.","author":"Legunsen Owolabi","year":"2016","unstructured":"Owolabi Legunsen, Wajih Ul Hassan, Xinyue Xu, Grigore Ro\u015fu, and Darko Marinov. 2016. How good are the specs? A study of the bug-finding effectiveness of existing Java API specifications. In ASE."},{"key":"e_1_2_1_80_1","doi-asserted-by":"crossref","unstructured":"Owolabi Legunsen Darko Marinov and Grigore Ro\u015fu. 2015. Evolution-aware monitoring-oriented programming. In ICSE NIER.","DOI":"10.1109\/ICSE.2015.206"},{"key":"e_1_2_1_81_1","volume-title":"STARTS: STAtic Regression Test Selection. In ASE.","author":"Legunsen Owolabi","year":"2017","unstructured":"Owolabi Legunsen, August Shi, and Darko Marinov. 2017. STARTS: STAtic Regression Test Selection. In ASE."},{"key":"e_1_2_1_82_1","doi-asserted-by":"crossref","unstructured":"Owolabi Legunsen Yi Zhang Milica Hadzi-Tanovic Grigore Ro\u015fu and Darko Marinov. 2019. Techniques for Evolution-Aware Runtime Verification. In ICST.","DOI":"10.1109\/ICST.2019.00037"},{"key":"e_1_2_1_83_1","doi-asserted-by":"crossref","unstructured":"Yu Liu Jiyang Zhang Pengyu Nie Milos Gligoric and Owolabi Legunsen. 2023. More precise regression test selection via reasoning about semantics-modifying changes. In ISSTA.","DOI":"10.1145\/3597926.3598086"},{"key":"e_1_2_1_84_1","unstructured":"Matt Luckcuck. 2020. Offline Runtime Verification of Safety Requirements using CSP. arXiv preprint arXiv:2007.03522."},{"key":"e_1_2_1_85_1","volume-title":"EALRTS: A predictive regression test selection tool. Master\u2019s thesis","author":"Lundsten Erik","year":"2019","unstructured":"Erik Lundsten. 2019. EALRTS: A predictive regression test selection tool. Master\u2019s thesis. KTH Royal Institute of Technology. Sweden."},{"key":"e_1_2_1_86_1","unstructured":"Qingzhou Luo Farah Hariri Lamyaa Eloussi and Darko Marinov. 2014. An empirical analysis of flaky tests. In FSE."},{"key":"e_1_2_1_87_1","volume-title":"Traian Florin \u015eerb\u0103nu\u0163\u0103, and Grigore Ro\u015fu.","author":"Luo Qingzhou","year":"2014","unstructured":"Qingzhou Luo, Yi Zhang, Choonghwan Lee, Dongyun Jin, Patrick O\u2019Neil Meredith, Traian Florin \u015eerb\u0103nu\u0163\u0103, and Grigore Ro\u015fu. 2014. RV-Monitor: Efficient Parametric Runtime Verification with Simultaneous Properties. In RV."},{"key":"e_1_2_1_88_1","doi-asserted-by":"crossref","unstructured":"Mateusz Machalica Alex Samylkin Meredith Porth and Satish Chandra. 2019. Predictive test selection. In ICSE SEIP.","DOI":"10.1109\/ICSE-SEIP.2019.00018"},{"key":"e_1_2_1_89_1","doi-asserted-by":"crossref","unstructured":"Daniel Marino Madanlal Musuvathi and Satish Narayanasamy. 2009. LiteRace: Effective sampling for lightweight data-race detection. In PLDI.","DOI":"10.1145\/1542476.1542491"},{"key":"e_1_2_1_90_1","doi-asserted-by":"crossref","unstructured":"Michael Martin Benjamin Livshits and Monica S Lam. 2005. Finding application errors and security flaws using PQL: a program query language. In OOPSLA.","DOI":"10.1145\/1094811.1094840"},{"key":"e_1_2_1_91_1","volume-title":"An overview of the MOP runtime verification framework. IJSTTT, 14, 3","author":"O\u2019Neil Meredith Patrick","year":"2012","unstructured":"Patrick O\u2019Neil Meredith, Dongyun Jin, Dennis Griffith, Feng Chen, and Grigore Ro\u015fu. 2012. An overview of the MOP runtime verification framework. IJSTTT, 14, 3 (2012)."},{"key":"e_1_2_1_92_1","doi-asserted-by":"crossref","unstructured":"Breno Miranda Igor Lima Owolabi Legunsen and Marcelo d\u2019Amorim. 2020. Prioritizing Runtime Verification Violations. In ICST.","DOI":"10.1109\/ICST46399.2020.00038"},{"key":"e_1_2_1_93_1","volume-title":"Borzoo Bonakdarpour, and Sebastian Fischmeister.","author":"Navabpour Samaneh","year":"2011","unstructured":"Samaneh Navabpour, Chun Wah Wallace Wu, Borzoo Bonakdarpour, and Sebastian Fischmeister. 2011. Efficient Techniques for Near-Optimal Instrumentation in Time-Triggered Runtime Verification. In RV."},{"key":"e_1_2_1_94_1","unstructured":"2014. java.lang.instrument. http:\/\/docs.oracle.com\/javase\/8\/docs\/api\/java\/lang\/instrument\/package-summary.html"},{"key":"e_1_2_1_95_1","volume-title":"A survey of flaky tests. TOSEM, 31, 1","author":"Parry Owain","year":"2021","unstructured":"Owain Parry, Gregory M Kapfhammer, Michael Hilton, and Phil McMinn. 2021. A survey of flaky tests. TOSEM, 31, 1 (2021)."},{"key":"e_1_2_1_96_1","doi-asserted-by":"crossref","unstructured":"Srinivas Pinisetty Gerardo Schneider and David Sands. 2018. Runtime verification of hyperproperties for deterministic programs. In FMSD.","DOI":"10.1145\/3193992.3193995"},{"key":"e_1_2_1_97_1","unstructured":"2011. PIT Mutation Testing. http:\/\/pitest.org\/"},{"key":"e_1_2_1_98_1","doi-asserted-by":"crossref","unstructured":"Rahul Purandare Matthew B. Dwyer and Sebastian Elbaum. 2010. Monitor Optimization via Stutter-equivalent Loop Transformation. In OOPSLA.","DOI":"10.1145\/1869459.1869483"},{"key":"e_1_2_1_99_1","doi-asserted-by":"crossref","unstructured":"Rahul Purandare Matthew B. Dwyer and Sebastian Elbaum. 2013. Optimizing Monitoring of Finite State Properties Through Monitor Compaction. In ISSTA.","DOI":"10.1145\/2483760.2483762"},{"key":"e_1_2_1_100_1","doi-asserted-by":"crossref","unstructured":"Shanto Rahman Aaron Massey Wing Lam August Shi and Jonathan Bell. 2024. Automatically reproducing timing-dependent flaky-test failures. In ICST.","DOI":"10.1109\/ICST60714.2024.00032"},{"key":"e_1_2_1_101_1","volume-title":"Helena Cuenca Cruz, and David Rydeheard","author":"Reger Giles","year":"2015","unstructured":"Giles Reger, Helena Cuenca Cruz, and David Rydeheard. 2015. MarQ: Monitoring at Runtime with QEA. In TACAS."},{"key":"e_1_2_1_102_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-47169-3_25"},{"key":"e_1_2_1_103_1","unstructured":"Adam Renberg. 2014. Test-inspired runtime verification: Using a unit test-like specification syntax for runtime verification. Master\u2019s thesis. KTH. Sweden."},{"key":"e_1_2_1_104_1","volume-title":"Automated API Property Inference Techniques. TSE, 39, 5","author":"Robillard Martin P.","year":"2013","unstructured":"Martin P. Robillard, Eric Bodden, David Kawrykow, Mira Mezini, and Tristan Ratchford. 2013. Automated API Property Inference Techniques. TSE, 39, 5 (2013)."},{"key":"e_1_2_1_105_1","volume-title":"Semantics and algorithms for parametric monitoring. LICS, 8","author":"Ro\u015fu Grigore","year":"2012","unstructured":"Grigore Ro\u015fu and Feng Chen. 2012. Semantics and algorithms for parametric monitoring. LICS, 8 (2012)."},{"key":"e_1_2_1_106_1","unstructured":"Gregg Rothermel and Mary Jean Harrold. 1993. A safe efficient algorithm for regression test selection. In ICSM."},{"key":"e_1_2_1_107_1","volume-title":"Analyzing regression test selection techniques. TSE, 22, 8","author":"Rothermel Gregg","year":"1996","unstructured":"Gregg Rothermel and Mary Jean Harrold. 1996. Analyzing regression test selection techniques. TSE, 22, 8 (1996)."},{"key":"e_1_2_1_108_1","volume-title":"A safe, efficient regression test selection technique. TOSEM, 6, 2","author":"Rothermel Gregg","year":"1997","unstructured":"Gregg Rothermel and Mary Jean Harrold. 1997. A safe, efficient regression test selection technique. TOSEM, 6, 2 (1997)."},{"key":"e_1_2_1_109_1","volume-title":"Empirical studies of a safe regression test selection technique. TOSEM, 24, 6","author":"Rothermel Gregg","year":"1998","unstructured":"Gregg Rothermel and Mary Jean Harrold. 1998. Empirical studies of a safe regression test selection technique. TOSEM, 24, 6 (1998)."},{"key":"e_1_2_1_110_1","volume-title":"Jeffery von Ronne, and Christie Hong.","author":"Rothermel Gregg","year":"2002","unstructured":"Gregg Rothermel, Mary Jean Harrold, Jeffery von Ronne, and Christie Hong. 2002. Empirical Studies of Test-Suite Reduction. STVR, 12, 4 (2002)."},{"key":"e_1_2_1_111_1","unstructured":"August Shi Alex Gyori Milos Gligoric Andrey Zaytsev and Darko Marinov. 2014. Balancing trade-offs in test-suite reduction. In FSE."},{"key":"e_1_2_1_112_1","unstructured":"August Shi Alex Gyori Owolabi Legunsen and Darko Marinov. 2016. Detecting Assumptions on Deterministic Implementations of Non-deterministic Specifications. In ICST."},{"key":"e_1_2_1_113_1","unstructured":"August Shi Alex Gyori Suleman Mahmood Peiyuan Zhao and Darko Marinov. 2018. Evaluating Test-suite Reduction in Real Software Evolution. In ISSTA."},{"key":"e_1_2_1_114_1","unstructured":"August Shi Milica Hadzi-Tanovic Lingming Zhang Darko Marinov and Owolabi Legunsen. 2019. Reflection-Aware Static Regression Test Selection. In OOPSLA."},{"key":"e_1_2_1_115_1","unstructured":"August Shi Wing Lam Reed Oei Tao Xie and Darko Marinov. 2019. iFixFlakies: A framework for automatically fixing order-dependent flaky tests. In FSE."},{"key":"e_1_2_1_116_1","unstructured":"August Shi Tifany Yung Alex Gyori and Darko Marinov. 2015. Comparing and combining test-suite reduction and regression test selection. In FSE."},{"key":"e_1_2_1_117_1","unstructured":"August Shi Peiyuan Zhao and Darko Marinov. 2019. Understanding and Improving Regression Test Selection in Continuous Integration. In ISSRE."},{"key":"e_1_2_1_118_1","volume-title":"The effects of computational resources on flaky tests. TSE, 50, 12","author":"Silva Denini","year":"2024","unstructured":"Denini Silva, Martin Gruber, Satyajit Gokhale, Ellen Arteca, Alexi Turcotte, Marcelo d\u2019Amorim, Wing Lam, Stefan Winter, and Jonathan Bell. 2024. The effects of computational resources on flaky tests. TSE, 50, 12 (2024)."},{"key":"e_1_2_1_119_1","unstructured":"Mats Skoglund and Per Runeson. 2005. A case study of the class firewall regression test selection technique on a large scale distributed software system. In ESEM."},{"key":"e_1_2_1_120_1","volume-title":"Improving class firewall regression test selection by removing the class firewall. JSEKE, 17, 3","author":"Skoglund Mats","year":"2007","unstructured":"Mats Skoglund and Per Runeson. 2007. Improving class firewall regression test selection by removing the class firewall. JSEKE, 17, 3 (2007)."},{"key":"e_1_2_1_121_1","doi-asserted-by":"crossref","unstructured":"Quinten David Soetens Serge Demeyer and Andy Zaidman. 2013. Change-Based Test Selection in the Presence of Developer Tests. In CSMR.","DOI":"10.1109\/CSMR.2013.20"},{"key":"e_1_2_1_122_1","doi-asserted-by":"crossref","unstructured":"Chukri Soueidi and Yli\u00e8s Falcone. 2022. Residual runtime verification via reachability analysis. In VSTTE. 148\u2013166.","DOI":"10.1007\/978-3-031-25803-9_9"},{"key":"e_1_2_1_123_1","volume-title":"Typestate: A programming language concept for enhancing software reliability. TSE.","author":"Strom Robert E","year":"1986","unstructured":"Robert E Strom and Shaula Yemini. 1986. Typestate: A programming language concept for enhancing software reliability. TSE."},{"key":"e_1_2_1_124_1","doi-asserted-by":"crossref","unstructured":"Leopoldo Teixeira Breno Miranda Henrique Reb\u00ealo and Marcelo d\u2019Amorim. 2021. Demystifying the challenges of formally specifying API properties for runtime verification. In ICST.","DOI":"10.1109\/ICST49551.2021.00020"},{"key":"e_1_2_1_125_1","unstructured":"2024. TraceMOP: A Trace-Aware Runtime Verification Tool for Java.. https:\/\/github.com\/SoftEngResearch\/tracemop"},{"key":"e_1_2_1_126_1","volume-title":"Embury","author":"Willmor David","year":"2005","unstructured":"David Willmor and Suzanne M. Embury. 2005. A Safe Regression Test Selection Technique for Database Driven Applications. In ICSM."},{"key":"e_1_2_1_127_1","unstructured":"Chun Wah Wallace Wu Deepak Kumar Borzoo Bonakdarpour and Sebastian Fischmeister. 2013. Reducing Monitoring Overhead by Integrating Event- and Time-Triggered Techniques. In RV."},{"key":"e_1_2_1_128_1","unstructured":"Guoqing Xu and Atanas Rountev. 2007. Regression test selection for AspectJ software. In ICSE."},{"key":"e_1_2_1_129_1","volume-title":"A Method for Selecting Regression Test Cases Based on Software Changes and Software Faults. Master\u2019s thesis","author":"Yilmaz Ugur","unstructured":"Ugur Yilmaz. 2019. A Method for Selecting Regression Test Cases Based on Software Changes and Software Faults. Master\u2019s thesis. Hacettepe University. Turkey."},{"key":"e_1_2_1_130_1","volume-title":"Regression Testing Minimization, Selection and Prioritization: A Survey. STVR, 22, 2","author":"Yoo Shin","year":"2012","unstructured":"Shin Yoo and Mark Harman. 2012. Regression Testing Minimization, Selection and Prioritization: A Survey. STVR, 22, 2 (2012)."},{"key":"e_1_2_1_131_1","doi-asserted-by":"crossref","unstructured":"Ayaka Yorihiro Pengyue Jiang Valeria Marques Benjamin Carleton and Owolabi Legunsen. 2023. eMOP: A Maven Plugin for Evolution-Aware Runtime Verification. In RV.","DOI":"10.1007\/978-3-031-44267-4_20"},{"key":"e_1_2_1_132_1","doi-asserted-by":"crossref","unstructured":"Guofeng Zhang Luyao Liu Zhenbang Chen and Ji Wang. 2024. Hybrid Regression Test Selection by Integrating File and Method Dependences. In ASE.","DOI":"10.1145\/3691620.3695525"},{"key":"e_1_2_1_133_1","doi-asserted-by":"crossref","unstructured":"Jiyang Zhang Yu Liu Milos Gligoric Owolabi Legunsen and August Shi. 2022. Comparing and combining analysis-based and learning-based regression test selection. In AST.","DOI":"10.1145\/3524481.3527230"},{"key":"e_1_2_1_134_1","doi-asserted-by":"crossref","unstructured":"Lingming Zhang. 2018. Hybrid Regression Test Selection. In ICSE.","DOI":"10.1145\/3180155.3180198"},{"key":"e_1_2_1_135_1","doi-asserted-by":"crossref","unstructured":"Lingming Zhang Darko Marinov Lu Zhang and Sarfraz Khurshid. 2011. An Empirical Study of JUnit Test-Suite Reduction. In ISSRE.","DOI":"10.1109\/ISSRE.2011.26"},{"key":"e_1_2_1_136_1","unstructured":"Chenguang Zhu Owolabi Legunsen August Shi and Milos Gligoric. 2019. A framework for checking regression test selection tools. In ICSE."}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3763183","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:45:58Z","timestamp":1760031958000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3763183"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,9]]},"references-count":136,"journal-issue":{"issue":"OOPSLA2","published-print":{"date-parts":[[2025,10,9]]}},"alternative-id":["10.1145\/3763183"],"URL":"https:\/\/doi.org\/10.1145\/3763183","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,9]]},"assertion":[{"value":"2025-03-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-12","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}