{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T16:55:29Z","timestamp":1781974529880,"version":"3.54.5"},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T00:00:00Z","timestamp":1713830400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Australian Research Council (ARC) Discovery Project","award":["DP200102299"],"award-info":[{"award-number":["DP200102299"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Internet Things"],"published-print":{"date-parts":[[2024,5,31]]},"abstract":"<jats:p>Real-time applications increasingly rely on context information to provide relevant and dependable features. Context queries require large-scale retrieval, inferencing, aggregation, and delivery of context using only limited computing resources, especially in a distributed environment. If this is slow, inconsistent, and too expensive to access context information, the dependability and relevancy of real-time applications may fail to exist. This paper argues, transiency of context (i.e., the limited validity period), variations in the features of context query loads (e.g., the request rate, different Quality of Service (QoS), and Quality of Context (QoC) requirements), and lack of prior knowledge about context to make near real-time adaptations as fundamental challenges that need to be addressed to overcome these shortcomings. Hence, we propose a performance metric driven reinforcement learning based adaptive context caching approach aiming to maximize both cost- and performance-efficiency for middleware-based Context Management Systems (CMSs). Although context-aware caching has been thoroughly investigated in the literature, our approach is novel because existing techniques are not fully applicable to caching context due to (i) the underlying fundamental challenges and (ii) not addressing the limitations hindering dependability and consistency of context. Unlike previously tested modes of CMS operations and traditional data caching techniques, our approach can provide real-time pervasive applications with lower cost, faster, and fresher high quality context information. Compared to existing context-aware data caching algorithms, our technique is bespoken for caching context information, which is different from traditional data. We also show that our full-cycle context lifecycle-based approach can maximize both cost- and performance-efficiency while maintaining adequate QoC solely based on real-time performance metrics and our heuristic techniques without depending on any previous knowledge about the context, variations in query features, or quality demands, unlike any previous work. We demonstrate using a real world inspired scenario and a prototype middleware based CMS integrated with our adaptive context caching approach that we have implemented, how realtime applications that are 85% faster can be more relevant and dependable to users, while costing 60.22% less than using existing techniques to access context information. Our model is also at least twice as fast and more flexible to adapt compared to existing benchmarks even under uncertainty and lack of prior knowledge about context, transiency, and variable context query loads.<\/jats:p>","DOI":"10.1145\/3648571","type":"journal-article","created":{"date-parts":[[2024,2,16]],"date-time":"2024-02-16T12:38:02Z","timestamp":1708087082000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Reinforcement Learning Based Approaches to Adaptive Context Caching in Distributed Context Management Systems"],"prefix":"10.1145","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9287-3420","authenticated-orcid":false,"given":"Shakthi","family":"Weerasinghe","sequence":"first","affiliation":[{"name":"School of Information Technology, Deakin University, Burwood, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1990-5734","authenticated-orcid":false,"given":"Arkady","family":"Zaslavsky","sequence":"additional","affiliation":[{"name":"School of Information Technology, Deakin University, Burwood, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5339-9305","authenticated-orcid":false,"given":"Seng W.","family":"Loke","sequence":"additional","affiliation":[{"name":"School of Information Technology, Deakin University, Burwood, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7562-6257","authenticated-orcid":false,"given":"Alexey","family":"Medvedev","sequence":"additional","affiliation":[{"name":"School of Information Technology, Deakin University, Burwood, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0605-5083","authenticated-orcid":false,"given":"Amin","family":"Abken","sequence":"additional","affiliation":[{"name":"School of Information Technology, Deakin University, Burwood, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4770-8183","authenticated-orcid":false,"given":"Alireza","family":"Hassani","sequence":"additional","affiliation":[{"name":"School of Information Technology, Deakin University, Burwood, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8698-2946","authenticated-orcid":false,"given":"Guang-Li","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Information Technology, Deakin University, Burwood, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,4,23]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-48157-5_29"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/LCNW.2013.6758526"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-15160-6_3"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ECTICon.2013.6559553"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICACT.2014.6779016"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICC.2014.6883600"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14891-0_1"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2012.07.019"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2020.06.013"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.5555\/2930611.2930636"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/icc.2011.5963361"},{"key":"e_1_3_2_13_2","unstructured":"FIWARE. FIWARE Platform. Retrieved June 7 2021 from https:\/\/www.fiware.org"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3007120.3007137"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.3390\/s19061478"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/PERCOMW.2018.8480240"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.3390\/s19245457"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2019.2937608"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-45866-2_14"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/313451.313549"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2018.2883762"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.3390\/s22041632"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/PERCOM.2003.1192735"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/SCC55611.2022.00025"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341216.3342214"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/11569510_31"},{"key":"e_1_3_2_27_2","unstructured":"Timothy P. Lillicrap Jonathan J. Hunt Alexander Pritzel Nicolas Heess Tom Erez Yuval Tassa David Silver and Daan Wierstra. 2019. Continuous control with deep reinforcement learning. ArXiv150902971 CS Stat. (July 2019). Retrieved November 14 2021 from http:\/\/arxiv.org\/abs\/1509.02971"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0269888914000034"},{"key":"e_1_3_2_29_2","volume-title":"Performance and Cost Driven Data Storage and Processing for IoT Context Management Platforms","author":"Medvedev Alexey","year":"2020","unstructured":"Alexey Medvedev. 2020. Performance and Cost Driven Data Storage and Processing for IoT Context Management Platforms. Doctoral Thesis. Monash University, Australia. Retrieved July 27, 2021 from https:\/\/bridges.monash.edu\/articles\/Performance_and_Cost_Driven_Data_Storage_and_Processing_for_IoT_Context_Management_Platforms\/12423479"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/GIOTS.2018.8534571"},{"key":"e_1_3_2_31_2","article-title":"Discrete sequential prediction of continuous actions for deep RL","author":"Metz Luke","year":"2019","unstructured":"Luke Metz, Julian Ibarz, Navdeep Jaitly, and James Davidson. 2019. Discrete sequential prediction of continuous actions for deep RL. ArXiv170505035 Cs Stat (June 2019). Retrieved November 14, 2021 from http:\/\/arxiv.org\/abs\/1705.05035","journal-title":"ArXiv170505035 Cs Stat"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCC49849.2020.9238811"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/GreenCom.2012.128"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/SURV.2013.042313.00197"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2019.04.020"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/PIMRC.2007.4394813"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/IECON43393.2020.9255111"},{"key":"e_1_3_2_38_2","article-title":"JSON-LD 1.0 - A JSON-based serialization for linked data","author":"Sporny Manu","year":"2014","unstructured":"Manu Sporny, Gregg Kellogg, and Markus Lanthaler. 2014. JSON-LD 1.0 - A JSON-based serialization for linked data. W3C Recomm. (2014).","journal-title":"W3C Recomm"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM.2016.7524524"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-3664(01)00408-X"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-021-03265-9"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2016.2616359"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.3390\/informatics7040043"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3555776.3577602"},{"key":"e_1_3_2_45_2","unstructured":"Shakthi Weerasinghe Arkady Zaslavsky Seng W. Loke Alireza Hassani Amin Abken and Alexey Medvedev. 2022. From traditional adaptive data caching to adaptive context caching: A survey. ArXiv221111259 CsHC (November 2022) 35."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477314.3507075"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/IOTM.001.2200247"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.3390\/s23104767"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2006.12"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dcan.2018.10.002"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2018.2882583"},{"key":"e_1_3_2_52_2","unstructured":"2020. Context Information Management (CIM): Application Programming Interface (API). ETSI Industry Specification Group (ISG). Retrieved from https:\/\/www.etsi.org\/deliver\/etsi_gs\/CIM\/001_099\/004\/01.01.02_60\/gs_CIM004v010102p.pdf"},{"key":"e_1_3_2_53_2","unstructured":"2023. Amazon File Cache-Pricing. Amazon Web Services. Retrieved June 19 2023 from https:\/\/aws.amazon.com\/filecache\/pricing\/"},{"key":"e_1_3_2_54_2","unstructured":"FIWARE-Orion. Retrieved September 30 2021 from https:\/\/github.com\/telefonicaid\/fiware-orion"},{"key":"e_1_3_2_55_2","unstructured":"FIWARE Stream Oriented Generic Enabler - Overview. FIWARE Stream Oriented Generic Enabler - Overview - FIWARE-Stream-Oriented-GE documentation. Retrieved December 20 2022 from https:\/\/kurento.readthedocs.io\/en\/stable\/"},{"key":"e_1_3_2_56_2","unstructured":"Redis. Retrieved September 10 2021 from https:\/\/redis.io"},{"key":"e_1_3_2_57_2","unstructured":"IoT data simulator. Retrieved from https:\/\/github.com\/IBA-Group-IT\/IoT-data-simulator"}],"container-title":["ACM Transactions on Internet of Things"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3648571","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3648571","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:20Z","timestamp":1750287020000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3648571"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,23]]},"references-count":56,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,5,31]]}},"alternative-id":["10.1145\/3648571"],"URL":"https:\/\/doi.org\/10.1145\/3648571","relation":{},"ISSN":["2691-1914","2577-6207"],"issn-type":[{"value":"2691-1914","type":"print"},{"value":"2577-6207","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,23]]},"assertion":[{"value":"2022-12-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-06","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}