{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T14:37:52Z","timestamp":1774622272264,"version":"3.50.1"},"reference-count":59,"publisher":"PeerJ","license":[{"start":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T00:00:00Z","timestamp":1772582400000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>The swift advancement of Internet of Things (IoT) technology has revolutionized smart home settings; the prevalent automation systems are limited by their need for specific device identification and rigid rule-based configurations. These constraints impede natural human-device interaction, especially in dynamic or communal environments where spatial context is more instinctive than predetermined naming conventions. Current solutions frequently neglect spatial reasoning and multimodal inputs, resulting in heightened cognitive demands and diminished accessibility. The proposed work develops a spatial context-aware control system aimed at facilitating intuitive, vision-driven, and language-based interaction with smart devices to overcome these problems.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      The proposed model is a modular, multimodal framework that integrates computer vision, natural language processing, and spatial inference for context-aware smart device control. The system comprises six core components: (i) an Onboarding Inference Engine for extracting device information\n                      <jats:italic>via<\/jats:italic>\n                      natural language input, (ii) Zero-Shot Device Detection using Open-World Localization\u2013Vision Transformer (OWL-ViT) for object identification without prior training, (iii) Metadata Refinement and Filtering for structured annotation and disambiguation, (iv) a Geospatial Device Visualizer for annotated visual feedback, (v) Spatial Topology Inference using Generative Pre-trained Transformer 4 omni (GPT-4o) for reasoning about physical layouts, and (vi) Intent-Based Command Synthesis with Gemini Flash to generate precise, executable control commands. The final Agentic Execution Module interfaces with the Tuya Smart Device Application Programming Interface (API), ensuring vendor-agnostic actuation. The system supports multilingual input and adapts to various environmental contexts, including smart homes and assisted living facilities.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      A user study involving 15 participants (aged 18\u201380, diverse educational backgrounds) was conducted to evaluate the effectiveness of the proposed method in comparison to the Google Home Assistant. Quantitative findings demonstrate a statistically significant reduction in cognitive workload, with NASA Task Load Index (TLX) scores decreasing by an average of 13.17 points (\n                      <jats:italic>p<\/jats:italic>\n                      = 0.0013, Cohen\u2019s d = 1.0381). Participants rated the proposed method higher in terms of ease of use (mean = 4.67) compared to Google Home (mean = 3.8) on a 5-point Likert scale. Qualitative feedback highlighted the intuitive nature of spatial context commands, reduced cognitive burden due to the elimination of device name memorization, and enhanced accessibility\n                      <jats:italic>via<\/jats:italic>\n                      support for regional languages. 93.3% of users preferred the proposed method over the baseline system. These results affirm the feasibility and user-centric benefits of integrating vision-language models for context-aware smart device control.\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.7717\/peerj-cs.3546","type":"journal-article","created":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T08:09:36Z","timestamp":1772611776000},"page":"e3546","source":"Crossref","is-referenced-by-count":1,"title":["Intelligence of Things: a spatial context-aware control system for smart devices"],"prefix":"10.7717","volume":"12","author":[{"given":"Sukanth","family":"Kalivarathan","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Vellore Institute of Technology University, Chennai, India"}]},{"given":"Muhmmad Abrar Raja","family":"Mohamed","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Vellore Institute of Technology University, Chennai, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0897-6991","authenticated-orcid":true,"given":"Aswathy","family":"Ravikumar","sequence":"additional","affiliation":[{"name":"Sustainable Living Lab, Chennai, India"}]},{"given":"Harini","family":"Sriraman","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Vellore Institute of Technology University, Chennai, India"}]}],"member":"4443","published-online":{"date-parts":[[2026,3,4]]},"reference":[{"key":"10.7717\/peerj-cs.3546\/ref-1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2301.00134","article-title":"Exploring the use of data-driven approaches for anomaly detection in the internet of things (IoT) environment","author":"Achiluzzi","year":"2022"},{"key":"10.7717\/peerj-cs.3546\/ref-2","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1109\/CCECE49351.2022.9918499","article-title":"Virtual sensor middleware: managing IoT data for the fog-cloud platform","author":"AlMahamid","year":"2022"},{"issue":"1","key":"10.7717\/peerj-cs.3546\/ref-3","doi-asserted-by":"publisher","first-page":"101933","DOI":"10.1016\/j.seares.2020.101933","article-title":"Multifaceted linkages among eco-physiological factors, seasonal plankton dynamics and selective feeding behavior of the green mussel (Perna viridis) in the south-east coast of the Bay of Bengal","volume":"164","author":"Asaduzzaman","year":"2020","journal-title":"Journal of Sea Research"},{"key":"10.7717\/peerj-cs.3546\/ref-4","first-page":"10","article-title":"Big data: an ultimate solution in health care","volume":"106","author":"Baby","year":"2014","journal-title":"International Journal of Computer Applications"},{"key":"10.7717\/peerj-cs.3546\/ref-5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1903.00718","article-title":"Virtual representations for iterative IoT deployment","author":"Bader","year":"2019"},{"key":"10.7717\/peerj-cs.3546\/ref-6","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1909.00935","article-title":"Voice spoofing detection corpus for single and multi-order audio replays","author":"Baumann","year":"2019"},{"issue":"5","key":"10.7717\/peerj-cs.3546\/ref-7","doi-asserted-by":"publisher","first-page":"100433","DOI":"10.1016\/j.ese.2024.100433","article-title":"The synergistic interplay of artificial intelligence and digital twin in environmentally planning sustainable smart cities: a comprehensive systematic review","volume":"20","author":"Bibri","year":"2024","journal-title":"Environmental Science and Ecotechnology"},{"key":"10.7717\/peerj-cs.3546\/ref-8","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2005.13003","article-title":"Context-aware collaborative-intelligence with spatio-temporal in-sensor-analytics in a large-area IoT testbed","author":"Chatterjee","year":"2020"},{"key":"10.7717\/peerj-cs.3546\/ref-9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2401.12168","article-title":"SpatialVLM: endowing vision-language models with spatial reasoning capabilities","author":"Chen","year":"2024"},{"key":"10.7717\/peerj-cs.3546\/ref-10","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1903.10018","article-title":"The meeting of acquaintances: a cost-efficient authentication scheme for light-weight objects with transient trust level and plurality approach","author":"Dang","year":"2019"},{"issue":"9","key":"10.7717\/peerj-cs.3546\/ref-11","doi-asserted-by":"publisher","first-page":"53","DOI":"10.5120\/19858-1810","article-title":"A study from the perspective of nature-inspired metaheuristic optimization algorithms","volume":"113","author":"Deepthi","year":"2015","journal-title":"International Journal of Computer Applications"},{"issue":"10","key":"10.7717\/peerj-cs.3546\/ref-12","doi-asserted-by":"publisher","first-page":"212","DOI":"10.1016\/j.suscom.2019.01.013","article-title":"Intelligent context-awareness system for energy efficiency in smart building based on ontology","volume":"21","author":"Degha","year":"2019","journal-title":"Sustainable Computing: Informatics and Systems"},{"key":"10.7717\/peerj-cs.3546\/ref-13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.23919\/IFIPNetworking.2017.8264880","article-title":"Edge-ICN and its application to the internet of things","author":"Fotiou","year":"2017"},{"key":"10.7717\/peerj-cs.3546\/ref-14","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2207.12197","article-title":"LiPI: lightweight privacy-preserving data aggregation in IoT","author":"Goyal","year":"2022"},{"key":"10.7717\/peerj-cs.3546\/ref-15","doi-asserted-by":"publisher","first-page":"1012","DOI":"10.1109\/ICOSEC49089.2020.9215262","article-title":"Effect of parallel workload on dynamic voltage frequency scaling for dark silicon ameliorating","author":"Harini","year":"2020"},{"key":"10.7717\/peerj-cs.3546\/ref-16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.36680\/j.itcon.2019.001","article-title":"Rapid internet of things (IoT) prototype for accurate people counting towards energy efficient buildings","volume":"24","author":"Huang","year":"2019","journal-title":"Journal of Information Technology in Construction (ITcon)"},{"issue":"11","key":"10.7717\/peerj-cs.3546\/ref-17","doi-asserted-by":"publisher","first-page":"100196","DOI":"10.1016\/j.jii.2020.100196","article-title":"Digital twin to improve the virtual-real integration of industrial IoT","volume":"22","author":"Jiang","year":"2021","journal-title":"Journal of Industrial Information Integration"},{"key":"10.7717\/peerj-cs.3546\/ref-18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2504.13942","article-title":"Intelligence of things: a spatial context-aware control system for smart devices","author":"Kalivarathan","year":"2025"},{"issue":"2","key":"10.7717\/peerj-cs.3546\/ref-19","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.1109\/TWC.2023.3285250","article-title":"Direct link interference suppression for bistatic backscatter communication in distributed MIMO","volume":"23","author":"Kaplan","year":"2024","journal-title":"IEEE Transactions on Wireless Communications"},{"key":"10.7717\/peerj-cs.3546\/ref-20","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2005.06958","article-title":"Wearable internet of things for personalized healthcare study of trends and latent research","author":"Khan","year":"2020"},{"key":"10.7717\/peerj-cs.3546\/ref-21","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.14143","article-title":"Get ready for a party: exploring smarter smart spaces with help from large language models","author":"King","year":"2023"},{"issue":"1","key":"10.7717\/peerj-cs.3546\/ref-22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3643505","article-title":"Sasha: creative goal-oriented reasoning in smart homes with large language models","volume":"8","author":"King","year":"2024","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"issue":"4","key":"10.7717\/peerj-cs.3546\/ref-23","doi-asserted-by":"publisher","first-page":"1183","DOI":"10.24136\/oc.3109","article-title":"Enterprise generative artificial intelligence technologies, internet of things and blockchain-based fintech management, and digital twin industrial metaverse in the cognitive algorithmic economy","volume":"15","author":"Kliestik","year":"2024","journal-title":"Oeconomia Copernicana"},{"issue":"2","key":"10.7717\/peerj-cs.3546\/ref-24","doi-asserted-by":"publisher","first-page":"429","DOI":"10.24136\/eq.3108","article-title":"Generative artificial intelligence of things systems, multisensory immersive extended reality technologies, and algorithmic big data simulation and modelling tools in digital twin industrial metaverse","volume":"19","author":"Klie\u0161tik","year":"2024","journal-title":"Equilibrium"},{"key":"10.7717\/peerj-cs.3546\/ref-25","doi-asserted-by":"publisher","first-page":"156071\u2013156113","DOI":"10.1109\/ACCESS.2021.3127448","article-title":"The internet of federated things (IoFT): a vision for the future and in-depth survey of data-driven approaches for federated learning","volume":"9","author":"Kontar","year":"2021","journal-title":"IEEE Access"},{"key":"10.7717\/peerj-cs.3546\/ref-26","doi-asserted-by":"publisher","first-page":"3836","DOI":"10.1109\/TIP.2023.3286259","article-title":"Joint answering and explanation for visual commonsense reasoning","volume":"32","author":"Li","year":"2023","journal-title":"IEEE Transactions on Image Processing"},{"issue":"11","key":"10.7717\/peerj-cs.3546\/ref-27","doi-asserted-by":"publisher","first-page":"19232","DOI":"10.1109\/JIOT.2024.3377730","article-title":"Unraveling attacks to machine-learning-based IoT systems: a survey and the open libraries behind them","volume":"11","author":"Liu","year":"2024","journal-title":"IEEE Internet of Things Journal"},{"issue":"1","key":"10.7717\/peerj-cs.3546\/ref-28","doi-asserted-by":"publisher","first-page":"100028","DOI":"10.1016\/j.cec.2023.100028","article-title":"A global perspective on e-waste recycling","volume":"2","author":"Liu","year":"2023","journal-title":"Circular Economy"},{"issue":"4","key":"10.7717\/peerj-cs.3546\/ref-29","doi-asserted-by":"publisher","first-page":"2627","DOI":"10.1109\/JIOT.2020.3018677","article-title":"Zero-bias deep learning for accurate identification of internet-of-things (IoT) devices","volume":"8","author":"Liu","year":"2021","journal-title":"IEEE Internet of Things Journal"},{"issue":"3","key":"10.7717\/peerj-cs.3546\/ref-30","doi-asserted-by":"publisher","first-page":"6482088","DOI":"10.1155\/2021\/6482088","article-title":"Internet of things (IoT) technology for the development of intelligent decision support education platform","volume":"2021","author":"Liu","year":"2021","journal-title":"Scientific Programming"},{"issue":"2","key":"10.7717\/peerj-cs.3546\/ref-31","doi-asserted-by":"publisher","first-page":"1222","DOI":"10.1109\/COMST.2019.2962526","article-title":"Sensing, computing, and communication for energy harvesting IoTs: a survey","volume":"22","author":"Ma","year":"2020","journal-title":"IEEE Communications Surveys & Tutorials"},{"key":"10.7717\/peerj-cs.3546\/ref-32","article-title":"The power of patents: leveraging text mining and social network analysis to forecast IoT trends","author":"Maghsoudi","year":"2023"},{"key":"10.7717\/peerj-cs.3546\/ref-33","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2404.06442","article-title":"QueSTMaps: queryable semantic topological maps for 3D scene understanding","author":"Mehan","year":"2024"},{"key":"10.7717\/peerj-cs.3546\/ref-34","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2403.01586","article-title":"IoT device labeling using large language models","author":"Meyuhas","year":"2024"},{"key":"10.7717\/peerj-cs.3546\/ref-35","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1903.09507","article-title":"WSN and fog computing integration for intelligent data processing","author":"Mihai","year":"2019"},{"key":"10.7717\/peerj-cs.3546\/ref-36","doi-asserted-by":"publisher","first-page":"766","DOI":"10.1109\/SMC42975.2020.9282914","article-title":"Toward mixed reality hybrid objects with IoT avatar agents","author":"Morris","year":"2020"},{"key":"10.7717\/peerj-cs.3546\/ref-37","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2007.12250","article-title":"WiFi-based crowd monitoring and workspace planning for COVID-19 recovery","author":"Mu","year":"2020"},{"key":"10.7717\/peerj-cs.3546\/ref-38","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.02278","article-title":"Evaluation and comparison of visual language models for transportation engineering problems","author":"Prajapati","year":"2024"},{"key":"10.7717\/peerj-cs.3546\/ref-39","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2404.18705","article-title":"Wireless information and energy transfer in the era of 6G communications","author":"Psomas","year":"2024"},{"key":"10.7717\/peerj-cs.3546\/ref-40","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1407.2704","article-title":"When things matter: a data-centric view of the internet of things","author":"Qin","year":"2014"},{"issue":"3","key":"10.7717\/peerj-cs.3546\/ref-41","doi-asserted-by":"publisher","first-page":"125929","DOI":"10.1016\/j.eswa.2024.125929","article-title":"A comparative study of advanced technologies and methods in hand gesture analysis and recognition systems","volume":"266","author":"Rahman","year":"2025","journal-title":"Expert Systems with Applications"},{"key":"10.7717\/peerj-cs.3546\/ref-42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/ICCCNT.2013.6726819","article-title":"Support vector machine based prognostic analysis of renal transplantations","author":"Ravikumar","year":"2013"},{"key":"10.7717\/peerj-cs.3546\/ref-43","doi-asserted-by":"publisher","first-page":"45559","DOI":"10.1109\/ACCESS.2023.3274201","article-title":"Computationally efficient neural rendering for generator adversarial networks using a multi-GPU cluster in a cloud environment","volume":"11","author":"Ravikumar","year":"2023","journal-title":"IEEE Access"},{"key":"10.7717\/peerj-cs.3546\/ref-44","doi-asserted-by":"publisher","DOI":"10.36227\/techrxiv.174844330.01320055\/v1","article-title":"A review on LLMs for IoT ecosystem: state-of-the-art, lightweight models, use cases, key challenges, future directions","author":"Ray","year":"2025"},{"key":"10.7717\/peerj-cs.3546\/ref-45","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2311.00772","article-title":"SAGE: smart home agent with grounded execution","author":"Rivkin","year":"2024"},{"issue":"3","key":"10.7717\/peerj-cs.3546\/ref-46","doi-asserted-by":"publisher","first-page":"2458","DOI":"10.1109\/JIOT.2024.3471904","article-title":"AIoT smart home via autonomous LLM agents","volume":"12","author":"Rivkin","year":"2025","journal-title":"IEEE Internet of Things Journal"},{"key":"10.7717\/peerj-cs.3546\/ref-47","doi-asserted-by":"publisher","first-page":"224","DOI":"10.1109\/TNANO.2021.3060358","article-title":"Adaptive non-uniform compressive sensing using SOT-MRAM multi-bit precision crossbar arrays","volume":"20","author":"Salehi","year":"2021","journal-title":"IEEE Transactions on Nanotechnology"},{"key":"10.7717\/peerj-cs.3546\/ref-48","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/ICECS202256217.2022.9970865","article-title":"RF power transmission for self-sustaining miniaturized IoT devices","author":"Schulthess","year":"2022"},{"key":"10.7717\/peerj-cs.3546\/ref-49","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2209.11382","article-title":"Zero-forcing based downlink virtual MIMO-NOMA communications in IoT networks","author":"Shi","year":"2022"},{"key":"10.7717\/peerj-cs.3546\/ref-50","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1109\/MIPR62202.2024.00024","article-title":"ProxeGraph: scene graph generation utilizing proxemics for smart homes","author":"Spandan","year":"2024"},{"key":"10.7717\/peerj-cs.3546\/ref-51","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2106.00509","article-title":"Towards efficient compressive data collection in the internet of things","author":"Sun","year":"2021"},{"issue":"22","key":"10.7717\/peerj-cs.3546\/ref-52","doi-asserted-by":"publisher","first-page":"5243","DOI":"10.3390\/molecules25225243","article-title":"Important flavonoids and their role as a therapeutic agent","volume":"25","author":"Ullah","year":"2020","journal-title":"Molecules"},{"issue":"1","key":"10.7717\/peerj-cs.3546\/ref-53","doi-asserted-by":"publisher","first-page":"95","DOI":"10.24136\/oc.2814","article-title":"Digital twin simulation modeling, artificial intelligence-based internet of manufacturing things systems, and virtual machine and cognitive computing algorithms in the Industry 4.0-based Slovak labor market","volume":"15","author":"Valaskova","year":"2024","journal-title":"Oeconomia Copernicana"},{"key":"10.7717\/peerj-cs.3546\/ref-54","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2411.15714","article-title":"ROOT: VLM based system for indoor scene understanding and beyond","author":"Wang","year":"2024"},{"issue":"1","key":"10.7717\/peerj-cs.3546\/ref-55","doi-asserted-by":"publisher","first-page":"3091","DOI":"10.1038\/s41467-022-30714-9","article-title":"A federated graph neural network framework for privacy-preserving personalization","volume":"13","author":"Wu","year":"2022","journal-title":"Nature Communications"},{"key":"10.7717\/peerj-cs.3546\/ref-56","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2109.14348","article-title":"Smart-home anomaly detection using combination of in-home situation and user behavior","author":"Yamauchi","year":"2021"},{"key":"10.7717\/peerj-cs.3546\/ref-57","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1601.05569","article-title":"Towards a general software engineering methodology for the internet of things","author":"Zambonelli","year":"2016"},{"key":"10.7717\/peerj-cs.3546\/ref-58","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1145\/3301293.3302354","article-title":"AIDE: augmented onboarding of IoT devices at ease","author":"Zhang","year":"2019"},{"key":"10.7717\/peerj-cs.3546\/ref-59","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2406.06542","article-title":"vMCU: coordinated memory management and kernel optimization for DNN inference on MCUs","author":"Zheng","year":"2024"}],"container-title":["PeerJ Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/peerj.com\/articles\/cs-3546.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3546.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3546.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3546.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T08:09:51Z","timestamp":1772611791000},"score":1,"resource":{"primary":{"URL":"https:\/\/peerj.com\/articles\/cs-3546"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,4]]},"references-count":59,"alternative-id":["10.7717\/peerj-cs.3546"],"URL":"https:\/\/doi.org\/10.7717\/peerj-cs.3546","archive":["CLOCKSS","LOCKSS","Portico"],"relation":{},"ISSN":["2376-5992"],"issn-type":[{"value":"2376-5992","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,4]]},"article-number":"e3546"}}