{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T03:50:38Z","timestamp":1784173838745,"version":"3.55.0"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,1,25]],"date-time":"2023-01-25T00:00:00Z","timestamp":1674604800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:p>Fault detection is an essential task for large-scale industrial maintenance. However, in practical applications, due to the possible harm caused by the collection of fault data, the fault samples that lead to the labeling are usually very few. Most existing methods consider training unsupervised models with a large amount of unlabeled data while ignoring the rich knowledge that existed in a small amount of labeled data. To make full use of this prior knowledge, this article proposes a reinforcement learning model, namely, adversarial reinforcement learning in weakly supervised (WS-ARL), which performs significantly better by jointly learning small labeled anomaly data and large unlabeled data. We use an agent of the reinforcement learning model as a fault detector and add a new environment agent as a sample selector, by providing an opposite reward for two agents, and they learn in an adversarial environment. The feasibility and effectiveness of the model are verified by experimental analysis and compared the performance of the model with five state-of-the-art weakly\/un-supervised methods in the hydraulic press fault detection task.<\/jats:p>","DOI":"10.3389\/fcomp.2022.1007665","type":"journal-article","created":{"date-parts":[[2023,1,25]],"date-time":"2023-01-25T10:28:35Z","timestamp":1674642515000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Fault detection method based on adversarial reinforcement learning"],"prefix":"10.3389","volume":"4","author":[{"given":"Li","family":"Junhuai","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wu","family":"Yunwen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wang","family":"Huaijun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xu","family":"Jiang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,1,25]]},"reference":[{"key":"B1","unstructured":"Variational autoencoder based anomaly detection using reconstruction probability118\n            AnJ.\n            ChoS.\n          36359702Special Lecture IE22015"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2022.07.129","article-title":"TiWS-iForest: isolation forest in weakly supervised and tiny ML scenarios","author":"Barbariol","year":"2021","journal-title":"arXiv:2111.15432 [cs.LG]."},{"key":"B3","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE: Synthetic minority over-sampling technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Intell. Res."},{"key":"B4","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1109\/ICOIN53446.2022.9687205","article-title":"Multivariate time-series anomaly detection using SeqVAE-CNN hybrid model","volume-title":"2022 International Conference on Information Networking (ICOIN)","author":"Choi","year":"2022"},{"key":"B5","doi-asserted-by":"crossref","first-page":"2568","DOI":"10.1109\/CVPR.2015.7298872","article-title":"DevNet: a deep event network for multimedia event detection and evidence recounting","volume-title":"2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Gan","year":"2015"},{"key":"B6","doi-asserted-by":"publisher","first-page":"67249","DOI":"10.1109\/ACCESS.2021.3077014","article-title":"ConNet: deep semi-supervised anomaly detection based on sparse positive samples","volume":"9","author":"Gao","year":"2021","journal-title":"IEEE Access"},{"key":"B7","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/BigData50022.2020.9378139","article-title":"TadGAN: time series anomaly detection using generative adversarial networks","volume-title":"2020 IEEE International Conference on Big Data (Big Data)","author":"Geiger","year":"2020"},{"key":"B8","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1016\/j.comnet.2019.05.013","article-title":"Adversarial environment reinforcement learning algorithm for intrusion detection","author":"Guillermo","year":"2019","journal-title":"Comput. Netw."},{"key":"B9","doi-asserted-by":"publisher","first-page":"616","DOI":"10.5162\/sensor2015\/D8.1","article-title":"Detecting and compensating sensor faults in a hydraulic condition monitoring system","volume":"2015","author":"Helwig","year":"2015","journal-title":"Sensor"},{"key":"B10","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1609\/aimag.v35i4.2553","article-title":"Enhanced telemetry monitoring with novelty detection","volume":"35","author":"Heras","year":"2014","journal-title":"AI Mag"},{"key":"B11","doi-asserted-by":"crossref","DOI":"10.1109\/ICST46399.2020.00049","article-title":"Poster: is euclidean distance the best distance measurement for adaptive random testing?","volume-title":"2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)","author":"Huang","year":"2020"},{"key":"B12","doi-asserted-by":"crossref","first-page":"1244","DOI":"10.1109\/ICTC52510.2021.9621052","article-title":"Network anomaly detection based on GAN with scaling properties","volume-title":"2021 International Conference on Information and Communication Technology Convergence (ICTC)","author":"Kim","year":"2021"},{"key":"B13","doi-asserted-by":"publisher","first-page":"3581","DOI":"10.48550\/arXiv.1406.5298","article-title":"Semi-supervised learning with deep generative models","volume":"4","author":"Kingma","year":"2014","journal-title":"NIPS"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2006.11321","article-title":"AutoOD: automated outlier detection via curiosity-guided search and self-imitation learning","author":"Li","year":"2020","journal-title":"arXiv:2006.11321 [cs.LG]."},{"key":"B15","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1145\/2133360.2133363","article-title":"Isolation-based anomaly detection.isolation-based anomaly detection","volume":"6","author":"Liu","year":"2012","journal-title":"ACM Trans. Knowl. Discov. Data"},{"key":"B16","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1109\/TNSE.2020.3004312","article-title":"AESMOTE: adversarial reinforcement learning with SMOTE for anomaly detection","volume":"8","author":"Ma","year":"2020","journal-title":"IEEE Trans. Network Sci. Eng"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1602.05473","article-title":"Auxiliary deep generative models","author":"Maale","year":"2016","journal-title":"arXiv:1602.05473 [stat.ML]"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1607.00148","article-title":"Lstm-based encoder-decoder for multi-sensor anomaly detection","author":"Malhotra","year":"2016","journal-title":"arXiv 2016"},{"key":"B19","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1145\/335191.335388","article-title":"LOF: Identifying density-based local outliers","volume":"29","author":"Markus","year":"2000","journal-title":"ACM Sigmod Record"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1312.5602","article-title":"Playing atari with deep reinforcement learning","author":"Mnih","year":"2013","journal-title":"arXiv:1312.5602 [cs.LG]"},{"key":"B21","first-page":"2041","article-title":"Learning representations of ultrahigh-dimensional data for random distance-based outlierdetection","volume-title":"KDD","author":"Pang","year":"2018"},{"key":"B22","doi-asserted-by":"publisher","first-page":"3467417","DOI":"10.1145\/3447548.3467417","article-title":"Toward deep supervised anomaly detection: reinforcement learning from partially labeled anomaly data.knowledge discovery and data mining","volume":"2020","author":"Pang","year":"2020","journal-title":"ACM"},{"key":"B23","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1145\/3292500.3330871","article-title":"Deep anomaly detection with deviation networks","volume-title":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '19)","author":"Pang","year":"2019"},{"key":"B24","doi-asserted-by":"crossref","first-page":"5406","DOI":"10.1109\/IROS.2017.8206437","article-title":"A multimodal execution monitor with anomaly classification for robot-assisted feeding","volume-title":"2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Park","year":"2017"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1906.02694","article-title":"Deep Semi-supervised anomaly detection..international conference on learning representations","author":"Ruff","year":"2019","journal-title":"arXiv:1906.02694 [cs.LG]."},{"key":"B26","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1016\/j.jmsy.2019.11.004","article-title":"Big data and stream processing platforms for industry 4.0 requirements mapping for a predictive maintenance use case","volume":"54","author":"Sahal","year":"2020","journal-title":"J. Manuf. Syst"},{"key":"B27","doi-asserted-by":"crossref","first-page":"2828","DOI":"10.1145\/3292500.3330672","article-title":"Robust anomaly detection for multivariate time series through stochastic recurrent neural network","author":"Su","year":"2019","journal-title":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '19)"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1054","DOI":"10.1109\/TNN.1998.712192","article-title":"Reinforcement learning: an introduction","volume":"9","author":"Sutton","year":"1998","journal-title":"IEEE Trans. Neural Netw"},{"key":"B29","doi-asserted-by":"publisher","first-page":"2319","DOI":"10.1126\/science.290.5500.2319","article-title":"A global geometric framework for nonlinear dimensionality reduction","volume":"290","author":"Tenenbaum","year":"2000","journal-title":"Science"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1052","DOI":"10.1109\/TPWRS.2012.2224144","article-title":"Real-time intrusion detection in power system operations","volume":"28","author":"Valenzuela","year":"2013","journal-title":"IEEE Trans. Power Syst."},{"key":"B31","first-page":"807","article-title":"Rectified linear units improve restricted boltzmann machines","volume-title":"ICML","author":"Vinod","year":"2010"},{"key":"B32","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Volodymyr","year":"2015","journal-title":"Nature"},{"key":"B33","doi-asserted-by":"publisher","first-page":"44278","DOI":"10.1109\/ACCESS.2022.3165977","article-title":"Unsupervised anomaly video detection via a double-flow ConvLSTM variational autoencoder","volume":"10","author":"Wang","year":"2022","journal-title":"IEEE Access"},{"key":"B34","doi-asserted-by":"crossref","first-page":"5286","DOI":"10.1109\/BigData50022.2020.9378265","article-title":"Semi-unsupervised learning: Clustering and classifying using ultra-sparse labels","volume-title":"Proceedings of 2020 IEEE International Conference on Big Data (Big Data)","author":"Willetts","year":"2020"},{"key":"B35","doi-asserted-by":"crossref","first-page":"975","DOI":"10.1109\/ICDMW53433.2021.00127","article-title":"Surrogate supervision-based deep weakly-supervised anomaly detection","volume-title":"2021 International Conference on Data Mining Workshops (ICDMW)","author":"Wu","year":"2021"},{"key":"B36","doi-asserted-by":"publisher","first-page":"2444","DOI":"10.1109\/TNNLS.2021.3095150","article-title":"Semisupervised training of deep generative models for high-dimensional anomaly detection","volume":"33","author":"Xie","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3185996","article-title":"Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications","author":"Xu","year":"2018","journal-title":"arXiv:1802.03903 [cs.LG]"},{"key":"B38","first-page":"2828","article-title":"Robust anomaly detection for multivariate time series through stochastic recurrent neural network","volume-title":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '19)","author":"Ya","year":"2019"},{"key":"B39","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1109\/ICDM.2018.00088","article-title":"Adversarially learned anomaly detection","volume-title":"2018 IEEE International Conference on Data Mining (ICDM)","author":"Zenati","year":"2018"}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2022.1007665\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,25]],"date-time":"2023-01-25T10:28:46Z","timestamp":1674642526000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2022.1007665\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,25]]},"references-count":39,"alternative-id":["10.3389\/fcomp.2022.1007665"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2022.1007665","relation":{},"ISSN":["2624-9898"],"issn-type":[{"value":"2624-9898","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,25]]},"article-number":"1007665"}}