{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,3]],"date-time":"2025-08-03T01:11:52Z","timestamp":1754183512074,"version":"3.41.2"},"reference-count":39,"publisher":"Institute of Electronics, Information and Communications Engineers (IEICE)","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IEICE Trans. Inf. & Syst."],"published-print":{"date-parts":[[2025,8,1]]},"DOI":"10.1587\/transinf.2024dat0002","type":"journal-article","created":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T17:11:39Z","timestamp":1733850699000},"page":"863-871","source":"Crossref","is-referenced-by-count":0,"title":["Removing Mislabeled Data from Trained Models via Machine Unlearning"],"prefix":"10.1587","volume":"E108.D","author":[{"given":"Issa","family":"SUGIURA","sequence":"first","affiliation":[{"name":"Kyoto University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shingo","family":"OKAMURA","sequence":"additional","affiliation":[{"name":"Osaka University"},{"name":"National Institute of Technology, Nara College"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Naoto","family":"YANAI","sequence":"additional","affiliation":[{"name":"Panasonic Holdings Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"532","reference":[{"key":"1","doi-asserted-by":"publisher","unstructured":"[1] B. Frenay and M. Verleysen, \u201cClassification in the presence of label noise: a survey,\u201d IEEE Trans. Neural Netw. Learn. Syst., vol.25, no.5, pp.845-869, 2013. 10.1109\/tnnls.2013.2292894","DOI":"10.1109\/TNNLS.2013.2292894"},{"key":"2","doi-asserted-by":"publisher","unstructured":"[2] G. Paolacci, J. Chandler, and P.G. Ipeirotis, \u201cRunning experiments on amazon mechanical turk,\u201d Judgment and Decision making, vol.5, no.5, pp.411-419, 2010. 10.1017\/s1930297500002205","DOI":"10.1017\/S1930297500002205"},{"key":"3","doi-asserted-by":"publisher","unstructured":"[3] V. Cothey, \u201cWeb-crawling reliability,\u201d Journal of the American Society for Information Science and Technology, vol.55, no.14, pp.1228-1238, 2004. 10.1002\/asi.20078","DOI":"10.1002\/asi.20078"},{"key":"4","unstructured":"[4] O. Suciu, R. Marginean, Y. Kaya, H.D. III, and T. Dumitras, \u201cWhen does machine learning FAIL? generalized transferability for evasion and poisoning attacks,\u201d 27th USENIX Security Symposium (USENIX Security 18), Baltimore, MD, pp.1299-1316, USENIX Association, 2018."},{"key":"5","doi-asserted-by":"crossref","unstructured":"[5] H. Xiao, H. Xiao, and C. Eckert, \u201cAdversarial label flips attack on support vector machines,\u201d ECAI 2012, pp.870-875, IOS Press, 2012. 10.3233\/978-1-61499-098-7-870","DOI":"10.3233\/978-1-61499-098-7-870"},{"key":"6","unstructured":"[6] D. Arpit, S. Jastrz\u0119bski, N. Ballas, D. Krueger, E. Bengio, M.S. Kanwal, T. Maharaj, A. Fischer, A. Courville, Y. Bengio, and S. Lacoste-Julien, \u201cA closer look at memorization in deep networks,\u201d Proceedings of the 34th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol.70, pp.233-242, PMLR, 2017."},{"key":"7","doi-asserted-by":"crossref","unstructured":"[7] D. Tanaka, D. Ikami, T. Yamasaki, and K. Aizawa, \u201cJoint optimization framework for learning with noisy labels,\u201d Proceedings of the IEEE conference on computer vision and pattern recognition, pp.5552-5560, 2018. 10.1109\/cvpr.2018.00582","DOI":"10.1109\/CVPR.2018.00582"},{"key":"8","doi-asserted-by":"crossref","unstructured":"[8] G. Patrini, A. Rozza, A.K. Menon, R. Nock, and L. Qu, \u201cMaking deep neural networks robust to label noise: A loss correction approach,\u201d Proceedings of the IEEE conference on computer vision and pattern recognition, pp.2233-2241, 2017. 10.1109\/cvpr.2017.240","DOI":"10.1109\/CVPR.2017.240"},{"key":"9","unstructured":"[9] Z. Zhang and M. Sabuncu, \u201cGeneralized cross entropy loss for training deep neural networks with noisy labels,\u201d Advances in neural information processing systems, vol.31, 2018."},{"key":"10","doi-asserted-by":"crossref","unstructured":"[10] X. Zhang, \u201cAn Improved Method of Identifying Mislabeled Data and the Mislabeled Data in MNIST and CIFAR-10,\u201d SSRN Electronic Journal, 2017.","DOI":"10.2139\/ssrn.3080736"},{"key":"11","doi-asserted-by":"publisher","unstructured":"[11] Q. Jia, X. Li, L. Yu, J. Bian, P. Zhao, S. Li, H. Xiong, and D. Dou, \u201cLearning from training dynamics: identifying mislabeled data beyond manually designed features,\u201d Proceedings of the AAAI Conference on Artificial Intelligence, vol.37, no.7, pp.8041-8049, 2023. 10.1609\/aaai.v37i7.25972","DOI":"10.1609\/aaai.v37i7.25972"},{"key":"12","doi-asserted-by":"crossref","unstructured":"[12] Y. Cao and J. Yang, \u201cTowards making systems forget with machine unlearning,\u201d 2015 IEEE Symposium on Security and Privacy, pp.463-480, 2015. 10.1109\/sp.2015.35","DOI":"10.1109\/SP.2015.35"},{"key":"13","unstructured":"[13] P.W. Koh and P. Liang, \u201cUnderstanding black-box predictions via influence functions,\u201d Proceedings of the 34th International Conference on Machine Learning, ed. D. Precup and Y.W. Teh, Proceedings of Machine Learning Research, vol.70, pp.1885-1894, PMLR, 2017."},{"key":"14","unstructured":"[14] A. Peste, D. Alistarh, and C.H. Lampert, \u201cSSSE: Efficiently erasing samples from trained machine learning models,\u201d 2021."},{"key":"15","doi-asserted-by":"crossref","unstructured":"[15] A. Chan, A. Gujarati, K. Pattabiraman, and S. Gopalakrishnan, \u201cThe fault in our data stars: Studying mitigation techniques against faulty training data in machine learning applications,\u201d 2022 52nd Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN), pp.163-171, 2022. 10.1109\/dsn53405.2022.00027","DOI":"10.1109\/DSN53405.2022.00027"},{"key":"16","doi-asserted-by":"publisher","unstructured":"[16] H. Song, M. Kim, D. Park, Y. Shin, and J.-G. Lee, \u201cLearning from noisy labels with deep neural networks: A survey,\u201d IEEE Trans. Neural Netw. Learn. Syst., vol.34, no.11, pp.8135-8153, 2023. 10.1109\/tnnls.2022.3152527","DOI":"10.1109\/TNNLS.2022.3152527"},{"key":"17","doi-asserted-by":"crossref","unstructured":"[17] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, \u201cMembership inference attacks against machine learning models,\u201d 2017 IEEE Symposium on Security and Privacy (SP), Los Alamitos, CA, USA, pp.3-18, IEEE Computer Society, 2017. 10.1109\/sp.2017.41","DOI":"10.1109\/SP.2017.41"},{"key":"18","doi-asserted-by":"crossref","unstructured":"[18] M. Fredrikson, S. Jha, and T. Ristenpart, \u201cModel inversion attacks that exploit confidence information and basic countermeasures,\u201d Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, CCS \u201915, New York, NY, USA, pp.1322-1333, Association for Computing Machinery, 2015. 10.1145\/2810103.2813677","DOI":"10.1145\/2810103.2813677"},{"key":"19","unstructured":"[19] Y. Zeng, S. Chen, W. Park, Z. Mao, M. Jin, and R. Jia, \u201cAdversarial unlearning of backdoors via implicit hypergradient,\u201d Proc. of ICLR 2022, OpenReview.net, 2022."},{"key":"20","doi-asserted-by":"crossref","unstructured":"[20] L. Bourtoule, V. Chandrasekaran, C.A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, \u201cMachine unlearning,\u201d 2021 IEEE Symposium on Security and Privacy (SP), pp.141-159, 2021. 10.1109\/sp40001.2021.00019","DOI":"10.1109\/SP40001.2021.00019"},{"key":"21","unstructured":"[21] A. Ginart, M. Guan, G. Valiant, and J.Y. Zou, \u201cMaking ai forget you: Data deletion in machine learning,\u201d Advances in neural information processing systems, vol.32, 2019."},{"key":"22","doi-asserted-by":"crossref","unstructured":"[22] A. Golatkar, A. Achille, and S. Soatto, \u201cEternal sunshine of the spotless net: Selective forgetting in deep networks,\u201d Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.9301-9309, 2020. 10.1109\/cvpr42600.2020.00932","DOI":"10.1109\/CVPR42600.2020.00932"},{"key":"23","unstructured":"[23] Z. Izzo, M.A. Smart, K. Chaudhuri, and J. Zou, \u201cApproximate data deletion from machine learning models,\u201d International Conference on Artificial Intelligence and Statistics, pp.2008-2016, PMLR, 2021."},{"key":"24","unstructured":"[24] Y. Wu, E. Dobriban, and S. Davidson, \u201cDeltagrad: Rapid retraining of machine learning models,\u201d International Conference on Machine Learning, pp.10355-10366, PMLR, 2020."},{"key":"25","unstructured":"[25] A. Sinitsin, V. Plokhotnyuk, D. Pyrkin, S. Popov, and A. Babenko, \u201cEditable neural networks,\u201d International Conference on Learning Representations, 2020."},{"key":"26","unstructured":"[26] E. Mitchell, C. Lin, A. Bosselut, C. Finn, and C.D. Manning, \u201cFast model editing at scale,\u201d International Conference on Learning Representations, 2022."},{"key":"27","unstructured":"[27] E. Mitchell, C. Lin, A. Bosselut, C.D. Manning, and C. Finn, \u201cMemory-based model editing at scale,\u201d International Conference on Machine Learning, pp.15817-15831, PMLR, 2022."},{"key":"28","unstructured":"[28] T.T. Nguyen, T.T. Huynh, P.L. Nguyen, A.W. Liew, H. Yin, and Q.V.H. Nguyen, \u201cA survey of machine unlearning,\u201d CoRR, vol.abs\/2209.02299, 2022."},{"key":"29","unstructured":"[29] R. Chen, J. Yang, H. Xiong, J. Bai, T. Hu, J. Hao, Y. Feng, J.T. Zhou, J. Wu, and Z. Liu, \u201cFast model debias with machine unlearning,\u201d Advances in Neural Information Processing Systems, vol.36, 2024."},{"key":"30","unstructured":"[30] LLM-jp, \u201cLlm-jp: A cross-organizational project for the research and development of fully open japanese llms,\u201d 2024."},{"key":"31","unstructured":"[31] Y. LeCun and C. Cortes, \u201cMNIST handwritten digit database,\u201d 2010."},{"key":"32","unstructured":"[32] J. Bradbury, R. Frostig, P. Hawkins, M.J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, \u201cJAX: composable transformations of Python+NumPy programs,\u201d 2018."},{"key":"33","unstructured":"[33] DeepMind, I. Babuschkin, K. Baumli, A. Bell, S. Bhupatiraju, J. Bruce, P. Buchlovsky, D. Budden, T. Cai, A. Clark, I. Danihelka, A. Dedieu, C. Fantacci, J. Godwin, C. Jones, R. Hemsley, T. Hennigan, M. Hessel, S. Hou, S. Kapturowski, T. Keck, I. Kemaev, M. King, M. Kunesch, L. Martens, H. Merzic, V. Mikulik, T. Norman, G. Papamakarios, J. Quan, R. Ring, F. Ruiz, A. Sanchez, L. Sartran, R. Schneider, E. Sezener, S. Spencer, S. Srinivasan, M. Stanojevi\u0107, W. Stokowiec, L. Wang, G. Zhou, and F. Viola, \u201cThe DeepMind JAX Ecosystem,\u201d 2020."},{"key":"34","unstructured":"[34] J. Heek, A. Levskaya, A. Oliver, M. Ritter, B. Rondepierre, A. Steiner, and M. van Zee, \u201cFlax: A neural network library and ecosystem for JAX,\u201d 2023."},{"key":"35","unstructured":"[35] C.G. Northcutt, A. Athalye, and J. Mueller, \u201cPervasive label errors in test sets destabilize machine learning benchmarks,\u201d arXiv preprint arXiv:2103.14749, 2021."},{"key":"36","unstructured":"[36] A. Krizhevsky, G. Hinton, et al., \u201cLearning multiple layers of features from tiny images,\u201d 2009."},{"key":"37","unstructured":"[37] R. Grosse, J. Bae, C. Anil, N. Elhage, A. Tamkin, A. Tajdini, B. Steiner, D. Li, E. Durmus, E. Perez, E. Hubinger, K. Luko\u0161i\u016bt\u0117, K. Nguyen, N. Joseph, S. McCandlish, J. Kaplan, and S.R. Bowman, \u201cStudying large language model generalization with influence functions,\u201d 2023."},{"key":"38","unstructured":"[38] J. Martens and R. Grosse, \u201cOptimizing neural networks with kronecker-factored approximate curvature,\u201d International conference on machine learning, pp.2408-2417, PMLR, 2015."},{"key":"39","unstructured":"[39] F. Kunstner, P. Hennig, and L. Balles, \u201cLimitations of the empirical fisher approximation for natural gradient descent,\u201d Advances in neural information processing systems, vol.32, 2019."}],"container-title":["IEICE Transactions on Information and Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jstage.jst.go.jp\/article\/transinf\/E108.D\/8\/E108.D_2024DAT0002\/_pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T03:28:51Z","timestamp":1754105331000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jstage.jst.go.jp\/article\/transinf\/E108.D\/8\/E108.D_2024DAT0002\/_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,1]]},"references-count":39,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2025]]}},"URL":"https:\/\/doi.org\/10.1587\/transinf.2024dat0002","relation":{},"ISSN":["0916-8532","1745-1361"],"issn-type":[{"type":"print","value":"0916-8532"},{"type":"electronic","value":"1745-1361"}],"subject":[],"published":{"date-parts":[[2025,8,1]]},"article-number":"2024DAT0002"}}