{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T00:17:36Z","timestamp":1778285856416,"version":"3.51.4"},"reference-count":43,"publisher":"Institution of Engineering and Technology (IET)","issue":"1","license":[{"start":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T00:00:00Z","timestamp":1747699200000},"content-version":"vor","delay-in-days":139,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["ietresearch.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["IET Image Processing"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>Deep learning models require large training datasets. Incorporating additional data into small training datasets can enhance the model's performance. However, acquiring additional data may sometimes be challenging or beyond one's control. In such situations, data augmentation becomes essential to overcome the limited supply of labeled data by generating new data that preserves the essential properties of the original dataset. The primary objective of our research is to develop an iterative data distillation and augmentation (IDDA) method that enlarges the size of a limited image training dataset while preserving its properties. At every iteration, our method distills a set of images from the training set of the previous iteration utilizing the kernel inducing point (KIP) method, and the union of the training and distilled sets creates the new training set. However, our experiments show that IDDA is computationally expensive, increasing processing time by approximately 17%\u201327 for MNIST and Fashion\u2010MNIST, 31%\u201339 for CIFAR\u201010, and up to 48%\u201349 for CIFAR\u2010100 compared to state\u2010of\u2010the\u2010art augmentation methods, due to the additional step of applying KIP for image distillation. We have experimentally determined that for a few iterations the classification accuracy increases and then drops afterward. We validate the IDDA capabilities by comparing it with conventional augmenting methods and MixUp on the following publicly available image datasets: MNIST digit, Fashion\u2010MNIST, CIFAR\u201010, and CIFAR\u2010100. Our approach proves highly effective for very limited datasets, addressing the challenge of database expansion for improved performance of deep learning models.<\/jats:p>","DOI":"10.1049\/ipr2.70107","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T07:26:24Z","timestamp":1747725984000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Iterative Data Distillation and Augmentation for Enhancing Deep Learning Performance With Limited Image Training Data"],"prefix":"10.1049","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-2380-4030","authenticated-orcid":false,"given":"Avinash","family":"Singh","sequence":"first","affiliation":[{"name":"Electrical and Computer Engineering University of Alabama at Birmingham Birmingham Alabama USA"}]},{"given":"Namasivayam","family":"Ambalavanan","sequence":"additional","affiliation":[{"name":"Pediatrics Department University of Alabama at Birmingham Birmingham Alabama USA"}]},{"given":"Nikolay M.","family":"Sirakov","sequence":"additional","affiliation":[{"name":"Mathematics Department East Texas A &amp; M University\u2010Commerce Commerce Texas USA"}]},{"given":"Arie","family":"Nakhmani","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering University of Alabama at Birmingham Birmingham Alabama USA"}]}],"member":"265","published-online":{"date-parts":[[2025,5,20]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-021-00444-8"},{"key":"e_1_2_10_3_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14539"},{"key":"e_1_2_10_4_1","doi-asserted-by":"crossref","unstructured":"C.Sun A.Shrivastava S.Singh andA.Gupta \u201cRevisiting Unreasonable Effectiveness of Data in Deep Learning Era \u201d inProceedings of the IEEE International Conference on Computer Vision(IEEE 2017) 843\u2013852.","DOI":"10.1109\/ICCV.2017.97"},{"key":"e_1_2_10_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2014.2325029"},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-014-0007-7"},{"key":"e_1_2_10_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.01.026"},{"key":"e_1_2_10_8_1","unstructured":"S.Lei H.Zhang K.Wang andZ.Su \u201cHow Training Data Affect the Accuracy and Robustness of Neural Networks for Image Classification \u201d inProceedings of the International Conference on Learning Representations (ICLR)(IEEE Information Theory Society 2019) 1\u201314."},{"key":"e_1_2_10_9_1","doi-asserted-by":"publisher","DOI":"10.3390\/rs13030368"},{"key":"e_1_2_10_10_1","doi-asserted-by":"publisher","DOI":"10.3390\/app11020796"},{"key":"e_1_2_10_11_1","unstructured":"T.Nguyen Z.Chen andJ.Lee \u201cDataset Meta\u2010Learning From Kernel Ridge\u2010Regression \u201d inInternational Conference on Learning Representations(IEEE Information Theory Society 2021)."},{"key":"e_1_2_10_12_1","unstructured":"G. E.Hinton O.Vinyals andJ.Dean \u201cDistilling the Knowledge in a Neural Network \u201dCoRRabs\/1503.02531(2015)."},{"key":"e_1_2_10_13_1","unstructured":"T.Wang J.Zhu A.Torralba andA. A.Efros \u201cDataset Distillation \u201dCoRRabs\/1811.10959(2018)."},{"key":"e_1_2_10_14_1","unstructured":"B.Zhao K. R.Mopuri andH.Bilen \u201cDataset Condensation With Gradient Matching \u201d inInternational Conference on Learning Representations(IEEE Information Theory Society 2021)."},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.3390\/electronics13020282"},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/0098-3004(93)90090-R"},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1002\/qua.24939"},{"key":"e_1_2_10_18_1","first-page":"5186","volume-title":"Advances in Neural Information Processing Systems","author":"Nguyen T.","year":"2021"},{"key":"e_1_2_10_19_1","volume-title":"Advances in Neural Information Processing Systems","author":"Loo N.","year":"2022"},{"key":"e_1_2_10_20_1","unstructured":"N.Loo R.Hasani M.Lechner andD.Rus \u201cDataset Distillation With Convexified Implicit Gradients \u201d inProceedings of the 40th International Conference on Machine Learning ed.A.Krause E.Brunskill K.Cho B.Engelhardt S.Sabato andJ.Scarlett Vol.202(PMLR 2023) 22649\u201322674."},{"key":"e_1_2_10_21_1","doi-asserted-by":"crossref","unstructured":"G.Cazenavette T.Wang A.Torralba A. A.Efros andJ.\u2010Y.Zhu \u201cDataset Distillation by Matching Training Trajectories \u201d inProceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition(IEEE 2022) 4750\u20134759.","DOI":"10.1109\/CVPR52688.2022.01045"},{"key":"e_1_2_10_22_1","doi-asserted-by":"crossref","unstructured":"A.Dabouei S.Soleymani F.Taherkhani andN. M.Nasrabadi \u201cSupermix: Supervising the Mixing Data Augmentation \u201d in2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(IEEE 2021) 13789\u201313798.","DOI":"10.1109\/CVPR46437.2021.01358"},{"key":"e_1_2_10_23_1","unstructured":"H.Zhang M.Cisse Y. N.Dauphin andD.Lopez\u2010Paz \u201cMixup: Beyond Empirical Risk Minimization \u201d inInternational Conference on Learning Representations(IEEE Information Theory Society 2018)."},{"key":"e_1_2_10_24_1","doi-asserted-by":"crossref","unstructured":"S.Yun D.Han S. J.Oh S.Chun J.Choe andY.Yoo \u201cCutmix: Regularization Strategy to Train Strong Classifiers With Localizable Features \u201d inProceedings of the IEEE International Conference on Computer Vision(IEEE 2019) 6023\u20136032.","DOI":"10.1109\/ICCV.2019.00612"},{"key":"e_1_2_10_25_1","unstructured":"J.\u2010H.Kim W.Choo andH. O.Song \u201cPuzzlemix: Exploiting Saliency and Local Statistics for Optimal Mixup \u201d inProceedings of the 37th International Conference on Machine Learning(PMLR 2020) 5275\u20135285."},{"key":"e_1_2_10_26_1","unstructured":"J.Fu X.Geng B.Zhuang et\u00a0al. \u201cRole\u2010Wise Data Augmentation for Knowledge Distillation \u201darXiv:2004.08861(2020)."},{"key":"e_1_2_10_27_1","doi-asserted-by":"publisher","DOI":"10.3390\/math8101652"},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2021.3139038"},{"key":"e_1_2_10_29_1","article-title":"Differentially Private Kernel Inducing Points Using Features From Scatternets (DP\u2010KIP\u2010Scatternet) for Privacy Preserving Data Distillation","author":"Vinaroz M.","year":"2024","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_2_10_30_1","unstructured":"T.Dong B.Zhao andL.Lyu \u201cPrivacy for Free: How Does Dataset Condensation Help Privacy?\u201d inProceedings of the 39th International Conference on Machine Learning ed.K.Chaudhuri S.Jegelka L.Song C.Szepesvari G.Niu andS.Sabato Vol.162(PMLR 2022) 5378\u20135396."},{"key":"e_1_2_10_31_1","volume-title":"Advances in Neural Information Processing Systems","author":"Zhou Y.","year":"2022"},{"key":"e_1_2_10_32_1","unstructured":"Y.Song J.Zhang Z.Tian Y.Yang M.Huang andD.Li \u201cLlm\u2010Based Privacy Data Augmentation Guided by Knowledge Distillation With a Distribution Tutor for Medical Text Classification \u201darXiv:2402.16515(2024)."},{"key":"e_1_2_10_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2022.107189"},{"key":"e_1_2_10_34_1","unstructured":"A.Jacot F.Gabriel andC.Hongler \u201cNeural Tangent Kernel: Convergence and Generalization in Neural Networks \u201d inProceedings of the 32nd International Conference on Neural Information Processing Systems ser. NIPS'18 (Curran Associates Inc. 2018) 8580\u20138589."},{"key":"e_1_2_10_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-41136-6_11"},{"key":"e_1_2_10_36_1","unstructured":"S.Ruder \u201cAn Overview of Gradient Descent Optimization Algorithms \u201dCoRRabs\/1609.04747(2016)."},{"key":"e_1_2_10_37_1","unstructured":"Y.LeCun C.Cortes andC. J.Burges \u201cThe Mnist Database of Handwritten Digits \u201d2010 http:\/\/yann.lecun.com\/exdb\/mnist\/."},{"key":"e_1_2_10_38_1","unstructured":"A.Krizhevsky \u201cLearning Multiple Layers of Features From Tiny Images \u201d Technical Report UTML TR 2009\u2010001 (University of Toronto 2009) https:\/\/www.cs.toronto.edu\/kriz\/learning\u2010features\u20102009\u2010TR.pdf."},{"key":"e_1_2_10_39_1","unstructured":"H.Xiao K.Rasul andR.Vollgraf \u201cFashion\u2010MNIST: A Novel Image Dataset for Benchmarking Machine Learning Algorithms \u201dCoRRabs\/1708.07747(2017) http:\/\/arxiv.org\/abs\/1708.07747."},{"key":"e_1_2_10_40_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1989.1.4.541"},{"key":"e_1_2_10_41_1","unstructured":"F.Chollet et\u00a0al. \u201cKeras Documentation \u201d accessed January 28 2024 https:\/\/keras.io."},{"key":"e_1_2_10_42_1","first-page":"125","article-title":"The Beginning of the Monte Carlo Method","volume":"15","author":"Metropolis N.","year":"1987","journal-title":"Los Alamos Science"},{"key":"e_1_2_10_43_1","doi-asserted-by":"crossref","unstructured":"A.Antoniou A.Storkey andH.Edwards \u201cData Augmentation Generative Adversarial Networks \u201d2018 https:\/\/arxiv.org\/abs\/1711.04340.","DOI":"10.1007\/978-3-030-01424-7_58"},{"key":"e_1_2_10_44_1","doi-asserted-by":"crossref","unstructured":"B.Zhou A.Khosla A.Lapedriza A.Oliva andA.Torralba \u201cLearning Deep Features For Discriminative Localization \u201d inIEEE Conference on Computer Vision and Pattern Recognition (CVPR)(IEEE 2016).","DOI":"10.1109\/CVPR.2016.319"}],"container-title":["IET Image Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ietresearch.onlinelibrary.wiley.com\/doi\/pdf\/10.1049\/ipr2.70107","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/ietresearch.onlinelibrary.wiley.com\/doi\/full-xml\/10.1049\/ipr2.70107","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/ietresearch.onlinelibrary.wiley.com\/doi\/pdf\/10.1049\/ipr2.70107","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T00:05:19Z","timestamp":1778285119000},"score":1,"resource":{"primary":{"URL":"https:\/\/ietresearch.onlinelibrary.wiley.com\/doi\/10.1049\/ipr2.70107"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1]]},"references-count":43,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["10.1049\/ipr2.70107"],"URL":"https:\/\/doi.org\/10.1049\/ipr2.70107","archive":["Portico"],"relation":{},"ISSN":["1751-9659","1751-9667"],"issn-type":[{"value":"1751-9659","type":"print"},{"value":"1751-9667","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1]]},"assertion":[{"value":"2025-02-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70107"}}