{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T01:23:57Z","timestamp":1774574637093,"version":"3.50.1"},"reference-count":18,"publisher":"World Scientific Pub Co Pte Lt","issue":"03","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Artif. Intell. Tools"],"published-print":{"date-parts":[[2008,6]]},"abstract":"<jats:p> Knowledge transfer is widely held to be a primary mechanism that enables humans to quickly learn new complex concepts when given only small training sets. In this paper, we apply knowledge transfer to deep convolutional neural nets, which we argue are particularly well suited for knowledge transfer. Our initial results demonstrate that components of a trained deep convolutional neural net can constructively transfer information to another such net. Furthermore, this transfer is completed in such a way that one can envision creating a net that could learn new concepts throughout its lifetime. <\/jats:p><jats:p> The experiments we performed involved training a Deep Convolutional Neural Net (DCNN) on a large training set containing 20 different classes of handwritten characters from the NIST Special Database 19. This net was then used as a foundation for training a new net on a set of 20 different character classes from the NIST Special Database 19. The new net would keep the bottom layers of the old net (i.e. those nearest to the input) and only allow the top layers to train on the new character classes. We purposely used small training sets for the new net to force it to rely as much as possible upon transferred knowledge as opposed to a large and varied training set to learn the new set of hand written characters. Our results show a clear advantage in relying upon transferred knowledge to learn new tasks when given small training sets, if the new tasks are sufficiently similar to the previously mastered one. However, this advantage decreases as training sets increase in size. <\/jats:p>","DOI":"10.1142\/s0218213008004059","type":"journal-article","created":{"date-parts":[[2008,6,24]],"date-time":"2008-06-24T05:38:40Z","timestamp":1214285920000},"page":"555-567","source":"Crossref","is-referenced-by-count":30,"title":["KNOWLEDGE TRANSFER IN DEEP CONVOLUTIONAL NEURAL NETS"],"prefix":"10.1142","volume":"17","author":[{"given":"STEVEN","family":"GUTSTEIN","sequence":"first","affiliation":[{"name":"Computer Science Department, University of Texas at El Paso, El Paso, Texas, 79968, USA"}]},{"given":"OLAC","family":"FUENTES","sequence":"additional","affiliation":[{"name":"Computer Science Department, University of Texas at El Paso, El Paso, Texas, 79968, USA"}]},{"given":"ERIC","family":"FREUDENTHAL","sequence":"additional","affiliation":[{"name":"Computer Science Department, University of Texas at El Paso, El Paso, Texas, 79968, USA"}]}],"member":"219","published-online":{"date-parts":[[2011,11,21]]},"reference":[{"key":"rf1","volume-title":"Machine Learning","author":"Mitchell Tom","year":"1997"},{"key":"rf2","volume":"10","author":"Abu-Mostafa Yasser","journal-title":"Journal of Complexity"},{"key":"rf3","unstructured":"Lorien Y.\u00a0Pratt, Advances in Neural Information Processing Systems\u00a05, eds. Stephen\u00a0Jos\u00e9 Hanson, Jack D.\u00a0Cowan and C.\u00a0Lee Giles (Morgan Kaufmann, San Mateo, CA, 1993)\u00a0pp. 204\u2013211."},{"key":"rf5","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007379606734"},{"key":"rf6","unstructured":"Rich\u00a0Caruana, Advances in Neural Information Processing Systems\u00a07, eds. G.\u00a0Tesauro, D.\u00a0Touretzky and T.\u00a0Leen (The MIT Press, 1995)\u00a0pp. 657\u2013664."},{"key":"rf7","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1080\/095400996116929","volume":"8","author":"Silver Daniel","journal-title":"Connection Science Special Issue: Transfer in Inductive Systems"},{"key":"rf8","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1613\/jair.731","volume":"12","author":"Baxter Jonathan","journal-title":"Journal of Artificial Intelligence Research"},{"key":"rf9","unstructured":"Sebastian\u00a0Thrun, Advances in Neural Information Processing Systems\u00a08, eds. David S.\u00a0Touretzky, Michael C.\u00a0Mozer and Michael E.\u00a0Hasselmo (The MIT Press, 1996)\u00a0pp. 640\u2013646."},{"key":"rf10","unstructured":"Jonathan\u00a0Baxter and Peter L.\u00a0Bartlett, Advances in Neural Information Processing Systems\u00a010, eds. Michael I.\u00a0Jordan, Michael J.\u00a0Kearns and Sara A.\u00a0Solla (1998)\u00a0pp. 245\u2013251."},{"key":"rf12","volume-title":"Advances in Neural Information Processing Systems","volume":"6","author":"Bromley J.","year":"1993"},{"key":"rf13","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-46805-6_19"},{"key":"rf14","doi-asserted-by":"publisher","DOI":"10.1007\/BF02551274"},{"key":"rf15","volume-title":"Large-Scale Kernel Machines","author":"Bengio Yoshua","year":"2007"},{"key":"rf16","doi-asserted-by":"publisher","DOI":"10.1162\/08997660260293319"},{"key":"rf17","doi-asserted-by":"publisher","DOI":"10.1007\/b11963"},{"key":"rf18","unstructured":"Scott E.\u00a0Fahlman and Christian\u00a0Lebiere, Advances in Neural Information Processing Systems, Denver 1989\u00a02, ed. D. S.\u00a0Touretzky (Morgan Kaufmann, San Mateo, 1990)\u00a0pp. 524\u2013532."},{"key":"rf20","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2006.18.7.1527"},{"key":"rf21","doi-asserted-by":"publisher","DOI":"10.1126\/science.1127647"}],"container-title":["International Journal on Artificial Intelligence Tools"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0218213008004059","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,8,7]],"date-time":"2019-08-07T13:10:13Z","timestamp":1565183413000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/abs\/10.1142\/S0218213008004059"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,6]]},"references-count":18,"journal-issue":{"issue":"03","published-online":{"date-parts":[[2011,11,21]]},"published-print":{"date-parts":[[2008,6]]}},"alternative-id":["10.1142\/S0218213008004059"],"URL":"https:\/\/doi.org\/10.1142\/s0218213008004059","relation":{},"ISSN":["0218-2130","1793-6349"],"issn-type":[{"value":"0218-2130","type":"print"},{"value":"1793-6349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2008,6]]}}}