{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T11:42:43Z","timestamp":1753875763982,"version":"3.41.2"},"reference-count":43,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2024,6,18]],"date-time":"2024-06-18T00:00:00Z","timestamp":1718668800000},"content-version":"vor","delay-in-days":899,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01GM134020","P41GM103712"],"award-info":[{"award-number":["R01GM134020","P41GM103712"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Deep learning models have achieved remarkable success in a wide range of natural-world tasks, such as vision, language, and speech recognition. These accomplishments are largely attributed to the availability of open-source large-scale datasets. More importantly, pre-trained foundational model learnings exhibit a surprising degree of transferability to downstream tasks, enabling efficient learning even with limited training examples. However, the application of such natural-domain models to the domain of tiny Cryo-Electron Tomography (Cryo-ET) images has been a relatively unexplored frontier. This research is motivated by the intuition that 3D Cryo-ET voxel data can be conceptually viewed as a sequence of progressively evolving video frames.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Leveraging the above insight, we propose a novel approach that involves the utilization of 3D models pre-trained on large-scale video datasets to enhance Cryo-ET subtomogram classification. Our experiments, conducted on both simulated and real Cryo-ET datasets, reveal compelling results. The use of video initialization not only demonstrates improvements in classification accuracy but also substantially reduces training costs. Further analyses provide additional evidence of the value of video initialization in enhancing subtomogram feature extraction. Additionally, we observe that video initialization yields similar positive effects when applied to medical 3D classification tasks, underscoring the potential of cross-domain knowledge transfer from video-based models to advance the state-of-the-art in a wide range of biological and medical data types.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>https:\/\/github.com\/xulabs\/aitom.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btae368","type":"journal-article","created":{"date-parts":[[2024,6,18]],"date-time":"2024-06-18T18:20:03Z","timestamp":1718734803000},"source":"Crossref","is-referenced-by-count":0,"title":["Knowledge transfer from macro-world to micro-world: enhancing 3D Cryo-ET classification through fine-tuning video-based deep models"],"prefix":"10.1093","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-1018-917X","authenticated-orcid":false,"given":"Sabhay","family":"Jain","sequence":"first","affiliation":[{"name":"Electrical Engineering Department, Indian Institute of Technology Kanpur , Kanpur, Uttar Pradesh, 208016, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8073-7552","authenticated-orcid":false,"given":"Xingjian","family":"Li","sequence":"additional","affiliation":[{"name":"Ray and Stephanie Lane Computational Biology Department, Carnegie Mellon University , Pittsburgh, Pennsylvania, 15213, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0881-5891","authenticated-orcid":false,"given":"Min","family":"Xu","sequence":"additional","affiliation":[{"name":"Ray and Stephanie Lane Computational Biology Department, Carnegie Mellon University , Pittsburgh, Pennsylvania, 15213, United States"}]}],"member":"286","published-online":{"date-parts":[[2024,6,18]]},"reference":[{"year":"2021","author":"Arnab","first-page":"6836","key":"2024072505183154500_btae368-B1"},{"key":"2024072505183154500_btae368-B2","doi-asserted-by":"crossref","first-page":"1227","DOI":"10.1007\/s00138-018-0949-4","article-title":"Improved deep learning-based macromolecules structure classification from electron cryo-tomograms","volume":"29","author":"Che","year":"2018","journal-title":"Mach Vis Appl"},{"key":"2024072505183154500_btae368-B3","first-page":"1906","article-title":"Catastrophic forgetting meets negative transfer: batch spectral shrinkage for safe transfer learning","volume":"32","author":"Chen","year":"2019","journal-title":"Adv Neural Inf Process Syst"},{"key":"2024072505183154500_btae368-B4","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.cobme.2018.12.005","article-title":"Cats or cat scans: transfer learning from natural or medical image source data sets?","volume":"9","author":"Cheplygina","year":"2019","journal-title":"Curr Opin Biomed Eng"},{"year":"2009","author":"Deng","first-page":"248","key":"2024072505183154500_btae368-B5"},{"key":"2024072505183154500_btae368-B6","doi-asserted-by":"crossref","first-page":"2340","DOI":"10.1093\/bioinformatics\/btab123","article-title":"Active learning to classify macromolecular structures in situ for less supervision in cryo-electron tomography","volume":"37","author":"Du","year":"2021","journal-title":"Bioinformatics"},{"key":"2024072505183154500_btae368-B7","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/S1364-6613(99)01294-2","article-title":"Catastrophic forgetting in connectionist networks","volume":"3","author":"French","year":"1999","journal-title":"Trends Cogn Sci"},{"key":"2024072505183154500_btae368-B8","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1017\/S0033583511000102","article-title":"Electron tomography of cells","volume":"45","author":"Gan","year":"2012","journal-title":"Q Rev Biophys"},{"key":"2024072505183154500_btae368-B9","doi-asserted-by":"crossref","first-page":"20533","DOI":"10.1007\/s11042-019-7404-z","article-title":"An attention mechanism based convolutional LSTM network for video action recognition","volume":"78","author":"Ge","year":"2019","journal-title":"Multimed Tools Appl"},{"year":"2021","author":"Gotkowski","first-page":"217","key":"2024072505183154500_btae368-B10"},{"year":"2021","author":"Gouk","key":"2024072505183154500_btae368-B11"},{"key":"2024072505183154500_btae368-B12","doi-asserted-by":"crossref","first-page":"957484","DOI":"10.3389\/fphys.2022.957484","article-title":"Self-supervised learning for macromolecular structure classification based on cryo-electron tomograms","volume":"13","author":"Gupta","year":"2022","journal-title":"Front Physiol"},{"year":"2017","author":"Hara","first-page":"3154","key":"2024072505183154500_btae368-B13"},{"year":"2018","author":"Hara","first-page":"6546","key":"2024072505183154500_btae368-B14"},{"year":"2016","author":"He","first-page":"770","key":"2024072505183154500_btae368-B15"},{"year":"2017","author":"Hon","first-page":"1166","key":"2024072505183154500_btae368-B16"},{"year":"2014","author":"Karpathy","first-page":"1725","key":"2024072505183154500_btae368-B17"},{"year":"2017","author":"Kay","key":"2024072505183154500_btae368-B18"},{"key":"2024072505183154500_btae368-B19","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1186\/s12880-022-00793-7","article-title":"Transfer learning for medical image classification: a literature review","volume":"22","author":"Kim","year":"2022","journal-title":"BMC Med Imaging"},{"key":"2024072505183154500_btae368-B20","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.bbe.2021.11.004","article-title":"Transfer learning techniques for medical image analysis: a review","volume":"42","author":"Kora","year":"2022","journal-title":"Biocybern Biomed Eng"},{"year":"2012","author":"Krizhevsky","key":"2024072505183154500_btae368-B21"},{"key":"2024072505183154500_btae368-B22","doi-asserted-by":"crossref","first-page":"e201207002","DOI":"10.5936\/csbj.201207002","article-title":"Limiting factors in single particle cryo electron tomography","volume":"1","author":"Kudryashev","year":"2012","journal-title":"Comput Struct Biotechnol J"},{"key":"2024072505183154500_btae368-B23","doi-asserted-by":"crossref","first-page":"2061516","DOI":"10.1155\/2018\/2061516","article-title":"Medical image classification based on deep features extracted by deep model and statistic feature fusion with multilayer perceptron","volume":"2018","author":"Lai","year":"2018","journal-title":"Comput Intell Neurosci"},{"key":"2024072505183154500_btae368-B24","first-page":"27249","article-title":"Improved regularization and robustness for fine-tuning in neural networks","volume":"34","author":"Li","year":"2021","journal-title":"Adv Neural Inf Process Syst"},{"year":"2019","author":"Li","key":"2024072505183154500_btae368-B25"},{"year":"2014","author":"Lin","first-page":"740","key":"2024072505183154500_btae368-B26"},{"year":"2019","author":"Liu","key":"2024072505183154500_btae368-B27"},{"key":"2024072505183154500_btae368-B28","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1186\/s12859-020-03660-w","article-title":"A unified framework for packing deformable and non-deformable subcellular structures in crowded cryo-electron tomogram simulation","volume":"21","author":"Liu","year":"2020","journal-title":"BMC Bioinformatics"},{"key":"2024072505183154500_btae368-B29","article-title":"Semi-supervised macromolecule structural classification in cellular electron cryo-tomograms using 3D autoencoding classifier","volume":"30","author":"Liu","year":"2019","journal-title":"In BMVC"},{"year":"2020","author":"Liu","first-page":"80","key":"2024072505183154500_btae368-B30"},{"key":"2024072505183154500_btae368-B31","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1083\/jcb.201304193","article-title":"Cryo-electron tomography: the challenge of doing structural biology in situ","volume":"202","author":"Lu\u010di\u0107","year":"2013","journal-title":"J Cell Biol"},{"key":"2024072505183154500_btae368-B32","doi-asserted-by":"crossref","first-page":"1386","DOI":"10.1038\/s41592-021-01275-4","article-title":"Deep learning improves macromolecule identification in 3D cellular cryo-electron tomograms","volume":"18","author":"Moebel","year":"2021","journal-title":"Nat Methods"},{"key":"2024072505183154500_btae368-B33","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1016\/j.bbagen.2017.07.020","article-title":"Cryo-electron microscopy for structural analysis of dynamic biological macromolecules","volume":"1862","author":"Murata","year":"2018","journal-title":"Biochim Biophys Acta Gen Subj"},{"key":"2024072505183154500_btae368-B34","doi-asserted-by":"crossref","first-page":"e34257","DOI":"10.7554\/eLife.34257","article-title":"Routine single particle cryoEM sample and grid characterization by tomography","volume":"7","author":"Noble","year":"2018","journal-title":"Elife"},{"key":"2024072505183154500_btae368-B35","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1186\/s12859-016-1283-3","article-title":"Simulating cryo electron tomograms of crowded cell cytoplasm for assessment of automated particle picking","volume":"17","author":"Pei","year":"2016","journal-title":"BMC Bioinformatics"},{"year":"2017","author":"Selvaraju","first-page":"618","key":"2024072505183154500_btae368-B36"},{"year":"2014","author":"Simonyan","key":"2024072505183154500_btae368-B37"},{"key":"2024072505183154500_btae368-B38","doi-asserted-by":"crossref","first-page":"3243","DOI":"10.1002\/1873-3468.13948","article-title":"The promise and the challenges of cryo-electron tomography","volume":"594","author":"Turk","year":"2020","journal-title":"FEBS Lett"},{"key":"2024072505183154500_btae368-B39","article-title":"Visualizing data using t-SNE","volume":"9","author":"Van der Maaten","year":"2008","journal-title":"J Mach Learn Res"},{"year":"2018","author":"Xuhong","first-page":"2825","key":"2024072505183154500_btae368-B40"},{"year":"2017","author":"Yang","first-page":"3891","key":"2024072505183154500_btae368-B41"},{"year":"2014","author":"Yosinski","first-page":"3320","key":"2024072505183154500_btae368-B42"},{"key":"2024072505183154500_btae368-B43","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1093\/bioinformatics\/btaa671","article-title":"Few shot domain adaptation for in situ macromolecule structural classification in cryoelectron tomograms","volume":"37","author":"Yu","year":"2021","journal-title":"Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btae368\/58272236\/btae368.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/40\/7\/btae368\/58644474\/btae368.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/40\/7\/btae368\/58644474\/btae368.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T06:19:38Z","timestamp":1721888378000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btae368\/7695863"}},"subtitle":[],"editor":[{"given":"Hanchuan","family":"Peng","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,1,1]]},"references-count":43,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2022,1,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btae368","relation":{},"ISSN":["1367-4811"],"issn-type":[{"type":"electronic","value":"1367-4811"}],"subject":[],"published-other":{"date-parts":[[2024,7]]},"published":{"date-parts":[[2022,1,1]]},"article-number":"btae368"}}