{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T11:17:54Z","timestamp":1784114274485,"version":"3.55.0"},"reference-count":47,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2025,1,19]],"date-time":"2025-01-19T00:00:00Z","timestamp":1737244800000},"content-version":"vor","delay-in-days":58,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,11,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Cryo-electron tomography (cryo-ET) is confronted with the intricate task of unveiling novel structures. General class discovery (GCD) seeks to identify new classes by learning a model that can pseudo-label unannotated (novel) instances solely using supervision from labeled (base) classes. While 2D GCD for image data has made strides, its 3D counterpart remains unexplored. Traditional methods encounter challenges due to model bias and limited feature transferability when clustering unlabeled 2D images into known and potentially novel categories based on labeled data. To address this limitation and extend GCD to 3D structures, we propose an innovative approach that harnesses a pretrained 2D transformer, enriched by an effective weight inflation strategy tailored for 3D adaptation, followed by a decoupled prototypical network. Incorporating the power of pretrained weight-inflated Transformers, we further integrate CLIP, a vision-language model to incorporate textual information. Our method synergizes a graph convolutional network with CLIP\u2019s frozen text encoder, preserving class neighborhood structure. In order to effectively represent unlabeled samples, we devise semantic distance distributions, by formulating a bipartite matching problem for category prototypes using a decoupled prototypical network. Empirical results unequivocally highlight our method\u2019s potential in unveiling hitherto unknown structures in cryo-ET. By bridging the gap between 2D GCD and the distinctive challenges of 3D cryo-ET data, our approach paves novel avenues for exploration and discovery in this domain.<\/jats:p>","DOI":"10.1093\/bib\/bbae570","type":"journal-article","created":{"date-parts":[[2025,1,19]],"date-time":"2025-01-19T23:29:29Z","timestamp":1737329369000},"source":"Crossref","is-referenced-by-count":3,"title":["Towards molecular structure discovery from cryo-ET density volumes via modelling auxiliary semantic prototypes"],"prefix":"10.1093","volume":"26","author":[{"given":"Ashwin","family":"Nair","sequence":"first","affiliation":[{"name":"Department of Data Science, Indian Institute of Science Education and Research , Vithura, 695551, Kerela,","place":["India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xingjian","family":"Li","sequence":"additional","affiliation":[{"name":"Computational Biology Department, Carnegie Mellon University , Pittsburgh, PA, 15213,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bhupendra","family":"Solanki","sequence":"additional","affiliation":[{"name":"Machine Learning and Visual Computing Lab, Indian Institute of Technology Bombay , Powai, 400076, Maharashtra,","place":["India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Souradeep","family":"Mukhopadhyay","sequence":"additional","affiliation":[{"name":"Computer Science and Automation, Indian Institute of Science , CV Raman Rd, 560012, Karnataka,","place":["India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ankit","family":"Jha","sequence":"additional","affiliation":[{"name":"Machine Learning and Visual Computing Lab, Indian Institute of Technology Bombay , Powai, 400076, Maharashtra,","place":["India"]},{"name":"Computer Science and Engineering, LNM Institute of Information Technology , Jaipur, 302031, Rajasthan,","place":["India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mostofa","family":"Rafid Uddin","sequence":"additional","affiliation":[{"name":"Computational Biology Department, Carnegie Mellon University , Pittsburgh, PA, 15213,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mainak","family":"Singha","sequence":"additional","affiliation":[{"name":"Machine Learning and Visual Computing Lab, Indian Institute of Technology Bombay , Powai, 400076, Maharashtra,","place":["India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Biplab","family":"Banerjee","sequence":"additional","affiliation":[{"name":"Machine Learning and Visual Computing Lab, Indian Institute of Technology Bombay , Powai, 400076, Maharashtra,","place":["India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min","family":"Xu","sequence":"additional","affiliation":[{"name":"Computational Biology Department, Carnegie Mellon University , Pittsburgh, PA, 15213,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,1,18]]},"reference":[{"key":"2025011923291596900_ref1","doi-asserted-by":"publisher","first-page":"1386","DOI":"10.1038\/s41592-021-01275-4","article-title":"Deep learning improves macromolecule identification in 3d cellular cryo-electron tomograms","volume":"18","author":"Moebel","year":"2020","journal-title":"Nat Methods"},{"key":"2025011923291596900_ref2","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1109\/TCBB.2021.3065986","article-title":"Macromolecules structural classification with a 3d dilated dense network in cryo-electron tomography","volume":"19","author":"Gao","year":"2021","journal-title":"IEEE\/ACM Trans Comput Biol Bioinform"},{"key":"2025011923291596900_ref3","first-page":"7482","article-title":"Generalized category discovery","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Sagar, Vaze","year":"2022"},{"key":"2025011923291596900_ref4","first-page":"3479","article-title":"Promptcal: contrastive affinity learning via auxiliary prompts for generalized novel category discovery","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zhang","year":"2023"},{"key":"2025011923291596900_ref5","article-title":"Adapting pre-trained vision transformers from 2d to 3d through weight inflation improves medical image segmentation","author":"Zhang","year":"2023"},{"key":"2025011923291596900_ref6","doi-asserted-by":"crossref","first-page":"4724","DOI":"10.1109\/CVPR.2017.502","article-title":"Quo vadis, action recognition? A new model and the kinetics dataset","volume-title":"2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Carreira","year":"2017"},{"key":"2025011923291596900_ref7","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI Blog"},{"key":"2025011923291596900_ref8","first-page":"8748","article-title":"Learning transferable visual models from natural language supervision","volume-title":"International Conference on Machine Learning","author":"Radford","year":"2021"},{"key":"2025011923291596900_ref9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40649-019-0069-y","article-title":"Graph convolutional networks: a comprehensive review","volume":"6","author":"Zhang","year":"2019","journal-title":"Comput. Soc. Netw."},{"key":"2025011923291596900_ref10","article-title":"Discovering new intents via constrained deep adaptive clustering with cluster refinement","author":"Lin","year":"2019"},{"key":"2025011923291596900_ref11","article-title":"Discovering new intents with deep aligned clustering","volume":"35","author":"Zhang","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"2025011923291596900_ref12","article-title":"Incremental false negative detection for contrastive learning","author":"Chen","year":"2021"},{"key":"2025011923291596900_ref13","doi-asserted-by":"crossref","first-page":"16070","DOI":"10.1109\/CVPR52688.2022.01562","article-title":"Slic: self-supervised learning with iterative clustering for human action videos","volume-title":"2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Khorasgani","year":"2022"},{"key":"2025011923291596900_ref14","article-title":"Transunet: transformers make strong encoders for medical image segmentation","author":"Chen","year":"2021"},{"key":"2025011923291596900_ref15","article-title":"Cotr: efficiently bridging cnn and transformer for 3d medical image segmentation","author":"Xie","journal-title":"Medical Image Computing and Computer Assisted Intervention\u2013MICCAI 2021: 24th International Conference, Strasbourg, France, September 27\u2013October 1, 2021, Proceedings, Part III 24 (pp. 171\u2013180)"},{"key":"2025011923291596900_ref16","article-title":"Open-world semi-supervised learning","author":"Cao","year":"2021"},{"key":"2025011923291596900_ref17","article-title":"Missformer: an effective medical image segmentation transformer","author":"Huang","year":"2021"},{"key":"2025011923291596900_ref18","article-title":"Utnet: a hybrid transformer architecture for medical image segmentation","author":"Gao","journal-title":"Medical Image Computing and Computer Assisted Intervention\u2013MICCAI 2021: 24th International Conference, Strasbourg, France, September 27\u2013October 1, 2021, Proceedings, Part III 24, pp. 61\u201371"},{"key":"2025011923291596900_ref19","first-page":"1748","article-title":"Unetr: transformers for 3d medical image segmentation","volume-title":"2022 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV)","author":"Hatamizadeh","year":"2021"},{"key":"2025011923291596900_ref20","first-page":"3156","article-title":"Show and tell: a neural image caption generator","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Vinyals","year":"2015"},{"key":"2025011923291596900_ref21","doi-asserted-by":"publisher","first-page":"584","DOI":"10.1007\/978-3-319-10590-1_38","article-title":"Neural codes for image retrieval","volume-title":"European Conference on Computer Vision","author":"Babenko","year":"2014"},{"key":"2025011923291596900_ref22","first-page":"4904","article-title":"Scaling up visual and vision-language representation learning with noisy text supervision","volume-title":"International Conference on Machine Learning","author":"Jia","year":"2021"},{"key":"2025011923291596900_ref23","doi-asserted-by":"publisher","first-page":"2337","DOI":"10.1007\/s11263-022-01653-1","article-title":"Learning to prompt for vision-language models","volume":"130","author":"Zhou","year":"2022","journal-title":"Int J Comput Vis"},{"key":"2025011923291596900_ref24","first-page":"16816","article-title":"Conditional prompt learning for vision-language models","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zhou","year":"2022"},{"key":"2025011923291596900_ref25","article-title":"Test-time prompt tuning for zero-shot generalization in vision-language models","volume-title":"Advances in Neural Information Processing Systems 35 (2022): 14274-14289","author":"Shu"},{"key":"2025011923291596900_ref26","article-title":"Prototypical networks for few-shot learning","author":"Snell","journal-title":"Advances in neural information processing systems 30 (2017)"},{"key":"2025011923291596900_ref27","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR46437.2021.01362","article-title":"Prototypical cross-domain self-supervised learning for few-shot unsupervised domain adaptation","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Yue","year":"2021"},{"key":"2025011923291596900_ref28","doi-asserted-by":"crossref","first-page":"2234","DOI":"10.1109\/CVPR.2019.00234","article-title":"Transferrable prototypical networks for unsupervised domain adaptation","volume-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Pan","year":"2019"},{"key":"2025011923291596900_ref29","volume-title":"This Looks Like That: Deep Learning for Interpretable Image Recognition","author":"Chen"},{"key":"2025011923291596900_ref30","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR46437.2021.01469","article-title":"Neural prototype trees for interpretable fine-grained image recognition","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Nauta"},{"key":"2025011923291596900_ref31","first-page":"248","article-title":"Imagenet: a large-scale hierarchical image database","volume-title":"2009 IEEE Conference on Computer Vision and Pattern Recognition","author":"Deng"},{"key":"2025011923291596900_ref32","article-title":"Rethinking knowledge graph propagation for zero-shot learning","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Xing Michael Kampffmeyer"},{"key":"2025011923291596900_ref33","article-title":"Generalized category discovery with decoupled prototypical network","volume":"37","author":"An","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"2025011923291596900_ref34","article-title":"The Hungarian method for the assignment problem","volume":"2","author":"Kuhn","journal-title":"Naval Research Logistics Quarterly"},{"key":"2025011923291596900_ref35","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2021.emnlp-main.144","article-title":"Few-shot intent detection via contrastive pre-training and fine-tuning","volume-title":"Conference on Empirical Methods in Natural Language Processing","author":"Zhang","year":"2021"},{"key":"2025011923291596900_ref36","article-title":"Promptcal: contrastive affinity learning via auxiliary prompts for generalized novel category discovery","author":"Zhang","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"2025011923291596900_ref37","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.34257","article-title":"Routine single particle cryoem sample and grid characterization by tomography","volume":"7","author":"Noble","journal-title":"Elife"},{"key":"2025011923291596900_ref38","doi-asserted-by":"publisher","first-page":"696","DOI":"10.1016\/j.cell.2017.12.030","article-title":"In situ structure of neuronal c9orf72 poly-ga aggregates reveals proteasome recruitment","volume":"172","author":"Guo","year":"2018","journal-title":"Cell"},{"key":"2025011923291596900_ref39","article-title":"k-means++: the advantages of careful seeding","volume-title":"SODA 07 Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms","author":"Arthur","year":"2007"},{"key":"2025011923291596900_ref40","first-page":"7579","article-title":"Dynamic conceptional contrastive learning for generalized category discovery","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Nan","year":"2023"},{"key":"2025011923291596900_ref41","first-page":"16590","article-title":"Parametric classification for generalized category discovery: a baseline study","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV)","author":"Wen","year":"2023"},{"key":"2025011923291596900_ref42","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1186\/s12859-016-1283-3","article-title":"Simulating cryo electron tomograms of crowded cell cytoplasm for assessment of automated particle picking","volume":"17","author":"Pei","year":"2016","journal-title":"BMC Bioinform"},{"key":"2025011923291596900_ref43","doi-asserted-by":"crossref","first-page":"9630","DOI":"10.1109\/ICCV48922.2021.00951","article-title":"Emerging properties in self-supervised vision transformers","volume-title":"2021 IEEE\/CVF International Conference on Computer Vision (ICCV)","author":"Caron","year":"2021"},{"key":"2025011923291596900_ref44","first-page":"11479","article-title":"Rethinking knowledge graph propagation for zero-shot learning","volume-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Kampffmeyer","year":"2018"},{"key":"2025011923291596900_ref45","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014"},{"key":"2025011923291596900_ref46","doi-asserted-by":"publisher","DOI":"10.1128\/JB.00100-17","article-title":"Uncharacterized bacterial structures revealed by electron cryotomography","volume":"199","author":"Dobro","journal-title":"Journal of bacteriology"},{"key":"2025011923291596900_ref47","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.52058","article-title":"Cryo electron tomography with Volta phase plate reveals novel structural foundations of the 96-nm axonemal repeat in the pathogen trypanosoma brucei","volume":"8","author":"Imhof","journal-title":"Elife"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/1\/bbae570\/61491635\/bbae570.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/1\/bbae570\/61491635\/bbae570.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,19]],"date-time":"2025-01-19T23:29:40Z","timestamp":1737329380000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbae570\/7960051"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,22]]},"references-count":47,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,11,22]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbae570","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,1]]},"published":{"date-parts":[[2024,11,22]]},"article-number":"bbae570"}}