{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T06:09:15Z","timestamp":1778220555033,"version":"3.51.4"},"reference-count":40,"publisher":"Oxford University Press (OUP)","issue":"16","license":[{"start":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T00:00:00Z","timestamp":1614038400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000002","name":"U.S. National Institutes of Health","doi-asserted-by":"publisher","award":["P41GM103712"],"award-info":[{"award-number":["P41GM103712"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"U.S. National Institutes of Health","doi-asserted-by":"publisher","award":["R01GM134020"],"award-info":[{"award-number":["R01GM134020"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"U.S. National Institutes of Health","doi-asserted-by":"publisher","award":["K01MH123896"],"award-info":[{"award-number":["K01MH123896"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"U.S. National Science Foundation","doi-asserted-by":"publisher","award":["DBI-1949629"],"award-info":[{"award-number":["DBI-1949629"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"U.S. National Science Foundation","doi-asserted-by":"publisher","award":["IIS-2007595"],"award-info":[{"award-number":["IIS-2007595"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014599","name":"Mark Foundation For Cancer Research","doi-asserted-by":"publisher","award":["19-044-ASP"],"award-info":[{"award-number":["19-044-ASP"]}],"id":[{"id":"10.13039\/100014599","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014599","name":"Mark Foundation For Cancer Research","doi-asserted-by":"publisher","award":["AMD COVID-19 HPC"],"award-info":[{"award-number":["AMD COVID-19 HPC"]}],"id":[{"id":"10.13039\/100014599","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008047","name":"Carnegie Mellon University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100008047","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100018489","name":"Center for Machine Learning and Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100018489","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,8,25]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Cryo-Electron Tomography (cryo-ET) is a 3D bioimaging tool that visualizes the structural and spatial organization of macromolecules at a near-native state in single cells, which has broad applications in life science. However, the systematic structural recognition and recovery of macromolecules captured by cryo-ET are difficult due to high structural complexity and imaging limits. Deep learning-based subtomogram classification has played critical roles for such tasks. As supervised approaches, however, their performance relies on sufficient and laborious annotation on a large training dataset.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>To alleviate this major labeling burden, we proposed a Hybrid Active Learning (HAL) framework for querying subtomograms for labeling from a large unlabeled subtomogram pool. Firstly, HAL adopts uncertainty sampling to select the subtomograms that have the most uncertain predictions. This strategy enforces the model to be aware of the inductive bias during classification and subtomogram selection, which satisfies the discriminativeness principle in AL literature. Moreover, to mitigate the sampling bias caused by such strategy, a discriminator is introduced to judge if a certain subtomogram is labeled or unlabeled and subsequently the model queries the subtomogram that have higher probabilities to be unlabeled. Such query strategy encourages to match the data distribution between the labeled and unlabeled subtomogram samples, which essentially encodes the representativeness criterion into the subtomogram selection process. Additionally, HAL introduces a subset sampling strategy to improve the diversity of the query set, so that the information overlap is decreased between the queried batches and the algorithmic efficiency is improved. Our experiments on subtomogram classification tasks using both simulated and real data demonstrate that we can achieve comparable testing performance (on average only 3% accuracy drop) by using less than 30% of the labeled subtomograms, which shows a very promising result for subtomogram classification task with limited labeling resources.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>https:\/\/github.com\/xulabs\/aitom.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btab123","type":"journal-article","created":{"date-parts":[[2021,2,22]],"date-time":"2021-02-22T20:11:51Z","timestamp":1614024711000},"page":"2340-2346","source":"Crossref","is-referenced-by-count":11,"title":["Active learning to classify macromolecular structures <i>in situ<\/i> for less supervision in cryo-electron tomography"],"prefix":"10.1093","volume":"37","author":[{"given":"Xuefeng","family":"Du","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Wisconsin-Madison , Madison, WI 53706,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haohan","family":"Wang","sequence":"additional","affiliation":[{"name":"Language Technologies Institute, Carnegie Mellon University , Pittsburgh, PA 15213,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhenxi","family":"Zhu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Beijing University of Posts and Telecommunications , Beijing 100876,","place":["China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiangrui","family":"Zeng","sequence":"additional","affiliation":[{"name":"Computational Biology Department, Carnegie Mellon University , Pittsburgh, PA 15213,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yi-Wei","family":"Chang","sequence":"additional","affiliation":[{"name":"Department of Biochemistry and Biophysics, University of Pennsylvania , Philadelphia, PA 19104,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jing","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of California\u2014Irvine , Irvine, CA 92697,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eric","family":"Xing","sequence":"additional","affiliation":[{"name":"Machine Learning Department, Carnegie Mellon University , Pittsburgh, PA 15213,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0881-5891","authenticated-orcid":false,"given":"Min","family":"Xu","sequence":"additional","affiliation":[{"name":"Computational Biology Department, Carnegie Mellon University , Pittsburgh, PA 15213,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,2,23]]},"reference":[{"key":"2025052320022473200_btab123-B1","author":"Ash","year":"2020"},{"key":"2025052320022473200_btab123-B2","author":"Azimi","year":"2012"},{"key":"2025052320022473200_btab123-B3","doi-asserted-by":"crossref","first-page":"930","DOI":"10.1109\/18.256500","article-title":"Universal approximation bounds for superpositions of a sigmoidal function","volume":"39","author":"Barron","year":"1993","journal-title":"IEEE Trans. Inf. Theory"},{"key":"2025052320022473200_btab123-B4","first-page":"9368","author":"Beluch","year":"2018"},{"key":"2025052320022473200_btab123-B5","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/s10994-009-5152-4","article-title":"A theory of learning from different domains","volume":"79","author":"Ben-David","year":"2010","journal-title":"Mach. Learn"},{"key":"2025052320022473200_btab123-B6","doi-asserted-by":"crossref","first-page":"1227","DOI":"10.1007\/s00138-018-0949-4","article-title":"Improved deep learning-based macromolecules structure classification from electron cryo-tomograms","volume":"29","author":"Che","year":"2018","journal-title":"Mach. Vis. Appl"},{"key":"2025052320022473200_btab123-B7","doi-asserted-by":"crossref","first-page":"1767","DOI":"10.1016\/j.tcs.2010.12.054","article-title":"Two faces of active learning","volume":"412","author":"Dasgupta","year":"2011","journal-title":"Theor. Comput. Sci"},{"key":"2025052320022473200_btab123-B8","first-page":"148","author":"Du","year":"2019"},{"key":"2025052320022473200_btab123-B9","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/0893-6080(89)90003-8","article-title":"On the approximate realization of continuous mappings by neural networks","volume":"2","author":"Funahashi","year":"1989","journal-title":"Neural Netw"},{"key":"2025052320022473200_btab123-B10","first-page":"1183","author":"Gal","year":"2017"},{"key":"2025052320022473200_btab123-B11","first-page":"1180","author":"Ganin","year":"2015"},{"key":"2025052320022473200_btab123-B12","first-page":"443","author":"Gilad-Bachrach","year":"2005"},{"key":"2025052320022473200_btab123-B13","author":"Gissin","year":"2019"},{"key":"2025052320022473200_btab123-B14","doi-asserted-by":"crossref","first-page":"696","DOI":"10.1016\/j.cell.2017.12.030","article-title":"In situ structure of neuronal c9orf72 poly-ga aggregates reveals proteasome recruitment","volume":"172","author":"Guo","year":"2018","journal-title":"Cell"},{"key":"2025052320022473200_btab123-B15","author":"Huang","year":"2016"},{"key":"2025052320022473200_btab123-B16","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1038\/nmeth.3806","article-title":"Empiar: a public archive for raw electron microscopy image data","volume":"13","author":"Iudin","year":"2016","journal-title":"Nat. Methods"},{"key":"2025052320022473200_btab123-B17","first-page":"2372","author":"Joshi","year":"2009"},{"key":"2025052320022473200_btab123-B18","first-page":"180","author":"Kifer","year":"2004"},{"key":"2025052320022473200_btab123-B19","author":"Kim","year":"2020"},{"key":"2025052320022473200_btab123-B20","first-page":"6","author":"Lee","year":"2019"},{"key":"2025052320022473200_btab123-B21","first-page":"271","author":"Liu","year":"2018"},{"key":"2025052320022473200_btab123-B22","first-page":"30","author":"Liu","year":"2019"},{"key":"2025052320022473200_btab123-B23","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1186\/s12859-016-1283-3","article-title":"Simulating cryo electron tomograms of crowded cell cytoplasm for assessment of automated particle picking","volume":"17","author":"Long","year":"2016","journal-title":"BMC Bioinformatics"},{"key":"2025052320022473200_btab123-B24","author":"Nguyen","year":"2004"},{"key":"2025052320022473200_btab123-B25"},{"key":"2025052320022473200_btab123-B26","author":"Sener","year":"2018"},{"key":"2025052320022473200_btab123-B27","first-page":"287","author":"Seung","year":"1992"},{"key":"2025052320022473200_btab123-B28","first-page":"1308","author":"Shui","year":"2020"},{"key":"2025052320022473200_btab123-B29","first-page":"5971","author":"Sinha","year":"2019"},{"key":"2025052320022473200_btab123-B30","first-page":"5117","author":"Tang","year":"2019"},{"key":"2025052320022473200_btab123-B31","first-page":"6295","author":"Tran","year":"2019"},{"key":"2025052320022473200_btab123-B32","first-page":"964","author":"Wang","year":"2018"},{"key":"2025052320022473200_btab123-B33","doi-asserted-by":"crossref","first-page":"2591","DOI":"10.1109\/TCSVT.2016.2589879","article-title":"Cost-effective active learning for deep image classification","volume":"27","author":"Wang","year":"2017","journal-title":"IEEE Trans. Circuits Syst. Video Technol"},{"key":"2025052320022473200_btab123-B34","first-page":"158","author":"Wang","year":"2013"},{"key":"2025052320022473200_btab123-B35","first-page":"683","author":"Woo","year":"2012"},{"key":"2025052320022473200_btab123-B36","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1006\/jsbi.1998.4080","article-title":"Situs: a package for docking crystal structures into low-resolution maps from electron microscopy","volume":"125","author":"Wriggers","year":"1999","journal-title":"J. Struct. Biol"},{"key":"2025052320022473200_btab123-B37","doi-asserted-by":"crossref","first-page":"i13","DOI":"10.1093\/bioinformatics\/btx230","article-title":"Deep learning-based subdivision approach for large scale macromolecules structure recovery from electron cryo tomograms","volume":"33","author":"Xu","year":"2017","journal-title":"Bioinformatics"},{"key":"2025052320022473200_btab123-B38","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1016\/j.patcog.2018.06.004","article-title":"A benchmark and comparison of active learning for logistic regression","volume":"83","author":"Yang","year":"2018","journal-title":"Pattern Recogn"},{"key":"2025052320022473200_btab123-B39","first-page":"575","author":"Yin","year":"2017"},{"key":"2025052320022473200_btab123-B40","author":"Zhu","year":"2017"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btab123\/36599597\/btab123.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/16\/2340\/59818313\/btab123.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/16\/2340\/59818313\/btab123.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T00:02:32Z","timestamp":1748044952000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/37\/16\/2340\/6147019"}},"subtitle":[],"editor":[{"given":"Jinbo","family":"Xu","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2021,2,23]]},"references-count":40,"journal-issue":{"issue":"16","published-print":{"date-parts":[[2021,8,25]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btab123","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,8,15]]},"published":{"date-parts":[[2021,2,23]]}}}