{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T03:24:52Z","timestamp":1790565892464,"version":"4.1.0"},"reference-count":27,"publisher":"Public Library of Science (PLoS)","issue":"5","license":[{"start":{"date-parts":[[2025,5,30]],"date-time":"2025-05-30T00:00:00Z","timestamp":1748563200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005189","name":"Estonian Research Competency Council","doi-asserted-by":"publisher","award":["PRG1604"],"award-info":[{"award-number":["PRG1604"]}],"id":[{"id":"10.13039\/501100005189","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003510","name":"Estonian Ministry of Education and Research","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003510","id-type":"DOI","asserted-by":"crossref"}]},{"name":"European Union\u2019s Horizon 2020 Research and Innovation Programme","award":["952060"],"award-info":[{"award-number":["952060"]}]},{"name":"Estonian Centre of Excellence in Artificial Intelligence"},{"name":"Revvity","award":["NA"],"award-info":[{"award-number":["NA"]}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>A fundamental problem in cell and tissue biology is finding cells in microscopy images. Traditionally, this detection has been performed by segmenting the pixel intensities. However, these methods struggle to delineate cells in more densely packed micrographs, where local decisions about boundaries are not trivial. Here, we develop a new methodology to decompose microscopy images into individual cells by making object-level decisions. We formulate the segmentation problem as training a flexible factorized representation of the image. To this end, we introduce Kaizen, an approach inspired by predictive coding in the brain that maintains an internal representation of an image while generating object hypotheses over the external image, and keeping the ones that improve the consistency of internal and external representations. We achieve this by training a Vector Quantised-Variational AutoEncoder (VQ-VAE). During inference, the VQ-VAE is iteratively applied on locations where the internal representation differs from the external image, making new guesses, and keeping only the ones that improve the overall image prediction until the internal representation matches the input. We demonstrate Kaizen\u2019s merits on two fluorescence microscopy datasets, improving the separation of nuclei and neuronal cells in cell culture images.<\/jats:p>","DOI":"10.1371\/journal.pone.0313549","type":"journal-article","created":{"date-parts":[[2025,5,30]],"date-time":"2025-05-30T13:25:03Z","timestamp":1748611503000},"page":"e0313549","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":3,"title":["Kaizen: Decomposing cellular images with VQ-VAE"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6037-4747","authenticated-orcid":true,"given":"Daniel","family":"Majoral","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5414-6089","authenticated-orcid":true,"given":"Marharyta","family":"Domnich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,5,30]]},"reference":[{"issue":"1","key":"pone.0313549.ref001","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1038\/s41746-020-00376-2","article-title":"Deep learning-enabled medical computer vision","volume":"4","author":"A Esteva","year":"2021","journal-title":"NPJ Digit Med"},{"key":"pone.0313549.ref002","doi-asserted-by":"crossref","unstructured":"Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2015, pp. 234\u201341.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"pone.0313549.ref003","doi-asserted-by":"crossref","unstructured":"He K, Gkioxari G, Doll\u00e1r P, Girshick R. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). IEEE; 2017, pp. 2961\u20139.","DOI":"10.1109\/ICCV.2017.322"},{"issue":"1038","key":"pone.0313549.ref004","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1098\/rstb.1980.0090","article-title":"Perceptions as hypotheses","volume":"290","author":"RL Gregory","year":"1980","journal-title":"Philos Trans R Soc Lond B Biol Sci"},{"key":"pone.0313549.ref005","unstructured":"Kingma DP, Welling M. Auto-encoding variational Bayes. In: 2nd International Conference on Learning Representations, ICLR 2014 - Conference Track Proceedings. 2013. doi: 10.48550\/arxiv.1312.6114"},{"key":"pone.0313549.ref006","unstructured":"Agustsson E, Mentzer F, Tschannen M, Cavigelli L, Timofte R, Benini L. Soft-to-hard vector quantization for end-to-end learning compressible representations. In: Advances in Neural Information Processing Systems. 2017."},{"key":"pone.0313549.ref007","unstructured":"Theis L, Shi W, Cunningham A, Husz\u00e1r F. Lossy image compression with compressive autoencoders. arXiv, preprint, 2017."},{"key":"pone.0313549.ref008","unstructured":"VanDenOord A, Vinyals O. Neural discrete representation learning. In: Advances in Neural Information Processing Systems (NeurIPS). 2017."},{"key":"pone.0313549.ref009","unstructured":"Razavi A, Van den Oord A, Vinyals O. Generating diverse high-fidelity images with VQ-VAE-2. In: Advances in Neural Information Processing Systems. 2019."},{"key":"pone.0313549.ref010","doi-asserted-by":"crossref","unstructured":"Hollandi R, Szkalisity A, Toth T, Tasnadi E, Molnar C, Mathe B. A deep learning framework for nucleus segmentation using image style transfer. bioRxiv, preprint, 2019:580605.","DOI":"10.1101\/580605"},{"issue":"1","key":"pone.0313549.ref011","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1111\/jmi.13038","article-title":"Practical segmentation of nuclei in brightfield cell images with neural networks trained on fluorescently labelled samples","volume":"284","author":"D Fishman","year":"2021","journal-title":"J Microsc"},{"key":"pone.0313549.ref012","doi-asserted-by":"crossref","unstructured":"Schmidt U, Weigert M, Broaddus C, Myers G. Cell detection with star-convex polygons. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2018, pp. 265\u201373.","DOI":"10.1007\/978-3-030-00934-2_30"},{"key":"pone.0313549.ref013","doi-asserted-by":"crossref","unstructured":"Weigert M, Schmidt U, Haase R, Sugawara K, Myers G. Star-convex polyhedra for 3D object detection and segmentation in microscopy. In: 2025 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE; 2020, pp. 3666\u201373.","DOI":"10.1109\/WACV45572.2020.9093435"},{"key":"pone.0313549.ref014","doi-asserted-by":"crossref","unstructured":"Walter F, Damrich S, Hamprecht F. Multistar: instance segmentation of overlapping objects with star-convex polygons. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). IEEE; 2021, pp. 295\u20138.","DOI":"10.1109\/ISBI48211.2021.9433769"},{"key":"pone.0313549.ref015","doi-asserted-by":"crossref","unstructured":"Mandal S, Uhlmann V. Splinedist: automated cell segmentation with spline curves. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). IEEE; 2021, pp. 1082\u20136. doi: 10.1109\/isbi48211.2021.9433928","DOI":"10.1109\/ISBI48211.2021.9433928"},{"issue":"1","key":"pone.0313549.ref016","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1038\/s41592-020-01018-x","article-title":"Cellpose: a generalist algorithm for cellular segmentation","volume":"18","author":"C Stringer","year":"2021","journal-title":"Nat Methods"},{"issue":"12","key":"pone.0313549.ref017","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41592-022-01663-4","article-title":"Cellpose 2.0: how to train your own model","volume":"19","author":"M Pachitariu","year":"2022","journal-title":"Nat Methods"},{"key":"pone.0313549.ref018","unstructured":"Jang W, Wei D, Zhang X, Leahy B, Yang H, Tompkin J. Learning vector quantized shape code for amodal blastomere instance segmentation. arXiv, preprint, 2020."},{"key":"pone.0313549.ref019","unstructured":"Park E, Berg A. Learning to decompose for object detection and instance segmentation. arXiv, preprint, 2015."},{"key":"pone.0313549.ref020","unstructured":"Burgess CP, Matthey L, Watters N, Kabra R, Higgins I, Botvinick M. Monet: unsupervised scene decomposition and representation. arXiv, preprint, 2019."},{"key":"pone.0313549.ref021","unstructured":"Greff K, Kaufman R, Kabra R, Watters N, Burgess C, Zoran D. Multi-object representation learning with iterative variational inference. In: Proceedings of the International Conference on Machine Learning. PMLR; 2019, pp. 2424\u201333."},{"key":"pone.0313549.ref022","unstructured":"Locatello F, Weissenborn D, Unterthiner T, Mahendran A, Heigold G, Uszkoreit J. Object-centric learning with slot attention. In: Advances in Neural Information Processing Systems 33 (NeurIPS 2020). 2020, pp. 11525\u201338."},{"key":"pone.0313549.ref023","unstructured":"Sabour S, Frosst N, Hinton GE. Dynamic routing between capsules. In: Advances in Neural Information Processing Systems 30 (NIPS 2017). 2017."},{"key":"pone.0313549.ref024","unstructured":"Huang L, Chen D, Liu Y, Shen Y, Zhao D, Zhou J. Composer: creative and controllable image synthesis with composable conditions. arXiv, preprint, 2023."},{"issue":"7","key":"pone.0313549.ref025","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1038\/nmeth.2083","article-title":"Annotated high-throughput microscopy image sets for validation","volume":"9","author":"V Ljosa","year":"2012","journal-title":"Nat Methods"},{"key":"pone.0313549.ref026","unstructured":"Yu W, Lee HK, Hariharan S, Bu WY, Ahmed S. CCDB: 6843, Mus musculus, neuroblastoma. Cell Image Library. 2019. doi: 10.7295\/W9CCDB6843"},{"key":"pone.0313549.ref027","unstructured":"Kingma D, Ba J. Adam: a method for stochastic optimization. arXiv, preprint, 2014."}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0313549","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,30]],"date-time":"2025-05-30T13:25:12Z","timestamp":1748611512000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0313549"}},"subtitle":[],"editor":[{"given":"Zeheng","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,5,30]]},"references-count":27,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5,30]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0313549","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2024.10.29.620832","asserted-by":"object"}]},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,30]]}}}