{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T17:20:57Z","timestamp":1783012857957,"version":"3.54.6"},"reference-count":35,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T00:00:00Z","timestamp":1781308800000},"content-version":"vor","delay-in-days":12,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008730","name":"Norwegian Cancer Society","doi-asserted-by":"publisher","award":["#215817"],"award-info":[{"award-number":["#215817"]}],"id":[{"id":"10.13039\/100008730","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Research Council of Norway projects","award":["#300740"],"award-info":[{"award-number":["#300740"]}]},{"name":"Research Council of Norway projects","award":["#331890"],"award-info":[{"award-number":["#331890"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Summary<\/jats:title>\n                    <jats:p>Protein language models (PLMs) capture intricate amino-acid dependencies, producing embeddings that encode rich structural, functional, and evolutionary information. Despite their potential, current extraction workflows rely on arbitrary choices, with respect to embedding layer, pooling, and padding, that frequently yield suboptimal representations for feature extraction and downstream analyses. Large-scale embedding generation is further limited by inefficiencies in computation and memory: (i) accumulating all model outputs in memory before writing to disk causes severe bottlenecks, and (ii) repeatedly embedding identical sequences to extract different modes introduces redundant computation and drastically reduces throughput and scalability. We introduce PEPE (Parallel Extraction for Protein Embeddings), a command-line tool and Python library that enables efficient, high-throughput, and multimodal extraction from protein language models. PEPE\u2019s parallelized and streaming-based architecture achieves runtimes several orders of magnitude faster than sequential approaches. Unlike conventional methods\u2014whose peak memory usage scales linearly with output size and fails when memory capacity is exceeded\u2014PEPE maintains stable, low memory consumption, enabling multimodal embedding extraction even beyond available RAM. PEPE supports a wide range of state-of-the-art and custom PLMs through a simple, flexible interface. By combining scalability, robustness, and ease of use, PEPE allows researchers to generate massive, information-rich embedding datasets efficiently, and facilitate the discovery of optimal representations for structural, functional, and evolutionary downstream tasks. By streamlining the generation of diverse embedding configurations, PEPE provides researchers with the necessary data to identify high-performing latent states for specific biological contexts without requiring additional computational resources.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and Implementation<\/jats:title>\n                    <jats:p>PEPE is a command-line tool written in Python and published under MIT license. The source code and documentation are available at https:\/\/github.com\/csi-greifflab\/pepe-cli. PEPE is also available for installation from PyPI under https:\/\/pypi.org\/project\/pepe-cli and deposited on Zenodo at https:\/\/zenodo.org\/records\/20268104.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btag375","type":"journal-article","created":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T11:51:44Z","timestamp":1781092304000},"source":"Crossref","is-referenced-by-count":0,"title":["PEPE: scalable extraction of multi-modal protein language model representations"],"prefix":"10.1093","volume":"42","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7247-1392","authenticated-orcid":false,"given":"Jahn","family":"Zhong","sequence":"first","affiliation":[{"name":"Department of Immunology, University of Oslo , Oslo,","place":["Norway"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-4021-1631","authenticated-orcid":false,"given":"Niccol\u00f2","family":"Cardente","sequence":"additional","affiliation":[{"name":"Department of Immunology, University of Oslo , Oslo,","place":["Norway"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4959-1409","authenticated-orcid":false,"given":"Geir Kjetil","family":"Sandve","sequence":"additional","affiliation":[{"name":"Scientific Computing and Machine Learning section, Department of Informatics, University of Oslo , Oslo,","place":["Norway"]},{"name":"UiORealArt Convergence Environment, University of Oslo , Oslo,","place":["Norway"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6660-1843","authenticated-orcid":false,"given":"Habib","family":"Bashour","sequence":"additional","affiliation":[{"name":"Department of Immunology, University of Oslo , Oslo,","place":["Norway"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-1718-8769","authenticated-orcid":false,"given":"Maria Francesca","family":"Abbate","sequence":"additional","affiliation":[{"name":"Department of Immunology, University of Oslo , Oslo,","place":["Norway"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2622-5032","authenticated-orcid":false,"given":"Victor","family":"Greiff","sequence":"additional","affiliation":[{"name":"Department of Immunology, University of Oslo , Oslo,","place":["Norway"]},{"name":"Imprint Labs, LLC , New York, NY,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,6,12]]},"reference":[{"key":"2026070211070675900_btag375-B1","doi-asserted-by":"publisher","author":"Barton","year":"2024","DOI":"10.1101\/2023.12.12.569610"},{"key":"2026070211070675900_btag375-B2","doi-asserted-by":"crossref","DOI":"10.1093\/bib\/bbaf381","article-title":"PLMFit: benchmarking transfer learning with protein language models for protein engineering","volume":"26","author":"Bikias","year":"2025","journal-title":"Brief Bioinform"},{"key":"2026070211070675900_btag375-B3","doi-asserted-by":"crossref","DOI":"10.7554\/eLife.82819","article-title":"Transformer-based deep learning for predicting protein properties in the life sciences","volume":"12","author":"Chandra","year":"2023","journal-title":"Elife"},{"key":"2026070211070675900_btag375-B4","author":"Chen","year":"2023"},{"key":"2026070211070675900_btag375-B5","doi-asserted-by":"publisher","author":"Chinery","year":"2024","DOI":"10.1101\/2024.03.26.586756"},{"key":"2026070211070675900_btag375-B6","doi-asserted-by":"crossref","first-page":"e113","DOI":"10.1002\/cpz1.113","article-title":"Learned embeddings from deep learning to visualize and predict protein sets","volume":"1","author":"Dallago","year":"2021","journal-title":"Curr Protoc."},{"key":"2026070211070675900_btag375-B7","doi-asserted-by":"crossref","first-page":"100036","DOI":"10.1016\/j.immuno.2024.100036","article-title":"Do domain-specific protein language models outperform general models on immunology-related tasks?","volume":"14","author":"Deutschmann","year":"2024","journal-title":"Immunoinformatics (Amst.)"},{"key":"2026070211070675900_btag375-B8","author":"Devlin","year":"2018"},{"key":"2026070211070675900_btag375-B9","author":"Elnaggar","year":"2020"},{"key":"2026070211070675900_btag375-B10","author":"Fan","year":"2025"},{"key":"2026070211070675900_btag375-B11","author":"Garbas","year":"2024"},{"key":"2026070211070675900_btag375-B12","doi-asserted-by":"crossref","first-page":"e0289030","DOI":"10.1371\/journal.pone.0289030","article-title":"Protein embeddings improve phage-host interaction prediction","volume":"18","author":"Gonzales","year":"2023","journal-title":"PLoS One"},{"key":"2026070211070675900_btag375-B13","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1038\/s41586-020-2649-2","article-title":"Array programming with NumPy","volume":"585","author":"Harris","year":"2020","journal-title":"Nature"},{"key":"2026070211070675900_btag375-B14","doi-asserted-by":"crossref","first-page":"lqae150","DOI":"10.1093\/nargab\/lqae150","article-title":"Bilingual language model for protein sequence and structure","volume":"6","author":"Heinzinger","year":"2024","journal-title":"NAR Genom Bioinform"},{"key":"2026070211070675900_btag375-B15","author":"H\u00f8ie","year":"2024"},{"key":"2026070211070675900_btag375-B16","author":"Howard","year":"2018"},{"key":"2026070211070675900_btag375-B17","doi-asserted-by":"publisher","author":"Hsu","year":"2022","DOI":"10.1101\/2022.04.10.487779"},{"key":"2026070211070675900_btag375-B18","doi-asserted-by":"publisher","author":"Hwang","year":"2023","DOI":"10.1101\/2023.04.07.536042"},{"key":"2026070211070675900_btag375-B19","doi-asserted-by":"publisher","author":"Li","year":"2024","DOI":"10.1101\/2024.02.05.578959"},{"key":"2026070211070675900_btag375-B20","doi-asserted-by":"crossref","first-page":"1123","DOI":"10.1126\/science.ade2574","article-title":"Evolutionary-scale prediction of atomic-level protein structure with a language model","volume":"379","author":"Lin","year":"2023","journal-title":"Science"},{"key":"2026070211070675900_btag375-B21","doi-asserted-by":"crossref","first-page":"bbad534","DOI":"10.1093\/bib\/bbad534","article-title":"Interpretable feature extraction and dimensionality reduction in ESM2 for protein localization prediction","volume":"25","author":"Luo","year":"2024","journal-title":"Brief Bioinform"},{"key":"2026070211070675900_btag375-B22","doi-asserted-by":"publisher","author":"Nickolls","year":"2008","DOI":"10.1145\/1401132.1401152"},{"key":"2026070211070675900_btag375-B23","doi-asserted-by":"crossref","first-page":"936","DOI":"10.1038\/s42256-021-00413-z","article-title":"The immuneML ecosystem for machine learning analysis of adaptive immune receptor repertoires","volume":"3","author":"Pavlovi\u0107","year":"2021","journal-title":"Nat Mach Intell."},{"key":"2026070211070675900_btag375-B24","author":"Radford","year":"2018"},{"key":"2026070211070675900_btag375-B25","author":"Rao","year":"2019"},{"key":"2026070211070675900_btag375-B26","author":"Rao","year":"2021"},{"key":"2026070211070675900_btag375-B27","doi-asserted-by":"crossref","first-page":"e2016239118","DOI":"10.1073\/pnas.2016239118","article-title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","volume":"118","author":"Rives","year":"2021","journal-title":"Proc Natl Acad Sci U S A"},{"key":"2026070211070675900_btag375-B28","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1038\/s41587-024-02123-4","article-title":"Designing proteins with language models","volume":"42","author":"Ruffolo","year":"2024","journal-title":"Nat Biotechnol"},{"key":"2026070211070675900_btag375-B29","author":"Tay","year":"2022"},{"key":"2026070211070675900_btag375-B30","author":"Fox System Overview\u2014University of Oslo","year":"2024"},{"key":"2026070211070675900_btag375-B31","author":"Vaswani","year":"2017"},{"key":"2026070211070675900_btag375-B32","doi-asserted-by":"crossref","first-page":"548","DOI":"10.1093\/nar\/gkad1128","article-title":"Language model-based B cell receptor sequence embeddings can effectively encode receptor specificity","volume":"52","author":"Wang","year":"2024","journal-title":"Nucleic Acids Res"},{"key":"2026070211070675900_btag375-B33","doi-asserted-by":"publisher","author":"Wang","DOI":"10.1101\/2025.03.21.644583"},{"key":"2026070211070675900_btag375-B34","author":"Wolf","year":"2019"},{"key":"2026070211070675900_btag375-B35","author":"Yosinski","year":"2014"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btag375\/68520347\/btag375.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/42\/6\/btag375\/68520347\/btag375.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/42\/6\/btag375\/68520347\/btag375.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T16:44:08Z","timestamp":1783010648000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btag375\/8707331"}},"subtitle":[],"editor":[{"given":"Pier Luigi","family":"Martelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,6,1]]},"references-count":35,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2026,6,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btag375","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,6]]},"published":{"date-parts":[[2026,6,1]]},"article-number":"btag375"}}