{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2022,4,2]],"date-time":"2022-04-02T12:55:12Z","timestamp":1648904112627},"reference-count":22,"publisher":"Springer Science and Business Media LLC","issue":"S16","license":[{"start":{"date-parts":[[2019,12,1]],"date-time":"2019-12-01T00:00:00Z","timestamp":1575158400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,12,2]],"date-time":"2019-12-02T00:00:00Z","timestamp":1575244800000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n<jats:title>Background<\/jats:title>\n<jats:p>Supercomputers have become indispensable infrastructures in science and industries. In particular, most state-of-the-art scientific results utilize massively parallel supercomputers ranked in TOP500. However, their use is still limited in the bioinformatics field due to the fundamental fact that the asynchronous parallel processing service of Grid Engine is not provided on them. To encourage the use of massively parallel supercomputers in bioinformatics, we developed middleware called <jats:italic>Virtual Grid Engine<\/jats:italic>, which enables software pipelines to automatically perform their tasks as MPI programs.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Result<\/jats:title>\n<jats:p>We conducted basic tests to check the time required to assign jobs to workers by VGE. The results showed that the overhead of the employed algorithm was 246 microseconds and our software can manage thousands of jobs smoothly on the K computer. We also tried a practical test in the bioinformatics field. This test included two tasks, the split and BWA alignment of input FASTQ data. 25,055 nodes (2,000,440 cores) were used for this calculation and accomplished it in three hours.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Conclusion<\/jats:title>\n<jats:p>We considered that there were four important requirements for this kind of software, non-privilege server program, multiple job handling, dependency control, and usability. We carefully designed and checked all requirements. And this software fulfilled all the requirements and achieved good performance in a large scale analysis.<\/jats:p>\n<\/jats:sec>","DOI":"10.1186\/s12859-019-3085-x","type":"journal-article","created":{"date-parts":[[2019,12,2]],"date-time":"2019-12-02T12:00:35Z","timestamp":1575288035000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Virtual Grid Engine: a simulated grid engine environment for large-scale supercomputers"],"prefix":"10.1186","volume":"20","author":[{"given":"Satoshi","family":"Ito","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Masaaki","family":"Yadome","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tatsuo","family":"Nishiki","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shigeru","family":"Ishiduki","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hikaru","family":"Inoue","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rui","family":"Yamaguchi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Satoru","family":"Miyano","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2019,12,2]]},"reference":[{"key":"3085_CR1","unstructured":"The Cost of Sequencing a Human Genome. 2018. https:\/\/www.genome.gov\/sequencingcosts\/. Accessed 25 Aug 2018."},{"issue":"4","key":"3085_CR2","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1073\/pnas.1322563111","volume":"111","author":"O Zuk","year":"2014","unstructured":"Zuk O, Schaffner SF, Samocha K, Do R, Hechter E, Kathiresan S, Daly MJ, Neale BM, Sunyaev SR, Lander ES. Searching for missing heritability: Designing rare variant association studies. Proc Nat Acad Sci. 2014; 111(4):455\u201364. https:\/\/doi.org\/10.1073\/pnas.1322563111. http:\/\/arxiv.org\/abs\/https:\/\/www.pnas.org\/content\/111\/4\/E455.full.pdf.","journal-title":"Proc Nat Acad Sci"},{"key":"3085_CR3","unstructured":"TOP, 500 Project. 2018. https:\/\/www.top500.org\/.. Accessed 25 Aug 2018."},{"issue":"11","key":"3085_CR4","doi-asserted-by":"publisher","first-page":"1508","DOI":"10.1093\/bioinformatics\/btu071","volume":"30","author":"EM McNally","year":"2014","unstructured":"McNally EM, Dorn I, Gerald W, Foster IT, Golbus JR, Dellefave-Castillo L, Pesce LL, Puckelwartz MJ, Day SM, Cappola TP, Nelakuditi V. Supercomputing for the parallelization of whole genome analysis. Bioinformatics. 2014; 30(11):1508\u201313. doi:10.1093\/bioinformatics\/btu071. http:\/\/oup.prod.sis.lan\/bioinformatics\/article-pdf\/30\/11\/1508\/794534\/btu071.pdf.","journal-title":"Bioinformatics"},{"key":"3085_CR5","doi-asserted-by":"publisher","unstructured":"Ito S, Shiraishi Y, Shimamura T, Chiba K, Miyano S. High performance computing of a fusion gene detection pipeline on the k computer. In: 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM): 2015. p. 1441\u20137. https:\/\/doi.org\/10.1109\/BIBM.2015.7359888.","DOI":"10.1109\/BIBM.2015.7359888"},{"issue":"1","key":"3085_CR6","doi-asserted-by":"publisher","first-page":"40","DOI":"10.11234\/gi.25.40","volume":"25","author":"Y TAMADA","year":"2011","unstructured":"TAMADA Y, SHIMAMURA T, YAMAGUCHI R, IMOTO S, NAGASAKI M, MIYANO S. Sign: Large-scale gene network estimation environment for high performance computing. Genome Informa. 2011; 25(1):40\u201352. https:\/\/doi.org\/10.11234\/gi.25.40.","journal-title":"Genome Informa"},{"key":"3085_CR7","unstructured":"Message Passing Interface Forum. 2018. https:\/\/www.mpi-forum.org\/.. Accessed 25 Aug 2018."},{"key":"3085_CR8","doi-asserted-by":"publisher","DOI":"10.5772\/66732","volume-title":"Cloud Computing - Architecture and Applications","author":"Shanrong Zhao","year":"2017","unstructured":"Zhao S, Watrous K, Zhang C, Zhang B. Cloud computing for next-generation sequencing data analysis In: Sen J, editor. Cloud Computing, Chap. 2. Rijeka: IntechOpen: 2017. https:\/\/doi.org\/10.5772\/66732."},{"issue":"6","key":"3085_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0129277","volume":"10","author":"SS Shringarpure","year":"2015","unstructured":"Shringarpure SS, Carroll A, De La Vega FM, Bustamante CD. Inexpensive and highly reproducible cloud-based variant calling of 2535 human genomes. PLOS ONE. 2015; 10(6):1\u201310. https:\/\/doi.org\/10.1371\/journal.pone.0129277.","journal-title":"PLOS ONE"},{"key":"3085_CR10","doi-asserted-by":"publisher","unstructured":"Miller N, G. Farrow E, Gibson M, K. Willig L, Twist G, Yoo B, Marrs T, Corder S, Krivohlavek L, Walter A, Petrikin J, Saunders C, Thiffault I, Soden S, Smith L, Dinwiddie D, Herd S, Cakici J, Catreux S, Kingsmore S. A 26-hour system of highly sensitive whole genome sequencing for emergency management of genetic diseases. Genome Med. 2015; 7. https:\/\/doi.org\/10.1186\/s13073-015-0221-8.","DOI":"10.1186\/s13073-015-0221-8"},{"key":"3085_CR11","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/j.theriogenology.2017.08.027","volume":"105","author":"Y Jia","year":"2018","unstructured":"Jia Y, Nan P, Zhang W, Wang F, Zhang R, Liang T, Ji X, Du Q, Chang Z. Transcriptome analysis of three critical periods of ovarian development in yellow river carp (cyprinus carpio). Theriogenology. 2018; 105:15\u201326. https:\/\/doi.org\/10.1016\/j.theriogenology.2017.08.027.","journal-title":"Theriogenology"},{"key":"3085_CR12","doi-asserted-by":"publisher","DOI":"10.1145\/1188455.1188464","volume-title":"Proceedings of the 2006 ACM\/IEEE Conference on Supercomputing. SC \u201906","author":"G Staples","year":"2006","unstructured":"Staples G. Torque resource manager. In: Proceedings of the 2006 ACM\/IEEE Conference on Supercomputing. SC \u201906. New York: ACM: 2006. https:\/\/doi.org\/10.1145\/1188455.1188464."},{"key":"3085_CR13","doi-asserted-by":"publisher","unstructured":"Tange O. GNU Parallel 20150322 (\u2019Hellwig\u2019). GNU Parallel is a general parallelizer to run multiple serial command line programs in parallel without changing them. 2015. https:\/\/doi.org\/10.5281\/zenodo.16303.","DOI":"10.5281\/zenodo.16303"},{"key":"3085_CR14","doi-asserted-by":"publisher","first-page":"1124","DOI":"10.1016\/j.advwatres.2011.04.013","volume":"34","author":"LD Dalcin","year":"2011","unstructured":"Dalcin LD, Paz RR, Kler PA, Cosimo A. Parallel distributed computing using Python. Adv Water Res. 2011; 34:1124\u201339. https:\/\/doi.org\/10.1016\/j.advwatres.2011.04.013.","journal-title":"Adv Water Res"},{"issue":"5","key":"3085_CR15","doi-asserted-by":"publisher","first-page":"655","DOI":"10.1016\/j.jpdc.2007.09.005","volume":"68","author":"Lisandro Dalc\u00edn","year":"2008","unstructured":"Dalcin L, Paz R, Storti M, D\u00e3El\u00c3a J. Mpi for python: Performance improvements and mpi-2 extensions. J Paral Distrib Comput. 2008; 68(5):655\u201362. URL https:\/\/doi.org\/10.1016\/j.jpdc.2007.09.005.","journal-title":"Journal of Parallel and Distributed Computing"},{"issue":"9","key":"3085_CR16","doi-asserted-by":"publisher","first-page":"1108","DOI":"10.1016\/j.jpdc.2005.03.010","volume":"65","author":"L Dalcin","year":"2005","unstructured":"Dalcin L, Paz R, Storti M. Mpi for python. J Paral Distrib Comput. 2005; 65(9):1108\u201315. https:\/\/doi.org\/10.1016\/j.jpdc.2005.03.010.","journal-title":"J Paral Distrib Comput"},{"key":"3085_CR17","unstructured":"K Computer. 2018. https:\/\/www.r-ccs.riken.jp\/en\/k-computer\/system.. Accessed 25 Aug 2018."},{"issue":"14","key":"3085_CR18","doi-asserted-by":"publisher","first-page":"1754","DOI":"10.1093\/bioinformatics\/btp324","volume":"25","author":"H Li","year":"2009","unstructured":"Li H, Durbin R. Fast and accurate short read alignment with Burrows Wheeler transform. Bioinformatics. 2009; 25(14):1754\u201360. doi:10.1093\/bioinformatics\/btp324. http:\/\/oup.prod.sis.lan\/bioinformatics\/article-pdf\/25\/14\/1754\/605544\/btp324.pdf.","journal-title":"Bioinformatics"},{"key":"3085_CR19","unstructured":"FASTQ Format Specification. 2018. http:\/\/maq.sourceforge.net\/fastq.shtml.. Accessed 25 Aug 2018."},{"issue":"1","key":"3085_CR20","first-page":"302","volume":"48","author":"K Sakai","year":"2012","unstructured":"Sakai K, Sumimoto S, Kurokawa M. High-performance and highly reliable file system for the k computer. Fujitsu Sci Tech J. 2012; 48(1):302\u20139.","journal-title":"Fujitsu Sci Tech J"},{"key":"3085_CR21","unstructured":"The Lustre Filesystem. 2019. http:\/\/lustre.org\/.. Accessed 20 Mar 2019."},{"issue":"2","key":"3085_CR22","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1145\/3152891","volume":"51","author":"FZ Boito","year":"2018","unstructured":"Boito FZ, Inacio EC, Bez JL, Navaux POA, Dantas MAR, Denneulin Y. A checkpoint of research on parallel i\/o for high-performance computing. ACM Comput Surv. 2018; 51(2):23\u201312335. https:\/\/doi.org\/10.1145\/3152891.","journal-title":"ACM Comput Surv"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-019-3085-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12859-019-3085-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-019-3085-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,12,1]],"date-time":"2020-12-01T00:20:17Z","timestamp":1606782017000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-019-3085-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12]]},"references-count":22,"journal-issue":{"issue":"S16","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["3085"],"URL":"https:\/\/doi.org\/10.1186\/s12859-019-3085-x","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12]]},"assertion":[{"value":"2 December 2019","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"591"}}