{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T05:33:22Z","timestamp":1769751202846,"version":"3.49.0"},"reference-count":84,"publisher":"Springer Science and Business Media LLC","issue":"S2","license":[{"start":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T00:00:00Z","timestamp":1670803200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T00:00:00Z","timestamp":1670803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100012352","name":"Universit\u00e0 degli Studi di Milano","doi-asserted-by":"crossref","award":["PSR 2020 project \u201cMachine Learning and Data Analytics for Genomic Medicine\u201d"],"award-info":[{"award-number":["PSR 2020 project \u201cMachine Learning and Data Analytics for Genomic Medicine\u201d"]}],"id":[{"id":"10.13039\/100012352","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100002839","name":"Charit\u00e9 - Universit\u00e4tsmedizin Berlin","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002839","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Cis-regulatory regions (CRRs) are non-coding regions of the DNA that fine control the spatio-temporal pattern of transcription; they are involved in a wide range of pivotal processes such as the development of specific cell-lines\/tissues and the dynamic cell response to physiological stimuli. Recent studies showed that genetic variants occurring in CRRs are strongly correlated with pathogenicity or deleteriousness. Considering the central role of CRRs in the regulation of physiological and pathological conditions, the correct identification of CRRs and of their tissue-specific activity status through Machine Learning methods plays a major role in dissecting the impact of genetic variants on human diseases. Unfortunately, the problem is still open, though some promising results have been already reported by (deep) machine-learning based methods that predict active promoters and enhancers in specific tissues or cell lines by encoding epigenetic or spectral features directly extracted from DNA sequences.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>We present the experiments we performed to compare two Deep Neural Networks, a Feed-Forward Neural Network model working on epigenomic features, and a Convolutional Neural Network model working only on genomic sequence, targeted to the identification of enhancer- and promoter-activity in specific cell lines. While performing experiments to understand how the experimental setup influences the prediction performance of the methods, we particularly focused on (1) automatic model selection performed by Bayesian optimization and (2) exploring different data rebalancing setups for reducing negative unbalancing effects.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>Results show that (1) automatic model selection by Bayesian optimization improves the quality of the learner; (2) data rebalancing considerably impacts the prediction performance of the models; test set rebalancing may provide over-optimistic results, and should therefore be cautiously applied; (3) despite working on sequence data, convolutional models obtain performance close to those of feed forward models working on epigenomic information, which suggests that also sequence data carries informative content for CRR-activity prediction. We therefore suggest combining both models\/data types in future works.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12859-022-04582-5","type":"journal-article","created":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T07:05:54Z","timestamp":1670828754000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Boosting tissue-specific prediction of active cis-regulatory regions through deep learning and Bayesian optimization techniques"],"prefix":"10.1186","volume":"23","author":[{"given":"Luca","family":"Cappelletti","sequence":"first","affiliation":[]},{"given":"Alessandro","family":"Petrini","sequence":"additional","affiliation":[]},{"given":"Jessica","family":"Gliozzo","sequence":"additional","affiliation":[]},{"given":"Elena","family":"Casiraghi","sequence":"additional","affiliation":[]},{"given":"Max","family":"Schubach","sequence":"additional","affiliation":[]},{"given":"Martin","family":"Kircher","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5694-3919","authenticated-orcid":false,"given":"Giorgio","family":"Valentini","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,12,12]]},"reference":[{"issue":"6","key":"4582_CR1","first-page":"980","volume":"17","author":"A Mora","year":"2016","unstructured":"Mora A, Sandve GK, Gabrielsen OS, Eskeland R. In the loop: promoter\u2013enhancer interactions and bioinformatics. Brief Bioinform. 2016;17(6):980\u201395.","journal-title":"Brief Bioinform"},{"issue":"1","key":"4582_CR2","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1016\/j.cell.2008.06.030","volume":"134","author":"SB Carroll","year":"2008","unstructured":"Carroll SB. Evo-devo and an expanding evolutionary synthesis: a genetic theory of morphological evolution. Cell. 2008;134(1):25\u201336.","journal-title":"Cell"},{"issue":"4","key":"4582_CR3","doi-asserted-by":"publisher","first-page":"650","DOI":"10.1016\/j.cell.2018.01.029","volume":"172","author":"SA Lambert","year":"2018","unstructured":"Lambert SA, Jolma A, Campitelli LF, Das PK, Yin Y, Albu M, Chen X, Taipale J, Hughes TR, Weirauch MT. The human transcription factors. Cell. 2018;172(4):650\u201365.","journal-title":"Cell"},{"issue":"10","key":"4582_CR4","doi-asserted-by":"publisher","first-page":"1910","DOI":"10.1016\/j.bbadis.2014.03.011","volume":"1842","author":"B Hrdlickova","year":"2014","unstructured":"Hrdlickova B, de Almeida RC, Borek Z, Withoff S. Genetic variation in the non-coding genome: involvement of micro-RNAs and long non-coding RNAs in disease. Biochim et Biophys Acta (BBA)-Mol Basis Dis. 2014;1842(10):1910\u201322.","journal-title":"Biochim et Biophys Acta (BBA)-Mol Basis Dis"},{"issue":"39","key":"4582_CR5","doi-asserted-by":"publisher","first-page":"5003","DOI":"10.1038\/onc.2014.456","volume":"34","author":"H Ling","year":"2015","unstructured":"Ling H, Vincent K, Pichler M, Fodde R, Berindan-Neagoe I, Slack FJ, Calin GA. Junk DNA and the long non-coding RNA twist in cancer genetics. Oncogene. 2015;34(39):5003\u201311.","journal-title":"Oncogene"},{"key":"4582_CR6","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1038\/s41598-017-03011-5","volume":"7","author":"M Schubach","year":"2017","unstructured":"Schubach M, Re M, Robinson PN, Valentini G. Imbalance-aware machine learning for predicting rare and common disease-associated non-coding variants. Sci Rep. 2017;7:29\u201359. https:\/\/doi.org\/10.1038\/s41598-017-03011-5.","journal-title":"Sci Rep"},{"issue":"D1","key":"4582_CR7","doi-asserted-by":"publisher","first-page":"886","DOI":"10.1093\/nar\/gky1016","volume":"47","author":"P Rentzsch","year":"2019","unstructured":"Rentzsch P, Witten D, Cooper GM, Shendure J, Kircher M. CADD: predicting the deleteriousness of variants throughout the human genome. Nucleic Acids Res. 2019;47(D1):886\u201394. https:\/\/doi.org\/10.1093\/nar\/gky1016.","journal-title":"Nucleic Acids Res"},{"key":"4582_CR8","doi-asserted-by":"publisher","DOI":"10.1093\/gigascience\/giaa052","author":"A Petrini","year":"2020","unstructured":"Petrini A, Mesiti M, Schubach M, Frasca M, Danis D, Re M, Grossi G, Cappelletti L, Castrignan\u00f2 T, Robinson PN, Valentini G. parSMURF, a high-performance computing tool for the genome-wide detection of pathogenic variants. GigaScience. 2020. https:\/\/doi.org\/10.1093\/gigascience\/giaa052.","journal-title":"GigaScience"},{"issue":"6099","key":"4582_CR9","doi-asserted-by":"publisher","first-page":"1190","DOI":"10.1126\/science.1222794","volume":"337","author":"MT Maurano","year":"2012","unstructured":"Maurano MT, Humbert R, Rynes E, Thurman RE, Haugen E, Wang H, Reynolds AP, Sandstrom R, Qu H, Brody J, et al. Systematic localization of common disease-associated variation in regulatory DNA. Science. 2012;337(6099):1190\u20135.","journal-title":"Science"},{"issue":"5","key":"4582_CR10","doi-asserted-by":"publisher","first-page":"724","DOI":"10.1158\/2159-8290.CD-19-1128","volume":"10","author":"K Li","year":"2020","unstructured":"Li K, Zhang Y, Liu X, Liu Y, Gu Z, Cao H, Dickerson KE, Chen M, Chen W, Shao Z, Ni M, Xu J. Noncoding variants connect enhancer dysregulation with nuclear receptor signaling in hematopoietic malignancies. Cancer Discov. 2020;10(5):724\u201345. https:\/\/doi.org\/10.1158\/2159-8290.CD-19-1128.","journal-title":"Cancer Discov"},{"issue":"6","key":"4582_CR11","doi-asserted-by":"publisher","first-page":"874","DOI":"10.1016\/j.ajhg.2018.10.018","volume":"103","author":"SA McClymont","year":"2018","unstructured":"McClymont SA, Hook PW, Soto AI, Reed X, Law WD, Kerans SJ, Waite EL, Briceno NJ, Thole JF, Heckman MG, Diehl NN, Wszolek ZK, Moore CD, Zhu H, Akiyama JA, Dickel DE, Visel A, Pennacchio LA, Ross OA, Beer MA, McCallion AS. Parkinson-associated SNCA enhancer variants revealed by open chromatin in mouse dopamine neurons. Am J Hum Genet. 2018;103(6):874\u201392. https:\/\/doi.org\/10.1016\/j.ajhg.2018.10.018.","journal-title":"Am J Hum Genet"},{"issue":"10","key":"4582_CR12","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1186\/s13073-014-0085-3","volume":"6","author":"O Corradin","year":"2014","unstructured":"Corradin O, Scacheri PC. Enhancer variants: evaluating functions in common disease. Genome Med. 2014;6(10):85\u201385. https:\/\/doi.org\/10.1186\/s13073-014-0085-3.","journal-title":"Genome Med"},{"issue":"10","key":"4582_CR13","doi-asserted-by":"publisher","first-page":"1007436","DOI":"10.1371\/journal.pcbi.1007436","volume":"15","author":"T Gao","year":"2019","unstructured":"Gao T, Qian J. Eagle: an algorithm that utilizes a small number of genomic features to predict tissue\/cell type-specific enhancer-gene interactions. PLoS Comput Biol. 2019;15(10):1007436\u20131007436. https:\/\/doi.org\/10.1371\/journal.pcbi.1007436.","journal-title":"PLoS Comput Biol"},{"issue":"4","key":"4582_CR14","doi-asserted-by":"publisher","first-page":"934","DOI":"10.1016\/j.cell.2013.09.053","volume":"155","author":"D Hnisz","year":"2013","unstructured":"Hnisz D, Abraham BJ, Lee TI, Lau A, Saint-Andr\u00e9 V, Sigova AA, Hoke HA, Young RA. Super-enhancers in the control of cell identity and disease. Cell. 2013;155(4):934\u201347. https:\/\/doi.org\/10.1016\/j.cell.2013.09.053.","journal-title":"Cell"},{"issue":"R2","key":"4582_CR15","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1093\/hmg\/ddw216","volume":"25","author":"CJ Coppola","year":"2016","unstructured":"Coppola CJ, Ramaker CR, Mendenhall EM. Identification and function of enhancers in the human genome. Hum Mol Genet. 2016;25(R2):190\u20137. https:\/\/doi.org\/10.1093\/hmg\/ddw216.","journal-title":"Hum Mol Genet"},{"issue":"1","key":"4582_CR16","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1186\/s13059-017-1345-5","volume":"18","author":"Y Liu","year":"2017","unstructured":"Liu Y, Yu S, Dhiman VK, Brunetti T, Eckart H, White KP. Functional assessment of human enhancer activities using whole-genome starr-sequencing. Genome Biol. 2017;18(1):219. https:\/\/doi.org\/10.1186\/s13059-017-1345-5.","journal-title":"Genome Biol"},{"issue":"5","key":"4582_CR17","doi-asserted-by":"publisher","first-page":"1369","DOI":"10.1016\/j.cell.2016.09.037","volume":"167","author":"BM Javierre","year":"2016","unstructured":"Javierre BM, Burren OS, Wilder SP, Kreuzhuber R, Hill SM, Sewitz S, Cairns J, Wingett SW, Varnai C, Thiecke MJ, et al. Lineage-specific genome architecture links enhancers and non-coding disease variants to target gene promoters. Cell. 2016;167(5):1369\u201384.","journal-title":"Cell"},{"issue":"10","key":"4582_CR18","doi-asserted-by":"publisher","first-page":"1045","DOI":"10.1038\/nbt1010-1045","volume":"28","author":"BE Bernstein","year":"2010","unstructured":"Bernstein BE, Stamatoyannopoulos JA, Costello JF, Ren B, Milosavljevic A, Meissner A, Kellis M, Marra MA, Beaudet AL, Ecker JR, et al. The NIH roadmap epigenomics mapping consortium. Nat Biotechnol. 2010;28(10):1045.","journal-title":"Nat Biotechnol"},{"issue":"7414","key":"4582_CR19","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1038\/nature11247","volume":"489","author":"TEP Consortium","year":"2012","unstructured":"Consortium TEP. An integrated encyclopedia of DNA elements in the human genome. Nature. 2012;489(7414):57\u201374. https:\/\/doi.org\/10.1038\/nature11247.","journal-title":"Nature"},{"issue":"7409","key":"4582_CR20","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1038\/nature11243","volume":"488","author":"Y Shen","year":"2012","unstructured":"Shen Y, Yue F, McCleary DF, Ye Z, Edsall L, Kuan S, Wagner U, Dixon J, Lee L, Lobanenkov VV, Ren B. A map of the cis-regulatory sequences in the mouse genome. Nature. 2012;488(7409):116\u201320. https:\/\/doi.org\/10.1038\/nature11243.","journal-title":"Nature"},{"issue":"3","key":"4582_CR21","doi-asserted-by":"publisher","first-page":"642","DOI":"10.1016\/j.cell.2012.12.033","volume":"152","author":"J Zhu","year":"2013","unstructured":"Zhu J, Adli M, Zou JY, Verstappen G, Coyne M, Zhang X, Durham T, Miri M, Deshpande V, Jager PLD, Bennett DA, Houmard JA, Muoio DM, Onder TT, Camahort R, Cowan CA, Meissner A, Epstein CB, Shoresh N, Bernstein BE. Genome-wide chromatin state transitions associated with developmental and environmental cues. Cell. 2013;152(3):642\u201354. https:\/\/doi.org\/10.1016\/j.cell.2012.12.033.","journal-title":"Cell"},{"issue":"1","key":"4582_CR22","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1186\/s13059-014-0560-6","volume":"16","author":"M Lizio","year":"2015","unstructured":"Lizio M, Harshbarger J, Shimoji H, Severin J, Kasukawa T, Sahin S, Abugessaisa I, Fukuda S, Hori F, Ishikawa-Kato S, et al. Gateways to the fantom5 promoter level mammalian expression atlas. Genome Biol. 2015;16(1):22.","journal-title":"Genome Biol"},{"issue":"7539","key":"4582_CR23","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1038\/nature14248","volume":"518","author":"A Kundaje","year":"2015","unstructured":"Kundaje A, Meuleman W, Ernst J, Bilenky M, Yen A, Heravi-Moussavi A, Kheradpour P, Zhang Z, Wang J, Ziller MJ, et al. Integrative analysis of 111 reference human epigenomes. Nature. 2015;518(7539):317.","journal-title":"Nature"},{"key":"4582_CR24","doi-asserted-by":"publisher","first-page":"196299","DOI":"10.1109\/ACCESS.2020.3034032","volume":"8","author":"E Casiraghi","year":"2020","unstructured":"Casiraghi E, Malchiodi D, Trucco G, Frasca M, Cappelletti L, Fontana T, Esposito AA, Avola E, Jachetti A, Reese J, et al. Explainable machine learning for early assessment of covid-19 risk prediction in emergency departments. IEEE Access. 2020;8:196299\u2013325.","journal-title":"IEEE Access"},{"issue":"3","key":"4582_CR25","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1038\/nmeth.1906","volume":"9","author":"J Ernst","year":"2012","unstructured":"Ernst J, Kellis M. Chromhmm: automating chromatin-state discovery and characterization. Nat Methods. 2012;9(3):215\u20136. https:\/\/doi.org\/10.1038\/nmeth.1906.","journal-title":"Nat Methods"},{"issue":"5","key":"4582_CR26","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1038\/nmeth.1937","volume":"9","author":"MM Hoffman","year":"2012","unstructured":"Hoffman MM, Buske OJ, Wang J, Weng Z, Bilmes JA, Noble WS. Unsupervised pattern discovery in human chromatin structure through genomic segmentation. Nat Methods. 2012;9(5):473\u20136. https:\/\/doi.org\/10.1038\/nmeth.1937.","journal-title":"Nat Methods"},{"issue":"10","key":"4582_CR27","doi-asserted-by":"publisher","first-page":"1595","DOI":"10.1101\/gr.173518.114","volume":"24","author":"JC Kwasnieski","year":"2014","unstructured":"Kwasnieski JC, Fiore C, Chaudhari HG, Cohen BA. High-throughput functional testing of encode segmentation predictions. Genome Res. 2014;24(10):1595\u2013602.","journal-title":"Genome Res"},{"issue":"6","key":"4582_CR28","doi-asserted-by":"publisher","first-page":"0130622","DOI":"10.1371\/journal.pone.0130622","volume":"10","author":"Y Lu","year":"2015","unstructured":"Lu Y, Qu W, Shan G, Zhang C. Delta: a distal enhancer locating tool based on adaboost algorithm and shape features of chromatin modifications. PLoS ONE. 2015;10(6):0130622\u20130130622. https:\/\/doi.org\/10.1371\/journal.pone.0130622.","journal-title":"PLoS ONE"},{"issue":"9","key":"4582_CR29","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1186\/gb-2012-13-9-r48","volume":"13","author":"KY Yip","year":"2012","unstructured":"Yip KY, Cheng C, Bhardwaj N, Brown JB, Leng J, Kundaje A, Rozowsky J, Birney E, Bickel P, Snyder M, Gerstein M. Classification of human genomic regions based on experimentally determined binding sites of more than 100 transcription-related factors. Genome Biol. 2012;13(9):48. https:\/\/doi.org\/10.1186\/gb-2012-13-9-r48.","journal-title":"Genome Biol"},{"issue":"7493","key":"4582_CR30","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1038\/nature12787","volume":"507","author":"R Andersson","year":"2014","unstructured":"Andersson R, Gebhard C, Miguel I, Hoof I, Bornholdt J, Boyd M, Chen Y, Zhao X, Schmidl C, Suzuki T, Ntini E, Arner E, Valen E, Li K, Schwarzfischer L, Glatz D, Raithel J, Lilje B, Rapin N, Bagger FO, J\u00f8rgensen M, Andersen PR, Bertin N, Rackham O, Burroughs AM, Baillie JK, Ishizu Y, Shimizu Y, Furuhata E, Maeda S, Negishi Y, Mungall CJ, Meehan TF, Lassmann T, Itoh M, Kawaji H, Kondo N, Kawai J, Lennartsson A, Daub CO, Heutink P, Hume DA, Jensen TH, Suzuki H, Hayashizaki Y, M\u00fcller F, Forrest ARR, Carninci P, Rehli M, Sandelin A, Consortium TF. An atlas of active enhancers across human cell types and tissues. Nature. 2014;507(7493):455\u201361. https:\/\/doi.org\/10.1038\/nature12787.","journal-title":"Nature"},{"issue":"1","key":"4582_CR31","doi-asserted-by":"publisher","first-page":"170112","DOI":"10.1038\/sdata.2017.112","volume":"4","author":"S Noguchi","year":"2017","unstructured":"Noguchi S, Arakawa T, Fukuda S, Furuno M, Hasegawa A, Hori F, Ishikawa-Kato S, Kaida K, Kaiho A, Kanamori-Katayama M, Kawashima T, Kojima M, Kubosaki A, Manabe R-I, Murata M, Nagao-Sato S, Nakazato K, Ninomiya N, Nishiyori-Sueki H, Noma S, Saijyo E, Saka A, Sakai M, et al. Fantom5 cage profiles of human and mouse samples. Sci Data. 2017;4(1):170112. https:\/\/doi.org\/10.1038\/sdata.2017.112.","journal-title":"Sci Data"},{"issue":"1","key":"4582_CR32","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1093\/nar\/gku1058","volume":"43","author":"D Kleftogiannis","year":"2015","unstructured":"Kleftogiannis D, Kalnis P, Bajic VB. Deep: a general computational framework for predicting enhancers. Nucleic Acids Res. 2015;43(1):6\u20136. https:\/\/doi.org\/10.1093\/nar\/gku1058.","journal-title":"Nucleic Acids Res"},{"issue":"13","key":"4582_CR33","doi-asserted-by":"publisher","first-page":"478","DOI":"10.1186\/s12859-017-1878-3","volume":"18","author":"X Min","year":"2017","unstructured":"Min X, Zeng W, Chen S, Chen N, Chen T, Jiang R. Predicting enhancers with deep convolutional neural networks. BMC Bioinform. 2017;18(13):478. https:\/\/doi.org\/10.1186\/s12859-017-1878-3.","journal-title":"BMC Bioinform"},{"issue":"1","key":"4582_CR34","doi-asserted-by":"publisher","first-page":"202","DOI":"10.1186\/s12859-018-2187-1","volume":"19","author":"Y Li","year":"2018","unstructured":"Li Y, Shi W, Wasserman WW. Genome-wide prediction of cis-regulatory regions using supervised deep learning methods. BMC Bioinform. 2018;19(1):202. https:\/\/doi.org\/10.1186\/s12859-018-2187-1.","journal-title":"BMC Bioinform"},{"key":"4582_CR35","doi-asserted-by":"publisher","unstructured":"Cappelletti L, Petrini A, Gliozzo J, Casiraghi E, Schubach M, Kircher M, Valentini G. Bayesian optimization improves tissue-specific prediction of active regulatory regions with deep neural networks. In: Springer (ed.) Bioinformatics and biomedical engineering, IWBBIO 2020. Lecture notes in computer science; 2020. https:\/\/doi.org\/10.1007\/978-3-030-45385-5_54","DOI":"10.1007\/978-3-030-45385-5_54"},{"key":"4582_CR36","unstructured":"Goodfellow I, Bengio Y, Courville A. Deep learning. MIT Press; 2016. http:\/\/www.deeplearningbook.org"},{"issue":"53","key":"4582_CR37","first-page":"1","volume":"20","author":"P Probst","year":"2019","unstructured":"Probst P, Boulesteix A-L, Bischl B. Tunability: importance of hyperparameters of machine learning algorithms. J Mach Learn Res. 2019;20(53):1\u201332.","journal-title":"J Mach Learn Res"},{"key":"4582_CR38","unstructured":"Swersky K, Snoek J, Adams RP. Multi-task bayesian optimization. In: Burges CJC, Bottou L, Welling M, Ghahramani Z, Weinberger KQ, editors. Advances in neural information processing systems 26. Lake Tahoe, Nevada, USA: Curran Associates Inc; 2013. p. 2004\u201312."},{"key":"4582_CR39","doi-asserted-by":"publisher","first-page":"272","DOI":"10.7551\/mitpress\/3206.001.0001","volume-title":"Gaussian Processes for machine learning (adaptive computation and machine learning)","author":"CE Rasmussen","year":"2005","unstructured":"Rasmussen CE, Williams CKI. Gaussian Processes for machine learning (adaptive computation and machine learning). Cambridge: The MIT Press; 2005. p. 272."},{"key":"4582_CR40","unstructured":"Malkomes G, Schaff C, Garnett R. Bayesian optimization for automated model selection. In: Lee D, Sugiyama M, Luxburg U, Guyon I, Garnett R (eds) Advances in neural information processing systems, vol 29. Barcelona, Spain: Curran Associates Inc; 2016."},{"issue":"2","key":"4582_CR41","doi-asserted-by":"publisher","first-page":"358","DOI":"10.1016\/j.cell.2014.08.027","volume":"159","author":"Z Liu","year":"2014","unstructured":"Liu Z, Merkurjev D, Yang F, Li W, Oh S, Friedman MJ, Song X, Zhang F, Ma Q, Ohgi KA, et al. Enhancer activation requires trans-recruitment of a mega transcription factor complex. Cell. 2014;159(2):358\u201373.","journal-title":"Cell"},{"issue":"9\u201310","key":"4582_CR42","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1007\/s00335-015-9593-8","volume":"26","author":"M de Hoon","year":"2015","unstructured":"de Hoon M, Shin JW, Carninci P. Paradigm shifts in genomics through the fantom projects. Mammalian Genome. 2015;26(9\u201310):391\u2013402.","journal-title":"Mammalian Genome"},{"issue":"1","key":"4582_CR43","doi-asserted-by":"publisher","first-page":"28517","DOI":"10.1038\/srep28517","volume":"6","author":"F Liu","year":"2016","unstructured":"Liu F, Li H, Ren C, Bo X, Shu W. Pedla: predicting enhancers with a deep learning-based algorithmic framework. Sci Rep. 2016;6(1):28517. https:\/\/doi.org\/10.1038\/srep28517.","journal-title":"Sci Rep"},{"issue":"13","key":"4582_CR44","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1093\/bioinformatics\/btx105","volume":"33","author":"B Yang","year":"2017","unstructured":"Yang B, Liu F, Ren C, Ouyang Z, Xie Z, Bo X, Shu W. BiRen: predicting enhancers with a deep-learning-based model using the DNA sequence alone. Bioinform. 2017;33(13):1930\u20136. https:\/\/doi.org\/10.1093\/bioinformatics\/btx105.","journal-title":"Bioinform"},{"issue":"8","key":"4582_CR45","doi-asserted-by":"publisher","first-page":"1034","DOI":"10.1101\/gr.3715005","volume":"15","author":"A Siepel","year":"2005","unstructured":"Siepel A, Bejerano G, Pedersen JS, Hinrichs AS, Hou M, Rosenbloom K, Clawson H, Spieth J, Hillier LW, Richards S, et al. Evolutionarily conserved elements in vertebrate, insect, worm, and yeast genomes. Genome Res. 2005;15(8):1034\u201350.","journal-title":"Genome Res"},{"key":"4582_CR46","unstructured":"Pasini ML, Yin J, Li YW, Eisenbach M. A greedy constructive algorithm for the optimization of neural network architectures. CoRR abs\/1909.03306 2019. arXiv:1909.03306."},{"key":"4582_CR47","doi-asserted-by":"crossref","unstructured":"Talathi SS. Hyper-parameter optimization of deep convolutional networks for object recognition. In: 2015 IEEE international conference on image processing (ICIP), pp. 3982\u20133986; 2015. IEEE","DOI":"10.1109\/ICIP.2015.7351553"},{"issue":"1","key":"4582_CR48","first-page":"281","volume":"13","author":"J Bergstra","year":"2012","unstructured":"Bergstra J, Bengio Y. Random search for hyper-parameter optimization. J Mach Learn Res. 2012;13(1):281\u2013305.","journal-title":"J Mach Learn Res"},{"key":"4582_CR49","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1016\/j.neucom.2015.12.061","volume":"186","author":"FJ Pontes","year":"2016","unstructured":"Pontes FJ, Amorim G, Balestrassi PP, Paiva A, Ferreira JR. Design of experiments and focused grid search for neural network parameter optimization. Neurocomputing. 2016;186:22\u201334.","journal-title":"Neurocomputing"},{"key":"4582_CR50","doi-asserted-by":"crossref","unstructured":"Sinha T, Haidar A, Verma B. Particle swarm optimization based approach for finding optimal values of convolutional neural network parameters. In: 2018 IEEE congress on evolutionary computation (CEC), pp. 1\u20136; 2018. IEEE.","DOI":"10.1109\/CEC.2018.8477728"},{"key":"4582_CR51","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/j.ins.2012.03.005","volume":"210","author":"AR Yildiz","year":"2012","unstructured":"Yildiz AR. A comparative study of population-based optimization algorithms for turning operations. Inf Sci. 2012;210:81\u20138.","journal-title":"Inf Sci"},{"key":"4582_CR52","unstructured":"Hazan E, Klivans A, Yuan Y. Hyperparameter optimization. In: 6th international conference on learning representations, ICLR 2018; 2018."},{"key":"4582_CR53","doi-asserted-by":"publisher","unstructured":"Garnett R, Osborne MA, Roberts SJ. Sequential Bayesian prediction in the presence of changepoints. In: Proceedings of the 26th annual international conference on machine learning. ICML \u201909, pp. 345\u2013352. ACM, New York, NY, USA; 2009. https:\/\/doi.org\/10.1145\/1553374.1553418.","DOI":"10.1145\/1553374.1553418"},{"issue":"2","key":"4582_CR54","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1007\/s10514-009-9130-2","volume":"27","author":"R Martinez-Cantin","year":"2009","unstructured":"Martinez-Cantin R, de Freitas N, Brochu E, Castellanos J, Doucet A. A Bayesian exploration\u2013exploitation approach for optimal online sensing and planning with a visually guided mobile robot. Auton Robots. 2009;27(2):93\u2013103. https:\/\/doi.org\/10.1007\/s10514-009-9130-2.","journal-title":"Auton Robots"},{"issue":"2","key":"4582_CR55","doi-asserted-by":"publisher","first-page":"37","DOI":"10.3390\/computers9020037","volume":"9","author":"L Cappelletti","year":"2020","unstructured":"Cappelletti L, Fontana T, Donato GWD, Tucci LD, Casiraghi E, Valentini G. Complex data imputation by auto-encoders and convolutional neural networks-a case study on genome gap-filling. Computers. 2020;9(2):37.","journal-title":"Computers"},{"issue":"6","key":"4582_CR56","doi-asserted-by":"publisher","first-page":"508","DOI":"10.1186\/cc3000","volume":"8","author":"V Bewick","year":"2004","unstructured":"Bewick V, Cheek L, Ball J. Statistics review 13: receiver operating characteristic curves. Crit Care (London, England). 2004;8(6):508\u201312. https:\/\/doi.org\/10.1186\/cc3000.","journal-title":"Crit Care (London, England)"},{"key":"4582_CR57","first-page":"451","volume-title":"Machine learning and knowledge discovery in databases","author":"K Boyd","year":"2013","unstructured":"Boyd K, Eng KH, Page CD. Area under the precision-recall curve: point estimates and confidence intervals. In: Blockeel H, Kersting K, Nijssen S, \u017delezn\u00fd F, editors. Machine learning and knowledge discovery in databases. Berlin: Springer; 2013. p. 451\u201366."},{"issue":"8","key":"4582_CR58","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","volume":"27","author":"T Fawcett","year":"2006","unstructured":"Fawcett T. An introduction to ROC analysis. Pattern Recognit Lett. 2006;27(8):861\u201374. https:\/\/doi.org\/10.1016\/j.patrec.2005.10.010.","journal-title":"Pattern Recognit Lett"},{"issue":"9","key":"4582_CR59","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","volume":"21","author":"H He","year":"2009","unstructured":"He H, Garcia EA. Learning from imbalanced data. IEEE Trans Knowl Data Eng. 2009;21(9):1263\u201384. https:\/\/doi.org\/10.1109\/TKDE.2008.239.","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"3","key":"4582_CR60","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0118432","volume":"10","author":"T Saito","year":"2015","unstructured":"Saito T, Rehmsmeier M. The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE. 2015;10(3):1\u201321. https:\/\/doi.org\/10.1371\/journal.pone.0118432.","journal-title":"PLoS ONE"},{"issue":"6","key":"4582_CR61","doi-asserted-by":"publisher","first-page":"80","DOI":"10.2307\/3001968","volume":"1","author":"F Wilcoxon","year":"1945","unstructured":"Wilcoxon F. Individual comparisons by ranking methods. Biom Bull. 1945;1(6):80\u20133. https:\/\/doi.org\/10.2307\/3001968.","journal-title":"Biom Bull"},{"issue":"287","key":"4582_CR62","doi-asserted-by":"publisher","first-page":"655","DOI":"10.2307\/2282543","volume":"54","author":"JW Pratt","year":"1959","unstructured":"Pratt JW. Remarks on zeros and ties in the Wilcoxon signed rank procedures. J Am Stat Assoc. 1959;54(287):655\u201367. https:\/\/doi.org\/10.2307\/2282543.","journal-title":"J Am Stat Assoc"},{"key":"4582_CR63","unstructured":"Derrick B, White P. Comparing two samples from an individual likert question. Int J Math Stat. 2017;18(3)."},{"key":"4582_CR64","first-page":"2579","volume":"9","author":"Lvd Maaten","year":"2008","unstructured":"Maaten Lvd, Hinton G. Visualizing data using t-SNE. J Mach Learn Res. 2008;9:2579\u2013605.","journal-title":"J Mach Learn Res"},{"issue":"7","key":"4582_CR65","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1016\/j.cub.2019.02.034","volume":"29","author":"N Kriegeskorte","year":"2019","unstructured":"Kriegeskorte N, Golan T. Neural network models and deep learning. Curr Biol. 2019;29(7):231\u20136.","journal-title":"Curr Biol"},{"key":"4582_CR66","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.neucom.2016.12.038","volume":"234","author":"W Liu","year":"2017","unstructured":"Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE. A survey of deep neural network architectures and their applications. Neurocomputing. 2017;234:11\u201326.","journal-title":"Neurocomputing"},{"issue":"3","key":"4582_CR67","first-page":"1264","volume":"47","author":"P Sibi","year":"2013","unstructured":"Sibi P, Jones SA, Siddarth P. Analysis of different activation functions using back propagation neural networks. J Theor Appl Inf Technol. 2013;47(3):1264\u20138.","journal-title":"J Theor Appl Inf Technol"},{"key":"4582_CR68","doi-asserted-by":"publisher","DOI":"10.4467\/20838476SI.16.004.6185","author":"K Janocha","year":"2017","unstructured":"Janocha K, Czarnecki W. On loss functions for deep neural networks in classification. Schedae Informaticae. 2017. https:\/\/doi.org\/10.4467\/20838476SI.16.004.6185.","journal-title":"Schedae Informaticae"},{"key":"4582_CR69","doi-asserted-by":"crossref","unstructured":"Dogo E, Afolabi O, Nwulu N, Twala B, Aigbavboa C. A comparative analysis of gradient descent-based optimization algorithms on convolutional neural networks. In: 2018 international conference on computational techniques, electronics and mechanical systems (CTEMS), pp. 92\u201399 . IEEE 2018.","DOI":"10.1109\/CTEMS.2018.8769211"},{"key":"4582_CR70","unstructured":"Dozat T. Incorporating Nesterov momentum into adam; 2015."},{"key":"4582_CR71","unstructured":"Chollet F et al. Keras. GitHub; 2015."},{"key":"4582_CR72","unstructured":"Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S, Murray DG, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu, Y, Zheng X. Tensorflow: a system for large-scale machine learning. In: Proceedings of the 12th USENIX conference on operating systems design and implementation. OSDI\u201916, pp. 265\u2013283. USENIX Association, USA; 2016."},{"issue":"1","key":"4582_CR73","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1109\/JPROC.2015.2494218","volume":"104","author":"B Shahriari","year":"2016","unstructured":"Shahriari B, Swersky K, Wang Z, Adams RP, de Freitas N. Taking the human out of the loop: a review of Bayesian optimization. Proc IEEE. 2016;104(1):148\u201375. https:\/\/doi.org\/10.1109\/JPROC.2015.2494218.","journal-title":"Proc IEEE"},{"key":"4582_CR74","unstructured":"Moritz P, Nishihara R, Wang S, Tumanov A, Liaw R, Liang E, Elibol M, Yang Z, Paul W, Jordan MI et al. Ray: a distributed framework for emerging AI applications. In: 13th USENIX symposium on operating systems design and implementation (OSDI 18); 2018. p. 561\u2013577."},{"key":"4582_CR75","unstructured":"Bergstra J, Yamins D, Cox DD. Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In: Proceedings of the 30th international conference on machine learning\u2014volume 28. ICML\u201913. JMLR.org, Atlanta, GA, USA; 2013. p. 115\u2013123."},{"key":"4582_CR76","unstructured":"Nair V, Hinton GE. Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th international conference on international conference on machine learning. ICML\u201910. Omnipress, Madison, WI, USA; 2010. p. 807\u2013814."},{"key":"4582_CR77","doi-asserted-by":"publisher","unstructured":"LeCun Y, Bottou L, Orr G, M\u00fcller K. Efficient backprop. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol. 7700 LECTURE NO; 2012. p. 9\u201348. https:\/\/doi.org\/10.1007\/978-3-642-35289-8-3.","DOI":"10.1007\/978-3-642-35289-8-3"},{"key":"4582_CR78","unstructured":"Keskar NS, Mudigere D, Nocedal J, Smelyanskiy M, Tang PTP. On large-batch training for deep learning: generalization gap and sharp minima. CoRR abs\/1609.04836; 2016. arXiv:1609.04836."},{"key":"4582_CR79","unstructured":"Snoek J, Larochelle H, Adams RP. Practical Bayesian optimization of machine learning algorithms. In: Proceedings of the 25th international conference on neural information processing systems\u2014volume 2. NIPS\u201912. Curran Associates Inc., USA; 2012. p. 2951\u20132959."},{"key":"4582_CR80","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/1177932217700907","volume":"11","author":"R Alc\u00e1ntara-Silva","year":"2017","unstructured":"Alc\u00e1ntara-Silva R, Alvarado-Hermida M, D\u00edaz-Contreras G, S\u00e1nchez-Barrios M, Carrera S, Galv\u00e1n SC. Pisma: a visual representation of motif distribution in DNA sequences. Bioinform Biol Insights. 2017;11:1\u20139. https:\/\/doi.org\/10.1177\/1177932217700907.","journal-title":"Bioinform Biol Insights"},{"key":"4582_CR81","unstructured":"Kingma DP, Ba J. Adam: a method for stochastic optimization 2017. arXiv:1412.6980."},{"issue":"4","key":"4582_CR82","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1038\/s41576-018-0089-8","volume":"20","author":"SL Klemm","year":"2019","unstructured":"Klemm SL, Shipony Z, Greenleaf WJ. Chromatin accessibility and the regulatory epigenome. Nat Rev Genet. 2019;20(4):207\u201320.","journal-title":"Nat Rev Genet"},{"key":"4582_CR83","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1146\/annurev-genom-082509-141651","volume":"11","author":"JP Noonan","year":"2010","unstructured":"Noonan JP, McCallion AS. Genomics of long-range regulatory elements. Annu Rev Genom Hum Genet. 2010;11:1\u201323.","journal-title":"Annu Rev Genom Hum Genet"},{"issue":"7","key":"4582_CR84","first-page":"1","volume":"16","author":"W Hwang","year":"2015","unstructured":"Hwang W, Oliver VF, Merbs SL, Zhu H, Qian J. Prediction of promoters and enhancers using multiple DNA methylation-associated features. BMC Genom. 2015;16(7):1\u201313.","journal-title":"BMC Genom"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-022-04582-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-022-04582-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-022-04582-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,15]],"date-time":"2022-12-15T22:04:05Z","timestamp":1671141845000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-022-04582-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,12]]},"references-count":84,"journal-issue":{"issue":"S2","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["4582"],"URL":"https:\/\/doi.org\/10.1186\/s12859-022-04582-5","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,12]]},"assertion":[{"value":"17 January 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 January 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 December 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"154"}}