{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T13:49:52Z","timestamp":1784641792391,"version":"3.55.0"},"reference-count":67,"publisher":"Oxford University Press (OUP)","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2005,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Motivation: Cancer diagnosis is one of the most important emerging clinical applications of gene expression microarray technology. We are seeking to develop a computer system for powerful and reliable cancer diagnostic model creation based on microarray data. To keep a realistic perspective on clinical applications we focus on multicategory diagnosis. To equip the system with the optimum combination of classifier, gene selection and cross-validation methods, we performed a systematic and comprehensive evaluation of several major algorithms for multicategory classification, several gene selection methods, multiple ensemble classifier methods and two cross-validation designs using 11 datasets spanning 74 diagnostic categories and 41 cancer types and 12 normal tissue types.<\/jats:p>\n               <jats:p>Results: Multicategory support vector machines (MC-SVMs) are the most effective classifiers in performing accurate cancer diagnosis from gene expression data. The MC-SVM techniques by Crammer and Singer, Weston and Watkins and one-versus-rest were found to be the best methods in this domain. MC-SVMs outperform other popular machine learning algorithms, such as k-nearest neighbors, backpropagation and probabilistic neural networks, often to a remarkable degree. Gene selection techniques can significantly improve the classification performance of both MC-SVMs and other non-SVM learning algorithms. Ensemble classifiers do not generally improve performance of the best non-ensemble models. These results guided the construction of a software system GEMS (Gene Expression Model Selector) that automates high-quality model construction and enforces sound optimization and performance estimation procedures. This is the first such system to be informed by a rigorous comparative analysis of the available algorithms and datasets.<\/jats:p>\n               <jats:p>Availability: The software system GEMS is available for download from http:\/\/www.gems-system.org for non-commercial use.<\/jats:p>\n               <jats:p>Contact: \u00a0alexander.statnikov@vanderbilt.edu<\/jats:p>","DOI":"10.1093\/bioinformatics\/bti033","type":"journal-article","created":{"date-parts":[[2004,9,17]],"date-time":"2004-09-17T00:13:37Z","timestamp":1095380017000},"page":"631-643","source":"Crossref","is-referenced-by-count":645,"title":["A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis"],"prefix":"10.1093","volume":"21","author":[{"given":"Alexander","family":"Statnikov","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Constantin F.","family":"Aliferis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ioannis","family":"Tsamardinos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Douglas","family":"Hardin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shawn","family":"Levy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2004,9,16]]},"reference":[{"key":"2023013107212077700_B1","unstructured":"Aliferis, C.F., Tsamardinos, I., Massion, P., Statnikov, A., Fananapazir, N., Hardin, D. 2003Machine learning models for classification of lung cancer and selection of genomic markers using array gene expression data. Proceedings of the 16th International Florida Artificial Intelligence Research Society (FLAIRS) Conference , St Augustine, FL  May 12\u201314 AAAI Press,  pp. 67\u201371"},{"key":"2023013107212077700_B2","unstructured":"Aliferis, C.F., Tsamardinos, I., Massion, P., Statnikov, A., Hardin, D. 2003Why classification models using array gene expression data perform so well: a preliminary investigation of explanatory factors. Proceedings of the 2003 International Conference on Mathematics and Engineering Techniques in Medicine and Biological Sciences (METMBS) , Las Vegas, NV  June 23\u201326 CSREA Press"},{"key":"2023013107212077700_B3","unstructured":"Aliferis, C.F., Tsamardinos, I., Statnikov, A. 2003HITON, a novel Markov blanket algorithm for optimal variable selection. Proceedings of the 2003 American Medical Informatics Association (AMIA) Annual Symposium , Washington, DC ,  pp. 21\u201325 November 8\u201312"},{"key":"2023013107212077700_B4","unstructured":"Allwein, E.L., Schapire, R.E., Singer, Y. 2000Reducing multiclass to binary: a unifying approach for margin classifiers. J. Mach. Learn. Res.1113\u2013141"},{"key":"2023013107212077700_B5","unstructured":"Armstrong, S.A., Staunton, J.E., Silverman, L.B., Pieters, R., den Boer, M.L., Minden, M.D., Sallan, S.E., Lander, E.S., Golub, T.R., Korsmeyer, S.J. 2002MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia. Nat. Genet.3041\u201347"},{"key":"2023013107212077700_B6","unstructured":"Berrar, D., et al. 2003Multiclass cancer classification using gene expression profiling and probabilistic neural networks. Proceedings of the Pacific Symposium on Biocomputing (PSB) , Hawaii  January 3\u20137 Lihue"},{"key":"2023013107212077700_B7","doi-asserted-by":"crossref","unstructured":"Bhattacherjee, A., Richards, W.G., Staunton, J., Li, C., Monti, S., Vasa, P., Ladd, C., Beheshti, J., Bueno, R., Gillete, M., et al. 2001Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclasses. Proc. Natl Acad. Sci. USA98,  pp. 13790\u201313795","DOI":"10.1073\/pnas.191502998"},{"key":"2023013107212077700_B8","unstructured":"Chang, C.-C. and Lin, C.-J. 2003LIBSVM: a library for support vector machines"},{"key":"2023013107212077700_B9","unstructured":"Cortes, C., Jackel, L.D., Solla, S.A., Vapnik, V., Denker, J.S. 1993Learning curves: asymptotic values and rate of convergence. Advances in Neural Information Processing Systems (NIPS) , Denver, CO ,  pp. 327\u2013334"},{"key":"2023013107212077700_B10","unstructured":"Crammer, K. and Singer, Y. 2000On the learnability and design of output codes for multiclass problems. Proceedings of the Thirteen Annual Conference on Computational Learning Theory (COLT 2000) , Palo Alto, CA  June 28\u2013July 1 Standford University"},{"key":"2023013107212077700_B11","unstructured":"Demuth, H. and Beale, M. 2001Neural network toolbox user's guide. Matlab user's guide.  , Natick, MA  The MathWorks Inc"},{"key":"2023013107212077700_B12","doi-asserted-by":"crossref","unstructured":"Dietterich, T.G. 1998Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput.10,  pp. 1895\u20131923","DOI":"10.1162\/089976698300017197"},{"key":"2023013107212077700_B13","unstructured":"Duda, R.O., Hart, P.E., Stork, D.G. Pattern Classification2001 2nd edn , NY  John Wiley"},{"key":"2023013107212077700_B14","doi-asserted-by":"crossref","unstructured":"Dudoit, S., Fridlyand, J., Speed, T.P. 2002Comparison of discrimination methods for the classification of tumors using gene expression data. J. Am. Stat. Assoc.97,  pp. 77\u201387","DOI":"10.1198\/016214502753479248"},{"key":"2023013107212077700_B15","doi-asserted-by":"crossref","unstructured":"Ferri, C., Hern\u00e1ndez-Orallo, J., Salido, M.A. 2003Volume under the ROC surface for multi-class problems. Proceedings of the 14th European Conference on Machine Learning (ECML'03) , Cavtat-Dubrovnik, Croatia  September 22\u201326, LNAI 2837 Springer-Verlag,  pp. 108\u2013120","DOI":"10.1007\/978-3-540-39857-8_12"},{"key":"2023013107212077700_B16","doi-asserted-by":"crossref","unstructured":"Fortina, P., Surrey, S., Kricka, L.J. 2002Molecular diagnostics: hurdles for clinical implementation. Trends Mol. Med.8264\u2013266","DOI":"10.1016\/S1471-4914(02)02331-6"},{"key":"2023013107212077700_B17","unstructured":"Freund, Y. 1995Boosting a weak learning algorithm by majority. Inform. Comput.121256\u2013285"},{"key":"2023013107212077700_B18","unstructured":"Technical Report. Friedman, J. 1996Another approach to polychotomous classification.  , CA  Stanford University"},{"key":"2023013107212077700_B19","doi-asserted-by":"crossref","unstructured":"Furey, T.S., Cristianini, N., Duffy, N., Bednarski, D.W., Schummer, M., Haussler, D. 2000Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics16906\u2013914","DOI":"10.1093\/bioinformatics\/16.10.906"},{"key":"2023013107212077700_B20","doi-asserted-by":"crossref","unstructured":"Golub, T.R., Slonim, D.K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J.P., Coller, H., Loh, M.L., Downing, J.R., Caligiuri, M.A., Bloomfield, C.D., Lander, E.S. 1999Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science286531\u2013537","DOI":"10.1126\/science.286.5439.531"},{"key":"2023013107212077700_B21","unstructured":"Good, P.I. Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses2000 2nd edn , NY  Springer-Verlag"},{"key":"2023013107212077700_B22","unstructured":"Goodman, P.H. and Harrell, F.E. 2004NevProp Manual with Introduction to Artificial Neural Networks Theory"},{"key":"2023013107212077700_B23","doi-asserted-by":"crossref","unstructured":"Guyon, I., Weston, J., Barnhill, S., Vapnik, V. 2002Gene selection for cancer classification using support vector machines. Machine Learning46,  pp. 389\u2013422","DOI":"10.1023\/A:1012487302797"},{"key":"2023013107212077700_B24","unstructured":"Guyon, I., Weston, J., Barnhill, S., Vapnik, V. 2003Erratum: gene selection for cancer classification using support vector machines"},{"key":"2023013107212077700_B25","doi-asserted-by":"crossref","unstructured":"Hardin, D., Tsamardinos, I., Aliferis, C.F. 2004A theoretical characterization of linear SVM-based feature selection. Twenty-First International Conference on Machine Learning (ICML 2004) , Alberta  Canada, July 4\u20138 Banff","DOI":"10.1145\/1015330.1015421"},{"key":"2023013107212077700_B26","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., Friedman, J. Elements of Statistical Learning: Data Mining, Inference and Prediction2001, NY  Springer-Verlag","DOI":"10.1007\/978-0-387-21606-5"},{"key":"2023013107212077700_B27","doi-asserted-by":"crossref","unstructured":"Herbrich, R. Learning Kernel Classifiers: Theory and Algorithms2002, Cambridge, MA, USA  MIT Press","DOI":"10.7551\/mitpress\/4170.001.0001"},{"key":"2023013107212077700_B28","doi-asserted-by":"crossref","unstructured":"Ho, T.K., Hull, J.J., Srihari, S.N. 1994Decision combination in multiple classifier systems. IEEE Trans. Pattern Anal. Machine Intell.16,  pp. 66\u201376","DOI":"10.1109\/34.273716"},{"key":"2023013107212077700_B29","unstructured":"Hsu, C.-W. and Lin, C.-J. 2002A comparison of methods for multi-class support vector machines. IEEE Trans. Neural Netw.13415\u2013425"},{"key":"2023013107212077700_B30","unstructured":"Joachims, T. 1999Making large-scale SVM learning practical. In Sch\u00f6lkopf, B., Burges, C., Smola, A. (Eds.). Advances in Kernel Methods\u2014Support Vector Learning , Cambridge, MA, USA  MIT Press"},{"key":"2023013107212077700_B31","unstructured":"Jones, B. 1997Matlab Statistics Toolbox.  , Natick, MA  The MathWorks, Inc"},{"key":"2023013107212077700_B32","doi-asserted-by":"crossref","unstructured":"Khan, J., Wei, J.S., Ringner, M., Saal, L.H., Ladanyi, M., Westermann, F., Berthold, F., Schwab, M., Antonescu, C.R., Peterson, C., Meltzer, P.S. 2001Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nat. Med.7,  pp. 673\u2013679","DOI":"10.1038\/89044"},{"key":"2023013107212077700_B33","unstructured":"Kohavi, R. 1995A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the International Joint Conferences on Artificial Intelligence (IJCAI 1995) , Montreal, Quebec, Canada  August 20\u201325 Morgan Kaufmann Publishers,  pp. 1137\u20131145"},{"key":"2023013107212077700_B34","doi-asserted-by":"crossref","unstructured":"Kressel, U. 1999Pairwise classification and support vector machines. Advances in Kernel Methods: Support Vector Learning , Cambridge, MA, USA  (Chapter 15.) MIT Press","DOI":"10.7551\/mitpress\/1130.003.0020"},{"key":"2023013107212077700_B35","unstructured":"Kutin, S. and Niyogi, P. 2002Almost-everywhere algorithmic stability and generalization error. Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence (UAI-2002) , Edmonton, Canada  August 1\u20134 University of Alberta, Morgan Kaufmann Publishers,  pp. 275\u2013282"},{"key":"2023013107212077700_B36","doi-asserted-by":"crossref","unstructured":"Lee, Y. and Lee, C.-K. 2003Classification of multiple cancer types by multicategory support vector machines using gene expression data. Bioinformatics191132\u20131139","DOI":"10.1093\/bioinformatics\/btg102"},{"key":"2023013107212077700_B37","unstructured":"Lin, C.-J. and Mor\u00e9, J.J. 1999Newton's method for large bound-constrained optimization problems. SIAM J. Optimization91100\u20131127"},{"key":"2023013107212077700_B38","doi-asserted-by":"crossref","unstructured":"Lu, J., Hardy, S., Tao, W.L., Muse, S., Weir, B., Spruill, S. 2002Classical statistical approaches to molecular classification of cancer from gene expression profiling. In Lin, S.M. and Johnson, K.F. (Eds.). Methods of Microarray Data Analysis: Papers from CAMDA\u201900 , Boston\/Dordrecht\/London  Kluwer Academic Publishers,  pp. 97\u2013107","DOI":"10.1007\/978-1-4615-0873-1_8"},{"key":"2023013107212077700_B39","unstructured":"Mitchell, T.M. Machine Learning1997, New York, NY, USA  McGraw-Hill"},{"key":"2023013107212077700_B40","doi-asserted-by":"crossref","unstructured":"Mossman, D. 1999Three-way ROCs. Med. Decis. Making19,  pp. 78\u201389","DOI":"10.1177\/0272989X9901900110"},{"key":"2023013107212077700_B41","unstructured":"Mukherjee, S. Classifying Microarray Data Using Support Vector Machines, Understanding And Using Microarray Analysis Techniques: A Practical Guide2003, Boston, MA  Kluwer Academic Publishers"},{"key":"2023013107212077700_B42","unstructured":"Murthy, S.K. 1998Automatic construction of decision trees from data: a multi-disciplinary survey. Data Min. Knowl. Discov.2,  pp. 345\u2013389"},{"key":"2023013107212077700_B43","unstructured":"Ntzani, E.E. and Ioannidis, J.P. 2003Predictive ability of DNA microarrays for cancer outcomes and correlates: and empirical assessment. Lancet3621439\u20131444"},{"key":"2023013107212077700_B44","unstructured":"Nutt, C.L., Mani, D.R., Betensky, R.A., Tamayo, P., Cairncross, J.G., Ladd, C., Pohl, U., Hartmann, C., McLaughlin, M.E., Batchelor, T.T., et al. 2003Gene expression-based classification of malignant gliomas correlates better with survival than histological classification. Cancer Res.631602\u20131607"},{"key":"2023013107212077700_B45","doi-asserted-by":"crossref","unstructured":"Platt, J. 1999Fast training of support vector machines using sequential minimal optimization. In Sch\u00f6lkopf, B., Burges, C., Smola, A. (Eds.). Advances in Kernel Methods\u2014Support Vector Learning , Cambridge, MA, USA  MIT Press","DOI":"10.7551\/mitpress\/1130.003.0016"},{"key":"2023013107212077700_B46","unstructured":"Advances in Neural Information Processing Systems 12. Platt, J.C., Cristianini, N., Shawe-Taylor, J. Large margin DAGS for multiclass classification.  MIT Press,  pp. 547\u2013553"},{"key":"2023013107212077700_B47","unstructured":"Pomeroy, S.L., Tamayo, P., Gaasenbeek, M., Sturla, L.M., Angelo, M., McLaughlin, M.E., Kim, J.Y., Goumnerova, L.C., Black, P.M., Lau, C., et al. 2002Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature415436\u2013442"},{"key":"2023013107212077700_B48","doi-asserted-by":"crossref","unstructured":"Ramaswamy, S., Tamayo, P., Rifkin, R., Mukherjee, S., Yeang, C.H., Angelo, M., Ladd, C., Reich, M., Latulippe, E., Mesirov, J.P., et al. 2001Multiclass cancer diagnosis using tumor gene expression signatures. Proc. Natl Acad. Sci. USA9815149\u201315154","DOI":"10.1073\/pnas.211566398"},{"key":"2023013107212077700_B49","doi-asserted-by":"crossref","unstructured":"Reich, M., Ohm, K., Angelo, M., Tamayo, P., Mesirov, J.P. 2004GeneCluster 2.0: an advanced toolset for bioarray analysis. Bioinformatics201797\u20131798","DOI":"10.1093\/bioinformatics\/bth138"},{"key":"2023013107212077700_B50","unstructured":"Reunanen, J. 2003Overfitting in making comparisons between variable selection methods. J. Machine Learn. Res.31371\u20131382"},{"key":"2023013107212077700_B51","unstructured":"Romualdi, C., Campanaro, S., Campagna, D., Celegato, B., Cannata, N., Toppo, S., Valle, G., Lanfranchi, G. 2003Pattern recognition in gene expression profiling using DNA array: a comparative study of different statistical methods applied to cancer classification. Hum. Mol. Genet.12823\u2013836"},{"key":"2023013107212077700_B52","unstructured":"Schwarzer, G. and Vach, W. 2000On the misuses of artificial neural networks for prognostic and diagnostic classification in oncology. Stat. Med.19541\u2013561"},{"key":"2023013107212077700_B53","doi-asserted-by":"crossref","unstructured":"Sindwani, V., et al. 2001Information theoretic feature crediting in multiclass support vector machines. First SIAM International Conference on Data Mining (ICDM\u201901) , Chicago IL  April 5\u20137","DOI":"10.1137\/1.9781611972719.16"},{"key":"2023013107212077700_B54","doi-asserted-by":"crossref","unstructured":"Singh, D., Febbo, P.G., Ross, K., Jackson, D.G., Manola, J., Ladd, C., Tamayo, P., Renshaw, A.A., D\u2019Amico, A.V., Richie, J.P., et al. 2002Gene expression correlates of clinical prostate cancer behavior. Cancer Cell ,  pp. 203\u2013209","DOI":"10.1016\/S1535-6108(02)00030-2"},{"key":"2023013107212077700_B55","unstructured":"Sharkey, A.J.C. 1996On combining artificial neural net. Connection Sci.8299\u2013314"},{"key":"2023013107212077700_B56","doi-asserted-by":"crossref","unstructured":"Shipp, M.A., Ross, K.N., Tamayo, P., Weng, A.P., Kutok, J.L., Aguiar, R.C., Gaasenbeek, M., Angelo, M., Reich, M., Pinkus, G.S., et al. 2002Diffuse large B-cell lymphoma outcome prediction by gene expression profiling and supervised machine learning. Nat. Med.868\u201374","DOI":"10.1038\/nm0102-68"},{"key":"2023013107212077700_B57","unstructured":"Specht, D.F. 1990Probabilistic neural network. Neural Networks3109\u2013118"},{"key":"2023013107212077700_B58","doi-asserted-by":"crossref","unstructured":"Staunton, J.E., Slonim, D.K., Coller, H.A., Tamayo, P., Angelo, M.J., Park, J., Scherf, U., Lee, J.K., Reinhold, W.O., Weinstein, J.N., et al. 2001Chemosensitivity prediction by transcriptional profiling. Proc. Natl Acad. Sci. USA9810787\u201310792","DOI":"10.1073\/pnas.191368598"},{"key":"2023013107212077700_B59","unstructured":"Su, A.I., Welsh, J.B., Sapinoso, L.M., Kern, S.G., Dimitrov, P., Lapp, H., Schultz, P.G., Powell, S.M., Moskaluk, C.A., Frierson, H.F., Jr and Hampton, G.M. 2001Molecular classification of human carcinomas by use of gene expression signatures. Cancer Res.617388\u20137393"},{"key":"2023013107212077700_B60","doi-asserted-by":"crossref","unstructured":"Tsamardinos, I., Aliferis, C.F., Statnikov, A. 2003Time and sample efficient discovery of Markov blankets and direct causal relations. The 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2003) , Washington DC  August 24\u201327","DOI":"10.1145\/956750.956838"},{"key":"2023013107212077700_B61","unstructured":"Valentini, G., Muselli, M., Ruffino, F. 2003Bagged ensembles of SVMs for gene expression data analysis. The IEEE-INNS-ENNS International Joint Conference on Neural Networks (IJCNN2003) , Portland, OR"},{"key":"2023013107212077700_B62","unstructured":"Vapnik, V. Statistical Learning Theory1998, New York, NY, USA  Wiley-Interscience"},{"key":"2023013107212077700_B63","unstructured":"Weiss, S.M. and Kulikowski, C.A. Computer Systems that Learn1991, San Mateo, CA, USA  Morgan Kaufmann"},{"key":"2023013107212077700_B64","unstructured":"Proceedings of the Seventh European Symposium On Artificial Neural Networks. Weston, J. and Watkins, C. 1999Support vector machines for multi-class pattern recognition.   (ESANN 99), Bruges, April 21\u201323"},{"key":"2023013107212077700_B65","doi-asserted-by":"crossref","unstructured":"Wouters, L., Gohlmann, H.W., Bjinens, L., Kass, S.U., Molenberghs, G., Lewi, P.J. 2003Graphical exploration of gene expression data: a comparative study of three multivariate methods. Biometrics59,  pp. 1131\u20131139","DOI":"10.1111\/j.0006-341X.2003.00130.x"},{"key":"2023013107212077700_B66","doi-asserted-by":"crossref","unstructured":"Yeang, C., Ramaswamy, S., Tamayo, P., Rifkin, R., Mukherjee, S., Angelo, M., Reich, M., Lander, E., Mesirov, J., Golub, T. 2001Molecular classification of multiple tumor types. Proceedings of the Ninth International Conference on Intelligent Systems in Molecular Biology , Denmark  July 21\u201325 Copenhagen,  pp. 316\u2013322","DOI":"10.1093\/bioinformatics\/17.suppl_1.S316"},{"key":"2023013107212077700_B67","unstructured":"Technical Report AI Memo 2001\u2212018 CBCL Memo 206. Yeo, G. and Poggio, T. 2001Mutliclass classification of SRBCT tumors.   MIT Press"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/21\/5\/631\/48962354\/bioinformatics_21_5_631.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/21\/5\/631\/48962354\/bioinformatics_21_5_631.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T10:01:15Z","timestamp":1675159275000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/21\/5\/631\/219898"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2004,9,16]]},"references-count":67,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2005,3,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/bti033","relation":{},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"value":"1367-4811","type":"electronic"},{"value":"1367-4803","type":"print"}],"subject":[],"published-other":{"date-parts":[[2005,3,1]]},"published":{"date-parts":[[2004,9,16]]}}}