{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T09:25:02Z","timestamp":1790673902194,"version":"4.1.0"},"reference-count":51,"publisher":"SAGE Publications","issue":"3","license":[{"start":{"date-parts":[[2008,6,1]],"date-time":"2008-06-01T00:00:00Z","timestamp":1212278400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Clinical Trials"],"published-print":{"date-parts":[[2008,6]]},"abstract":"<jats:p>Background: The evaluation and validation of surrogate endpoints have been extensively studied in the last decade. Prentice [1] and Freedman, Graubard and Schatzkin [2] laid the foundations for the evaluation of surrogate endpoints in randomized clinical trials. Later, Buyse et al. [5] proposed a meta-analytic methodology, producing different methods for different settings, which was further studied by Alonso and Molenberghs [9], in their unifying approach based on information theory.<\/jats:p>\n                  <jats:p>Purpose: In this article, we focus our attention on the trial-level surrogacy and propose alternative procedures to evaluate such surrogacy measure, which do not pre-specify the type of association. A promising correction based on cross-validation is investigated. As well as the construction of confidence intervals for this measure.<\/jats:p>\n                  <jats:p>Methods: In order to avoid making assumption about the type of relationship between the treatment effects and its distribution, a collection of alternative methods, based on regression trees, bagging, random forests, and support vector machines, combined with bootstrap-based confidence interval and, should one wish, in conjunction with a cross-validation based correction, will be proposed and applied. We apply the various strategies to data from three clinical studies: in opthalmology, in advanced colorectal cancer, and in schizophrenia.<\/jats:p>\n                  <jats:p>Results: The results obtained for the three case studies are compared; they indicate that using random forest or bagging models produces larger estimated values for the surrogacy measure, which are in general stabler and the confidence interval narrower than linear regression and support vector regression. For the advanced colorectal cancer studies, we even found the trial-level surrogacy is considerably different from what has been reported.<\/jats:p>\n                  <jats:p>Limitations: In general the alternative methods are more computationally demanding, and specially the calculation of the confidence intervals, require more computational time that the delta-method counterpart.<\/jats:p>\n                  <jats:p>Conclusions: First, more flexible modeling techniques can be used, allowing for other type of association. Second, when no cross-validation-based correction is applied, overly optimistic trial-level surrogacy estimates will be found, thus cross-validation is highly recommendable. Third, the use of the delta method to calculate confidence intervals is not recommendable since it makes assumptions valid only in very large samples. It may also produce range-violating limits. We therefore recommend alternatives: bootstrap methods in general. Also, the information-theoretic approach produces comparable results with the bagging and random forest approaches, when cross-validation correction is applied. It is also important to observe that, even for the case in which the linear model might be a good option too, bagging methods perform well too, and their confidence intervals were more narrow. Clinical Trials 2008; 5: 194\u2014208. http:\/\/ctj.sagepub.com<\/jats:p>","DOI":"10.1177\/1740774508091677","type":"journal-article","created":{"date-parts":[[2008,6,16]],"date-time":"2008-06-16T12:18:06Z","timestamp":1213618686000},"page":"194-208","source":"Crossref","is-referenced-by-count":12,"title":["Alternative methods to evaluate trial level surrogacy"],"prefix":"10.1177","volume":"5","author":[{"given":"Jos\u00e8 Corti\u00f1as","family":"Abrahantes","sequence":"first","affiliation":[{"name":"Center for Statistics, Hasselt University, Campus Diepenbeek, B3590                         Diepenbeek, Belgium,"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ziv","family":"Shkedy","sequence":"additional","affiliation":[{"name":"Center for Statistics, Hasselt University, Campus Diepenbeek, B3590                         Diepenbeek, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Geert","family":"Molenberghs","sequence":"additional","affiliation":[{"name":"Center for Statistics, Hasselt University, Campus Diepenbeek, B3590                         Diepenbeek, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2008,6,16]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.4780080407"},{"key":"atypb2","doi-asserted-by":"publisher","DOI":"10.1002\/sim.4780110204"},{"key":"atypb3","doi-asserted-by":"publisher","DOI":"10.2307\/2533853"},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-0258(19970915)16:17<1965::AID-SIM630>3.0.CO;2-M"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.1093\/biostatistics\/1.1.49"},{"key":"atypb6","doi-asserted-by":"publisher","DOI":"10.1093\/biostatistics\/1.3.231"},{"key":"atypb7","doi-asserted-by":"publisher","DOI":"10.1007\/b138566"},{"key":"atypb8","doi-asserted-by":"publisher","DOI":"10.1080\/0094965031000062177"},{"key":"atypb9","doi-asserted-by":"publisher","DOI":"10.1111\/j.1541-0420.2006.00634.x"},{"key":"atypb10","first-page":"63","volume":"65","author":"Oosterlinck W.","year":"1997","journal-title":"Acta Urol Belg"},{"key":"atypb11","volume-title":"Linear Mixed Models for Longitudinal Data","author":"Verbeke G.","year":"2000"},{"key":"atypb12","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.1995.13.4.921"},{"key":"atypb13","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.1996.14.10.2674"},{"key":"atypb14","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-985X.2004.00293.x"},{"key":"atypb15","doi-asserted-by":"publisher","DOI":"10.1081\/BIP-120015741"},{"key":"atypb16","doi-asserted-by":"publisher","DOI":"10.1016\/j.csda.2003.12.003"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1016\/0165-1781(88)90038-8"},{"key":"atypb18","doi-asserted-by":"publisher","DOI":"10.1002\/bimj.200390061"},{"key":"atypb19","doi-asserted-by":"publisher","DOI":"10.1080\/03610929408831436"},{"key":"atypb20","volume-title":"Models for Discrete Longitudinal Data","author":"Molenberghs G.","year":"2005"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1016\/0167-9473(96)00002-3"},{"key":"atypb22","first-page":"269","volume":"48","author":"Verbyla AP","year":"1999","journal-title":"Applied Statistics"},{"key":"atypb23","unstructured":"Alonso Abad A. Investigating validity of psychiatric symptom scales and surrogate                 markers. Doctoral Thesis, 2004; 48-86,                         Center for statistics, Hasselt University,                         Belgium."},{"key":"atypb24","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.69.066138"},{"key":"atypb25","first-page":"548","volume":"92","author":"Efron B.","year":"1997","journal-title":"Journal of the American Statistical Association"},{"key":"atypb26","volume-title":"Artificial Neural Networks and Neural Information Processing","author":"Lendasse A."},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1093\/biostatistics\/kxi040"},{"key":"atypb28","volume-title":"Classification and Regression Trees","author":"Breiman L.","year":"1984"},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1007\/BF00058655"},{"key":"atypb30","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1032181158"},{"key":"atypb31","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"atypb32","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-2440-0"},{"key":"atypb33","volume-title":"Benchmarking support vector machines. Technical Report bfseries 78, SFB Adaptive Information Systems and Modeling in Economics and Management Science","author":"Meyer D.","year":"2002"},{"key":"atypb34","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1080\/10618600.1996.10474713","volume":"5","author":"Ihaka R.","year":"1996","journal-title":"Journal of Computional and Graphical Statistics"},{"key":"atypb35","volume-title":"An introduction to recursive partitioning using the part routines","author":"Therneau TM","year":"1997"},{"key":"atypb36","first-page":"18","volume":"2","author":"Liaw A.","year":"2002","journal-title":"The Newsletter of the R Project"},{"key":"atypb37","first-page":"23","volume":"1","author":"Meyer D.","year":"2001","journal-title":"The Newsletter of the R Project"},{"key":"atypb38","doi-asserted-by":"publisher","DOI":"10.1139\/x87-177"},{"key":"atypb39","doi-asserted-by":"publisher","DOI":"10.1021\/ci034160g"},{"key":"atypb40","first-page":"59","volume":"15","author":"Moore DE","year":"1991","journal-title":"Journal of Environmental Management"},{"key":"atypb41","doi-asserted-by":"publisher","DOI":"10.1016\/S0925-2312(03)00431-4"},{"key":"atypb42","unstructured":"Drucker H.                 ,                      Burges CJC                 ,                      Kaufman L.                 ,                      Smola A.                 ,                      Vapnik V. Support vector regression machines. In:                      Mozer MC                 ,                      Jordan MI                 ,                      Petsche T.                  (eds), Advances in Neural Information Processing Systems 9,                         MIT Press, Cambridge,                     MA, 1997, pp.             155-61."},{"key":"atypb43","doi-asserted-by":"crossref","unstructured":"M\u00fcller KR                 ,                      Smola A.                 ,                      R\u00e4tsch G.                 ,                      Sch\u00f6lkopf B.                 ,                      Kohlmorgen J.                 ,                      Vapnik V. Predicting time series with support vector machines. In:                      Gerstner W                 ,                      Germond A                 ,                      Hasler M                 , and                      Nicoud J.D.                  (eds), Artificial Neural Networks ICANN 97, Springer Lecture Notes in                     Computer Science,                      Berlin                 . 1327, 1997,             999-1004.","DOI":"10.1007\/BFb0020283"},{"key":"atypb44","doi-asserted-by":"crossref","unstructured":"Stitson M.                 ,                      Gammerman A.                 ,                      Vapnik V.                 ,                      Vovk V.                 ,                      Watkins C.                 ,                      Weston J. Support vector regression with ANOVA decomposition kernels. In:                      Sch\u00f6lkopf B                 ,                      Burges CJC                  ,                      Smola AJ.                  (eds), Advances in Kernel Methods-Support Vector Learning,                      MIT Press Cambridge, MA                 , 1999, pp. 285-92.","DOI":"10.7551\/mitpress\/1130.003.0023"},{"key":"atypb45","doi-asserted-by":"crossref","unstructured":"Mattera D.                 ,                      Haykin S. Support vector machines for dynamic reconstruction of a chaotic system .                 In:                      Sch\u00f6lkopf B                 ,                      Burges CJC                 ,                      Smola AJ.                  (eds), Advances in Kernel Methods-Support Vector Learning,                         MIT Press, Cambridge,                     MA, 1999, pp.             211-42.","DOI":"10.7551\/mitpress\/1130.003.0018"},{"key":"atypb46","first-page":"774","volume":"24","author":"Vapnik V.","year":"1963","journal-title":"Automation and Remote Control"},{"key":"atypb47","first-page":"821","volume":"25","author":"Vapnik V.","year":"1964","journal-title":"Automation and Remote Control"},{"key":"atypb48","volume-title":"Regression estimation with support vector learning machines","author":"Smola A.","year":"1996"},{"key":"atypb49","volume-title":"Practical Methods of Optimization","author":"Fletcher R.","year":"1989"},{"key":"atypb50","volume-title":"A Practical Guide to Support Vector Classification","author":"Hsu CW","year":"2001"},{"key":"atypb51","volume-title":"An Introduction to Multivariate Statistical Analysis","author":"Anderson TW","year":"1958"}],"container-title":["Clinical Trials"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1740774508091677","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1740774508091677","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T17:30:07Z","timestamp":1777397407000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1740774508091677"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,6]]},"references-count":51,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2008,6]]}},"alternative-id":["10.1177\/1740774508091677"],"URL":"https:\/\/doi.org\/10.1177\/1740774508091677","relation":{},"ISSN":["1740-7745","1740-7753"],"issn-type":[{"value":"1740-7745","type":"print"},{"value":"1740-7753","type":"electronic"}],"subject":[],"published":{"date-parts":[[2008,6]]}}}