{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T12:32:46Z","timestamp":1784550766408,"version":"3.55.0"},"reference-count":63,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2008,7,30]],"date-time":"2008-07-30T00:00:00Z","timestamp":1217376000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/2.0"},{"start":{"date-parts":[[2008,7,30]],"date-time":"2008-07-30T00:00:00Z","timestamp":1217376000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/2.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Data Min Knowl Disc"],"published-print":{"date-parts":[[2009,2]]},"DOI":"10.1007\/s10618-008-0114-1","type":"journal-article","created":{"date-parts":[[2008,7,29]],"date-time":"2008-07-29T19:50:28Z","timestamp":1217361028000},"page":"140-181","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":497,"title":["Controlled experiments on the web: survey and practical guide"],"prefix":"10.1007","volume":"18","author":[{"given":"Ron","family":"Kohavi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roger","family":"Longbotham","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dan","family":"Sommerfield","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Randal M.","family":"Henne","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2008,7,30]]},"reference":[{"key":"114_CR1","unstructured":"Alt B, Usborne N (2005) Market Exp J. [Online] December 29, 2005. http:\/\/www.marketingexperiments.com\/improving-website-conversion\/multivariable-testing.html"},{"issue":"2","key":"114_CR2","doi-asserted-by":"publisher","first-page":"121","DOI":"10.2307\/2686030","volume":"54","author":"DD Boos","year":"2000","unstructured":"Boos DD, Hughes-Oliver JM (2000) How large does n have to be for Z and t intervals?. Am Statist 54(2): 121\u2013128","journal-title":"Am Statist"},{"key":"114_CR3","unstructured":"Box GEP, Hunter JS, Hunter WG (2005) Statistics for experimenters: design, innovation, and discovery, 2nd edn. Wiley, ISBN: 0471718130"},{"key":"114_CR4","volume-title":"Web analytics spendings trends 2007","author":"M Burns","year":"2006","unstructured":"Burns M (2006) Web analytics spendings trends 2007. Forrester Research Inc., Cambridge"},{"key":"114_CR5","unstructured":"Charles RS, Melvin MM (2004) Quasi experimentation. [book auth.] In: Wholey JS, Hatry HP, Newcomer KE (eds) Handbook of practical program evaluation, 2nd edn. Jossey-Bass"},{"key":"114_CR6","unstructured":"Chatham B, Temkin BD, Amato M (2004) A primer on A\/B testing. Forrester Research"},{"issue":"6","key":"114_CR7","first-page":"121","volume":"233","author":"OL Davies","year":"1950","unstructured":"Davies OL, Hay WA (1950) Construction and uses of fractional factorial designs in industrial research. Biometrics 233(6): 121\u2013128","journal-title":"Biometrics"},{"key":"114_CR8","unstructured":"Eisenberg B (2003a) How to Decrease sales by 90%. ClickZ. [Online] Feb 21, 2003. http:\/\/www.clickz.com\/showPage.html?page=1588161"},{"key":"114_CR9","unstructured":"Eisenberg B (2003b) How to increase conversion rate 1,000%. ClickZ. [Online] Feb 28, 2003. http:\/\/www.clickz.com\/showPage.html?page=1756031"},{"key":"114_CR10","unstructured":"Eisenberg B (2004) A\/B testing for the mathematically disinclined. ClickZ. [Online] May 7, 2004. http:\/\/www.clickz.com\/showPage.html?page=3349901"},{"key":"114_CR11","unstructured":"Eisenberg B (2005) How to improve A\/B testing. ClickZ Netw. [Online] April 29, 2005. http:\/\/www.clickz.com\/showPage.html?page=3500811"},{"key":"114_CR12","unstructured":"Eisenberg B, Eisenberg J (2005) Call to action, secret formulas to improve online results. Wizard Academy Press, Austin, 2005. Making the dial move by testing, introducing A\/B testing"},{"key":"114_CR13","unstructured":"Eisenberg B, Garcia A (2006) Which sells best: a quick start guide to testing for retailers. Future now\u2019s publications. [Online] 2006. http:\/\/futurenowinc.com\/shop\/"},{"key":"114_CR14","unstructured":"Forrester Research (2005) The state of retailing online. Shop.org"},{"key":"114_CR15","unstructured":"Google Website Optimizer (2008) [Online] 2008. http:\/\/services.google.com\/websiteoptimizer"},{"key":"114_CR16","unstructured":"Hawthorne effect (2007) Wikipedia. [Online] 2007. http:\/\/en.wikipedia.org\/wiki\/Hawthorne_experiments"},{"key":"114_CR17","volume-title":"Scientific advertising","author":"C Hopkins","year":"1923","unstructured":"Hopkins C (1923) Scientific advertising. Crown Publishers Inc., New York City"},{"key":"114_CR18","unstructured":"Kaplan RS, Norton DP (1996) The balanced scorecard: translating strategy into action. Harvard Business School Press, ISBN: 0875846513"},{"key":"114_CR19","unstructured":"Kaushik A (2006) Experimentation and testing: a primer. Occam\u2019s Razor by Avinash Kaushik. [Online] May 22, 2006. http:\/\/www.kaushik.net\/avinash\/2006\/05\/experimentation-and-testing-a-primer.html"},{"key":"114_CR20","unstructured":"Keppel G, Saufley WH, Tokunaga H (1992) Introduction to design and analysis, 2nd edn. W.H. Freeman and Company"},{"key":"114_CR21","unstructured":"Kohavi R (2007) Emetrics 2007 practical guide to controlled experiments on the web. [Online] October 16, 2007. http:\/\/exp-platform.com\/Documents\/2007-10EmetricsExperimenation.pdf"},{"key":"114_CR22","unstructured":"Kohavi R, Parekh R (2003) Ten supplementary analyses to improve e-commerce web sites. WebKDD"},{"key":"114_CR23","unstructured":"Kohavi R, Round M (2004) In: Sterne J (ed) Front line internet analytics at Amazon.com. Santa Barbara, CA. http:\/\/ai.stanford.edu\/~ronnyk\/emetricsAmazon.pdf"},{"issue":"1\u20132","key":"114_CR24","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1023\/B:MACH.0000035473.11134.83","volume":"57","author":"R Kohavi","year":"2004","unstructured":"Kohavi R et al (2004) Lessons and challenges from mining retail e-commerce data. Machine Learn 57(1\u20132):83\u2013113. http:\/\/ai.stanford.edu\/~ronnyk\/lessonsInDM.pdf","journal-title":"Machine Learn"},{"key":"114_CR25","unstructured":"Koselka R (1996) The new mantra: MVT. Forbes. March 11, 1996, pp 114\u2013118"},{"key":"114_CR26","unstructured":"Linden G (2006a) Early Amazon: shopping cart recommendations. Geeking with Greg. [Online] April 25, 2006. http:\/\/glinden.blogspot.com\/2006\/04\/early-amazon-shopping-cart.html"},{"key":"114_CR27","unstructured":"Linden G (2006b) Make data useful. [Online] Dec 2006. http:\/\/home.blarg.net\/~glinden\/StanfordDataMining.2006-11-29.ppt"},{"key":"114_CR28","volume-title":"Don\u2019t rationalize bad site design","author":"H Manning","year":"2006","unstructured":"Manning H, Dorsey M, Carney CL (2006) Don\u2019t rationalize bad site design. Forrester Research, Cambridge"},{"key":"114_CR29","unstructured":"Marks HM (2000) The progress of experiment: science and therapeutic reform in the united states, 1900\u20131990. Cambridge University Press, ISBN: 978-0521785617"},{"key":"114_CR30","unstructured":"Maron O, Moore AW (1994) Hoeffding races: accelerating model selection search for classification and function approximation. http:\/\/citeseer.ist.psu.edu\/maron94hoeffding.html"},{"key":"114_CR31","unstructured":"Mason RL, Gunst RF, Hess JL (1989) Statistical design and analysis of experiments with applications to engineering and science. Wiley, ISBN: 047185364X"},{"key":"114_CR32","unstructured":"McGlaughlin F et al (2006) The power of small changes tested. Market Exp J. [Online] March 21, 2006. http:\/\/www.marketingexperiments.com\/improving-website-conversion\/power-small-change.html"},{"key":"114_CR33","unstructured":"Miller S (2006) The ConversionLab.com: how to experiment your way to increased web sales using split testing and Taguchi optimization. http:\/\/www.conversionlab.com\/"},{"key":"114_CR34","unstructured":"Miller S (2007) How to design a split test. Web marketing today, conversion\/testing. [Online] Jan 18, 2007. http:\/\/www.wilsonweb.com\/conversion\/"},{"key":"114_CR35","unstructured":"Moran M (2007) Do it wrong quickly: how the web changes the old marketing rules. IBM Press, ISBN: 0132255960"},{"key":"114_CR36","unstructured":"Nielsen J (2005) Putting A\/B testing in its place. Useit.com Alertbox. [Online] Aug 15, 2005. http:\/\/www.useit.com\/alertbox\/20050815.html"},{"key":"114_CR37","unstructured":"Omniture (2008) [Online] 2008. http:\/\/www.omniture.com\/products\/optimization\/offermatica"},{"key":"114_CR38","unstructured":"Optimost (2008) [Online] 2008. http:\/\/www.optimost.com"},{"key":"114_CR39","unstructured":"Peterson ET (2004) Web analytics demystified: a marketer\u2019s guide to understanding how your web site affects your business. Celilo Group Media and CafePress, ISBN: 0974358428"},{"key":"114_CR40","unstructured":"Peterson ET (2005) Web site measurement hacks. O\u2019Reilly Media, ISBN: 0596009887"},{"key":"114_CR41","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1093\/biomet\/33.4.305","volume":"33","author":"RL Plackett","year":"1946","unstructured":"Plackett RL, Burman JP (1946) The design of optimum multifactorial experiments. Biometrika 33: 305\u2013325","journal-title":"Biometrika"},{"key":"114_CR42","unstructured":"Quarto-vonTivadar J (2006) AB testing: too little, too soon. Future Now. [Online] 2006. http:\/\/www.futurenowinc.com\/abtesting.pdf"},{"key":"114_CR43","unstructured":"Rossi PH, Lipsey MW, Freeman HE (2003) Evaluation: a systematic approach, 7th edn. Sage Publications, Inc., ISBN: 0-7619-0894-3"},{"key":"114_CR44","unstructured":"Roy RK (2001) Design of experiments using the taguchi approach: 16 steps to product and process improvement. Wiley, ISBN: 0-471-36101-1"},{"key":"114_CR45","unstructured":"SiteSpect (2008) [Online] 2008. http:\/\/www.sitespect.com"},{"key":"114_CR46","unstructured":"Spool JM (2004) The cost of frustration. WebProNews. [Online] September 20, 2004. http:\/\/www.webpronews.com\/topnews\/2004\/09\/20\/the-cost-of-frustration"},{"key":"114_CR47","unstructured":"Sterne J (2002) Web metrics: proven methods for measuring web site success. Wiley, ISBN: 0-471-22072-8"},{"key":"114_CR48","unstructured":"Tan P-N, Kumar V (2002) Discovery of web robot sessions based on their navigational patterns. Data Min Knowl Dis"},{"key":"114_CR49","unstructured":"Thomke S (2001) Enlightened experimentation: the new imperative for innovation, Feb 2001"},{"key":"114_CR50","unstructured":"Thomke SH (2003) Experimentation matters: unlocking the potential of new technologies for innovation"},{"key":"114_CR51","unstructured":"Tyler ME, Ledford J (2006) Google analytics. Wiley, ISBN: 0470053852"},{"key":"114_CR52","unstructured":"Ulwick A (2005) What customers want: using outcome-driven innovation to create breakthrough products and services. McGraw-Hill, ISBN: 0071408673"},{"key":"114_CR53","unstructured":"Usborne N (2005) Design choices can cripple a website. A list apart. [Online] Nov 8, 2005. http:\/\/alistapart.com\/articles\/designcancripple"},{"key":"114_CR54","unstructured":"van Belle G (2002) Statistical rules of thumb. Wiley, ISBN: 0471402273"},{"key":"114_CR55","unstructured":"Varian HR (2007) Kaizen, that continuous improvement strategy, finds its ideal environment. New York Times. February 8, 2007. Online at http:\/\/www.nytimes.com\/2007\/02\/08\/business\/08scene.html?fta=y"},{"key":"114_CR56","unstructured":"Verster (2008) [Online] 2008. http:\/\/www.vertster.com"},{"key":"114_CR57","unstructured":"Weiss CH (1997) Evaluation: methods for studying programs and policies, 2nd edn. Prentice Hall, ISBN: 0-13-309725-0"},{"key":"114_CR58","unstructured":"Weiss TR (2000) Amazon apologizes for price-testing program that angered customers. http:\/\/www.Safecount.net. [Online] September 28, 2000. http:\/\/www.infoworld.com\/articles\/hn\/xml\/00\/09\/28\/000928hnamazondvd.html"},{"key":"114_CR59","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1080\/00401706.1974.10489174","volume":"16","author":"RE Wheeler","year":"1974","unstructured":"Wheeler RE (1974) Portable power. Technometrics 16:193\u2013201. http:\/\/www.bobwheeler.com\/stat\/Papers\/PortablePower.PDF","journal-title":"Technometrics"},{"issue":"2","key":"114_CR60","doi-asserted-by":"crossref","first-page":"177","DOI":"10.2307\/1268349","volume":"17","author":"RE Wheeler","year":"1975","unstructured":"Wheeler RE (1975) The validity of portable power. Technometrics 17(2):177\u2013179","journal-title":"Technometrics"},{"key":"114_CR61","unstructured":"Widemile (2008) [Online] 2008. http:\/\/www.widemile.com"},{"key":"114_CR62","unstructured":"Wikepedia (2008) Multi-armed bandit. Wikipedia. [Online] 2008. http:\/\/en.wikipedia.org\/wiki\/Multi-armed_bandit"},{"key":"114_CR63","doi-asserted-by":"crossref","unstructured":"Willan AR, Briggs AH (2006) Statistical analysis of cost-effectiveness data (statistics in practice). Wiley","DOI":"10.1002\/0470856289"}],"container-title":["Data Mining and Knowledge Discovery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-008-0114-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10618-008-0114-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-008-0114-1.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-008-0114-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,31]],"date-time":"2021-08-31T19:42:16Z","timestamp":1630438936000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10618-008-0114-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,7,30]]},"references-count":63,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2009,2]]}},"alternative-id":["114"],"URL":"https:\/\/doi.org\/10.1007\/s10618-008-0114-1","relation":{},"ISSN":["1384-5810","1573-756X"],"issn-type":[{"value":"1384-5810","type":"print"},{"value":"1573-756X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2008,7,30]]},"assertion":[{"value":"14 February 2008","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 June 2008","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 July 2008","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}