{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T01:01:37Z","timestamp":1792803697465,"version":"4.5.0"},"reference-count":147,"publisher":"Emerald","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2013,6,26]]},"abstract":"<jats:p>Experiments help to understand human\u2013computer interaction and to characterize the value of user interfaces. Yet, few intermediate guidelines exist on how to design, run, and report experiments. The present monograph presents such guidelines. We briefly argue why experiments are invaluable for advancing human\u2013computer interaction beyond technical innovation.We then identify heuristics of doing good experiments, including how to build on existing work in devising hypotheses and selecting measures; how to craft challenging comparisons, rather than biased win\u2013lose setups; how to design experiments so as to rule out alternative explanations; how to provide evidence for conclusions; and how to narrate findings. These heuristics are exemplified by excellent experiments in human\u2013computer interaction.<\/jats:p>","DOI":"10.1561\/1100000043","type":"journal-article","created":{"date-parts":[[2013,6,26]],"date-time":"2013-06-26T10:07:55Z","timestamp":1372241275000},"page":"299-373","source":"Crossref","is-referenced-by-count":41,"title":["Some Whys and Hows of Experiments in Human\u2014Computer Interaction"],"prefix":"10.1108","volume":"5","author":[{"given":"Kasper","family":"Hornb\u00e6k","sequence":"first","affiliation":[{"name":"University of Copenhagen Department of Computer Science, , DK-2300, Copenhagen S,","place":["Denmark"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"140","published-online":{"date-parts":[[2013,6,26]]},"reference":[{"key":"2026040213265217000_ref001","volume-title":"Statistics as Principled Argument","author":"Abelson","year":"1995"},{"key":"2026040213265217000_ref002","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1145\/258549.258760","article-title":"Beyond Fitts\u2019 law: models for trajectory-based HCI tasks","author":"Accot","year":"1997","journal-title":"Proceedings of the ACM SIGCHI Conference on Human factors in computing systems"},{"key":"2026040213265217000_ref003","doi-asserted-by":"crossref","first-page":"1197","DOI":"10.1145\/1357054.1357241","article-title":"Large scale analysis of web revisitation patterns","author":"Adar","year":"2008","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"3","key":"2026040213265217000_ref004","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1017\/S0003055401003100","article-title":"Measurement Validity: A shared Standard for Qualitative and Quantitative Research","volume":"95","author":"Adcock","year":"2001","journal-title":"American Political Science Review"},{"issue":"5","key":"2026040213265217000_ref005","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1080\/014492998119328","article-title":"Viewing personal history records: A comparison of tabular format and graphical presentation using LifeLines","volume":"17","author":"Alonso","year":"1998","journal-title":"Behaviour & Information Technology"},{"issue":"1","key":"2026040213265217000_ref006","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1111\/1467-8721.00002","article-title":"Research in the Psychological Laboratory: Truth or Triviality?","volume":"8","author":"Anderson","year":"1999","journal-title":"Current Directions in Psychological Science"},{"issue":"2","key":"2026040213265217000_ref007","doi-asserted-by":"crossref","first-page":"266","DOI":"10.7326\/0003-4819-108-2-266","article-title":"Guidelines for statistical reporting in articles for medical journals","volume":"108","author":"Bailar","year":"1998","journal-title":"Annals of Internal Medicine"},{"key":"2026040213265217000_ref008","first-page":"138","volume-title":"Handbook of research methods in social and personality psychology","author":"Bakeman","year":"2000"},{"key":"2026040213265217000_ref009","doi-asserted-by":"crossref","first-page":"2689","DOI":"10.1145\/1978942.1979336","article-title":"Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience","author":"Bargas-Avila","year":"2011","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref010","article-title":"From Mice to Men \u2014 24 Years of Evaluation in CHI","author":"Barkhuus","year":"2007","journal-title":"Paper presented at alt.chi"},{"key":"2026040213265217000_ref011","volume-title":"Simple & Direct","author":"Barzun","year":"2001"},{"issue":"3","key":"2026040213265217000_ref012","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1037\/0003-066X.37.3.245","article-title":"External validity is more than skin deep: Some answers to criticisms of laboratory experiments","volume":"37","author":"Berkowitz","year":"1982","journal-title":"American psychologist"},{"issue":"2","key":"2026040213265217000_ref013","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/BF00402715","article-title":"Measuring usability as quality of use","volume":"4","author":"Bevan","year":"1995","journal-title":"Software Quality Journal"},{"key":"2026040213265217000_ref014","volume-title":"Research methods for Human Computer Interaction","author":"Blandford","year":"2008"},{"key":"2026040213265217000_ref015","volume-title":"Causality and modern science","author":"Bunge","year":"1979","edition":"3rd ed."},{"issue":"5","key":"2026040213265217000_ref016","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.intcom.2011.06.004","article-title":"Gender pluralism in problem-solving software","volume":"23","author":"Burnett","year":"2011","journal-title":"Interacting with Computers"},{"key":"2026040213265217000_ref017","first-page":"195","article-title":"HCI... not as it should be: inferential statistics in HCI research","author":"Cairns","year":"2007","journal-title":"Proceedings of the 21st British HCI Group Annual Conference on People and Computers"},{"key":"2026040213265217000_ref018","first-page":"269","volume-title":"Generalizing from laboratory to field settings: Research findings from industrial-organizational psychology, organizational behavior, and human resource management","author":"Campbell","year":"1986"},{"key":"2026040213265217000_ref019","volume-title":"What to study: Generating and developing research questions","author":"Campbell","year":"1982"},{"issue":"8","key":"2026040213265217000_ref020","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1080\/00140137808931762","article-title":"Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys for text selection on a CRT","volume":"21","author":"Card","year":"1978","journal-title":"Ergonomics"},{"key":"2026040213265217000_ref021","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1007\/978-3-540-70956-5_2","volume-title":"Information Visualization: Human-Centered Issues and Perspectives","author":"Carpendale","year":"2008"},{"issue":"366","key":"2026040213265217000_ref022","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1126\/science.ns-15.366.92","article-title":"The method of multiple working hypotheses","volume":"15","author":"Chamberlin","year":"1890","journal-title":"Science"},{"key":"2026040213265217000_ref023","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1145\/57167.57203","article-title":"Development of an instrument measuring user satisfaction of the human-computer interface","author":"Chin","year":"1988","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref024","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1145\/1240624.1240723","article-title":"A predictive model of menu performance","author":"Cockburn","year":"2007","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref025","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1145\/503376.503413","article-title":"Evaluating the effectiveness of spatial memory in 2D and 3D physical and virtual environments","author":"Cockburn","year":"2002","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref026","volume-title":"Statistical power analysis for the behavioral sciences","author":"Cohen","year":"1988"},{"issue":"1","key":"2026040213265217000_ref027","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1037\/0033-2909.112.1.155","article-title":"A power primer","volume":"112","author":"Cohen","year":"1992","journal-title":"Psychological bulletin"},{"key":"2026040213265217000_ref028","volume-title":"Synthesizing research: A guide for literature reviews","author":"Cooper","year":"1998","edition":"third ed."},{"key":"2026040213265217000_ref029","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1037\/0003-066X.60.2.170","article-title":"Inference by eye: Confidence intervals and how to read pictures of data","volume":"60","author":"Cumming","year":"2005","journal-title":"American Psychologist"},{"issue":"2","key":"2026040213265217000_ref030","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1177\/004839317100100211","article-title":"That\u2019s interesting","volume":"1","author":"Davis","year":"1971","journal-title":"Philosophy of the Social Sciences"},{"key":"2026040213265217000_ref031","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1145\/302979.303042","article-title":"Testing pointing device performance and user assessment with the ISO 9241, Part 9 standard","author":"Douglas","year":"1999","journal-title":"Proceedings of the SIGCHI conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref032","doi-asserted-by":"crossref","first-page":"2009","DOI":"10.1145\/1978942.1979233","article-title":"Temporal distortion for animated transitions","author":"Dragicevic","year":"2011","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"3","key":"2026040213265217000_ref033","doi-asserted-by":"crossref","first-page":"86","DOI":"10.4018\/jmhci.2009070108","article-title":"Paper Rejected (p &gt; 0.05): An Introduction to the Debate on Appropriateness of Null-Hypothesis Testing","volume":"1","author":"Dunlop","year":"2009","journal-title":"International Journal of Mobile Human Computer Interaction (IJMHCI)"},{"key":"2026040213265217000_ref034","doi-asserted-by":"crossref","first-page":"1150","DOI":"10.1002\/0470048204.ch44","volume-title":"Handbook of Human Factors and Ergonomics","author":"Edwards","year":"2006"},{"issue":"1","key":"2026040213265217000_ref035","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1145\/64789.64790","article-title":"Formative design evaluation of superbook","volume":"7","author":"Egan","year":"1989","journal-title":"ACM Transactions on Information Systems"},{"issue":"3","key":"2026040213265217000_ref036","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1007\/s11192-011-0494-7","article-title":"Negative results are disappearing from most disciplines and countries","volume":"90","author":"Fanelli","year":"2012","journal-title":"Scientometrics"},{"key":"2026040213265217000_ref037","doi-asserted-by":"crossref","DOI":"10.1515\/9780691190433","volume-title":"Seven rules for social research","author":"Firebaugh","year":"2008"},{"key":"2026040213265217000_ref038","volume-title":"Statistical methods for rates and proportions","author":"Fleiss","year":"1981","edition":"second ed."},{"key":"2026040213265217000_ref039","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1145\/1240624.1240726","article-title":"Direct-touch vs. mouse input for tabletop displays","author":"Forlines","year":"2007","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref040","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1145\/1518701.1518708","article-title":"Non-universal usability?: a survey of how usability is understood by Chinese and Danish users","author":"Frandsen-Thorlacius","year":"2009","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref041","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1145\/332040.332455","article-title":"Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?","author":"Fr\u00f8kj\u00e6r","year":"2000","journal-title":"Proceedings of the SIGCHI conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref042","volume-title":"Handbook of Industrial and Organizational Psychology","author":"Fromkin","year":"1976"},{"issue":"4","key":"2026040213265217000_ref043","doi-asserted-by":"crossref","first-page":"475","DOI":"10.2307\/1423524","article-title":"Satisficing in hypothesis generation","volume":"115","author":"Garst","year":"2002","journal-title":"The American journal of psychology"},{"key":"2026040213265217000_ref044","doi-asserted-by":"crossref","DOI":"10.1201\/9781420064506","volume-title":"Human performance, workload, and situational awareness measures handbook","author":"Gawron","year":"2008"},{"key":"2026040213265217000_ref045","volume-title":"Field experiments: Design, analysis, and interpretation","author":"Gerber","year":"2012"},{"issue":"6","key":"2026040213265217000_ref046","doi-asserted-by":"crossref","first-page":"562","DOI":"10.1177\/1745691612457576","article-title":"Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science","volume":"7","author":"Giner-Sorolla","year":"2012","journal-title":"Perspectives on Psychological Science"},{"issue":"6","key":"2026040213265217000_ref047","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1145\/990680.990686","article-title":"An analysis of research in computing disciplines","volume":"47","author":"Glass","year":"2004","journal-title":"Communications of the ACM"},{"issue":"4","key":"2026040213265217000_ref048","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1145\/2163.358100","article-title":"Composing letters with a simulated listening typewriter","volume":"26","author":"Gould","year":"1983","journal-title":"Communications of the ACM"},{"issue":"3","key":"2026040213265217000_ref049","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1207\/s15327051hci1303_2","article-title":"Damaged merchandise? A review of experiments that compare usability evaluation methods","volume":"13","author":"Gray","year":"1998","journal-title":"Human\u2013Computer Interaction"},{"key":"2026040213265217000_ref050","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1145\/1357054.1357074","article-title":"Usability evaluation considered harmful (some of the time)","author":"Greenberg","year":"2008","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"2","key":"2026040213265217000_ref051","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1037\/0033-2909.83.2.314","article-title":"Within-subjects designs: To use or not to use?","volume":"83","author":"Greenwald","year":"1976","journal-title":"Psychological Bulletin"},{"issue":"2","key":"2026040213265217000_ref052","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1037\/0033-295X.93.2.216","article-title":"Under what conditions does theory obstruct research progress?","volume":"93","author":"Greenwald","year":"1986","journal-title":"Psychological review"},{"key":"2026040213265217000_ref053","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1145\/1054972.1055012","article-title":"The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor\u2019s activation area","author":"Grossman","year":"2005","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref054","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1145\/1518701.1518803","article-title":"A survey of software learnability: metrics, methodologies and guidelines","author":"Grossman","year":"2009","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref055","doi-asserted-by":"crossref","first-page":"1253","DOI":"10.1145\/1124772.1124961","article-title":"The validity of the stimulated retrospective think-aloud method as measured by eye tracking","author":"Guan","year":"2006","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref056","first-page":"511","article-title":"Effects of awareness support on groupware usability","author":"Gutwin","year":"1998","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref057","volume-title":"Designing and reporting experiments in psychology","author":"Harris","year":"2008"},{"key":"2026040213265217000_ref058","first-page":"139","volume-title":"Human mental workload","author":"Hart","year":"1988"},{"issue":"1","key":"2026040213265217000_ref059","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/978-3-031-02191-6","article-title":"Experience Design: Technology for all the right reasons","volume":"3","author":"Hassenzahl","year":"2010","journal-title":"Synthesis Lectures on Human-Centered Informatics"},{"key":"2026040213265217000_ref060","first-page":"187","article-title":"AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualit\u00e4t","author":"Hassenzahl","year":"2003","journal-title":"Mensch & computer"},{"issue":"3","key":"2026040213265217000_ref061","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1080\/07370024.2010.500139","article-title":"The inference of perceived usability from beauty","volume":"25","author":"Hassenzahl","year":"2010","journal-title":"Human\u2013Computer Interaction"},{"issue":"2-3","key":"2026040213265217000_ref062","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1017\/S0140525X0999152X","article-title":"The weirdest people in the world","volume":"33","author":"Henrich","year":"2010","journal-title":"Behavioral and Brain Sciences"},{"key":"2026040213265217000_ref063","doi-asserted-by":"crossref","first-page":"2659","DOI":"10.1145\/2207676.2208658","article-title":"Observational and experimental investigation of typing behaviour using virtual keyboards for mobile devices","author":"Henze","year":"2012","journal-title":"Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems"},{"issue":"2","key":"2026040213265217000_ref064","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1145\/230562.230570","article-title":"Browsing and querying in online documentation: a study of user interfaces and the interaction process","volume":"3","author":"Hertzum","year":"1996","journal-title":"ACM Transactions on Computer-Human Interaction"},{"issue":"4","key":"2026040213265217000_ref065","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1145\/371578.371593","article-title":"Extracting usability information from user interface events","volume":"32","author":"Hilbert","year":"2000","journal-title":"ACM Computing Surveys"},{"key":"2026040213265217000_ref066","doi-asserted-by":"crossref","first-page":"717","DOI":"10.1145\/371920.372188","article-title":"WebQuilt: a framework for capturing and visualizing the web experience","author":"Hong","year":"2001","journal-title":"Proceedings of the 10th international conference on World Wide Web"},{"issue":"2","key":"2026040213265217000_ref067","first-page":"79","article-title":"Current practice in measuring usability: Challenges to usability studies and research","volume":"64","author":"Hornb\u00e6k","year":"2006","journal-title":"International Journal of Man-Machine Studies"},{"issue":"1","key":"2026040213265217000_ref068","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1080\/01449290801939400","article-title":"Dogmas in the assessment of usability evaluation methods","volume":"29","author":"Hornb\u00e6k","year":"2010","journal-title":"Behaviour & Information Technology"},{"issue":"2","key":"2026040213265217000_ref069","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1145\/772047.772050","article-title":"Reading patterns and usability in visualizations of electronic documents","volume":"10","author":"Hornb\u00e6k","year":"2003","journal-title":"ACM Transactions on Computer-Human Interaction"},{"key":"2026040213265217000_ref070","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1145\/1240624.1240722","article-title":"Meta-analysis of correlations among usability measures","author":"Hornb\u00e6k","year":"2007","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"5","key":"2026040213265217000_ref071","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1145\/1735223.1735255","article-title":"Number of people required for usability evaluation: the 10\u00b12 rule","volume":"53","author":"Hwang","year":"2010","journal-title":"Communications of the ACM"},{"key":"2026040213265217000_ref072","volume-title":"ISO 9241-11: Ergonomic requirements for office work with visual display terminals (VDTs) \u2013 Part 11: Guidance on usability","author":"ISO","year":"1998"},{"key":"2026040213265217000_ref073","first-page":"377","article-title":"Evaluating a fisheye view of source code","author":"Jakobsen","year":"2006","journal-title":"Proceedings of the 2006 Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref074","doi-asserted-by":"crossref","first-page":"2593","DOI":"10.1145\/2470654.2481359","article-title":"Evaluating the Efficiency of Physical Visualizations","author":"Jansen","year":"2013","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"2","key":"2026040213265217000_ref075","doi-asserted-by":"crossref","first-page":"141","DOI":"10.2307\/249115","article-title":"Methodological issues in experimental IS research: experiences and recommendations","volume":"9","author":"Jarvenpaa","year":"1985","journal-title":"MIS quarterly"},{"key":"2026040213265217000_ref076","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1145\/108844.108862","article-title":"User interface evaluation in the real world: a comparison of four techniques","author":"Jeffries","year":"1991","journal-title":"Proceedings of the SIGCHI conference on Human factors in computing systems"},{"issue":"4","key":"2026040213265217000_ref077","first-page":"1","article-title":"Evidence-based practice in human-computer interaction and evidence maps","volume":"30","author":"John","year":"2005","journal-title":"ACM SIGSOFT Software Engineering Notes"},{"issue":"4-5","key":"2026040213265217000_ref078","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1080\/014492997119789","article-title":"Tracking the effectiveness of usability evaluation methods","volume":"16","author":"John","year":"1997","journal-title":"Behaviour & Information Technology"},{"key":"2026040213265217000_ref079","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1145\/2207676.2208557","article-title":"Rethinking statistical analysis methods for CHI","author":"Kaptein","year":"2012","journal-title":"Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref080","doi-asserted-by":"crossref","first-page":"2391","DOI":"10.1145\/1753326.1753686","article-title":"Powerful and consistent analysis of likert-type ratingscales","author":"Kaptein","year":"2010","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"1-2","key":"2026040213265217000_ref081","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/1500000012","article-title":"Methods for evaluating interactive information retrieval systems with users","volume":"3","author":"Kelly","year":"2009","journal-title":"Foundations and Trends in Information Retrieval"},{"key":"2026040213265217000_ref082","volume-title":"A Handbook for Data Analysis in the Behaviorial Sciences","author":"Keren","year":"1992"},{"issue":"3","key":"2026040213265217000_ref083","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1207\/s15327957pspr0203_4","article-title":"HARKing: Hypothesizing after the results are known","volume":"2","author":"Kerr","year":"1998","journal-title":"Personality and Social Psychology Review"},{"key":"2026040213265217000_ref084","first-page":"529","article-title":"Is it worth the hassle? Exploring the added value of evaluating the usability of context-aware mobile systems in the field","author":"Kjeldskov","year":"2004","journal-title":"Mobile Human-Computer Interaction"},{"key":"2026040213265217000_ref085","first-page":"21","article-title":"Increasing the utility of quantitative empirical studies for meta-analysis","author":"Lam","year":"2008","journal-title":"Proceedings of the 2008 conference on BEyond time and errors: novel evaLuation methods for Information Visualization"},{"key":"2026040213265217000_ref086","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1016\/B978-044481862-1.50075-3","volume-title":"Handbook of human-computer interaction","author":"Landauer","year":"1997","edition":"second ed."},{"key":"2026040213265217000_ref087","volume-title":"Technology in Mental Health Care Delivery Systems","author":"Lang","year":"1980"},{"key":"2026040213265217000_ref088","first-page":"25","article-title":"Web page design: implications of memory, structure and scent for information retrieval","author":"Larson","year":"1998","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"3","key":"2026040213265217000_ref089","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1016\/j.ijhcs.2003.09.002","article-title":"Assessing dimensions of perceived visual aesthetics of web sites","volume":"60","author":"Lavie","year":"2004","journal-title":"International Journal of Human-Computer Studies"},{"key":"2026040213265217000_ref090","volume-title":"Research methods in human-computer interaction","author":"Lazar","year":"2010"},{"issue":"1","key":"2026040213265217000_ref091","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1080\/10447319509526110","article-title":"IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use","volume":"7","author":"Lewis","year":"1995","journal-title":"International Journal of Human-Computer Interaction"},{"issue":"4","key":"2026040213265217000_ref092","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1111\/j.1442-9993.2009.01946.x","article-title":"Misprescription and misuse of one-tailed tests","volume":"34","author":"Lombardi","year":"2009","journal-title":"Austral Ecology"},{"issue":"1","key":"2026040213265217000_ref093","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1146\/annurev.psych.49.1.259","article-title":"Biases in the interpretation and use of research results","volume":"49","author":"MacCoun","year":"1998","journal-title":"Annual review of psychology"},{"key":"2026040213265217000_ref094","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1145\/1240624.1240762","article-title":"Studying antecedents of emotional experiences in interactive contexts","author":"Mahlke","year":"2007","journal-title":"Proceedings of the SIGCHI conference on Human factors in computing systems"},{"key":"2026040213265217000_ref095","volume-title":"Doing psychology experiments","author":"Martin","year":"2007"},{"key":"2026040213265217000_ref096","first-page":"152","volume-title":"Human-Computer Interaction: Toward the Year 2000","author":"McGrath","year":"1995"},{"issue":"1","key":"2026040213265217000_ref097","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1146\/annurev.psych.48.1.1","article-title":"Creative hypothesis generating in psychology: Some useful heuristics","volume":"48","author":"McGuire","year":"1997","journal-title":"Annual Review of Psychology"},{"key":"2026040213265217000_ref098","volume-title":"Conceptual aspects of human factors","author":"Meister","year":"1989"},{"issue":"1","key":"2026040213265217000_ref099","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1080\/01449290310001638496","article-title":"Why are mobile phones annoying?","volume":"23","author":"Monk","year":"2004","journal-title":"Behaviour & Information Technology"},{"issue":"4","key":"2026040213265217000_ref100","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1037\/0003-066X.38.4.379","article-title":"In defense of external invalidity","volume":"38","author":"Mook","year":"1983","journal-title":"American psychologist"},{"issue":"10","key":"2026040213265217000_ref101","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1016\/j.ijhcs.2010.05.006","article-title":"Facets of visual aesthetics","volume":"68","author":"Moshagen","year":"2010","journal-title":"International Journal of Human-Computer Studies"},{"key":"2026040213265217000_ref102","first-page":"561","article-title":"Exertion interfaces: sports over a distance for social bonding and fun","author":"Mueller","year":"2003","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref103","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1007\/978-3-540-70956-5_6","volume-title":"Information Visualization: Human-Centered Issues and Perspectives","author":"Munzner","year":"2008"},{"key":"2026040213265217000_ref104","first-page":"228","article-title":"Can computer personalities be human personalities?","author":"Nass","year":"1995","journal-title":"Conference Companion on Human Factors in Computing Systems"},{"key":"2026040213265217000_ref105","first-page":"605","article-title":"Towards a methodology employing critical parameters to deliver performance improvements in interactive systems","author":"Newman","year":"1999","journal-title":"Proceedings IFIP TC13 Seventh International Conference on Human-Computer Interaction"},{"key":"2026040213265217000_ref106","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1145\/191666.191766","article-title":"A preliminary analysis of the products of HCI research, using pro forma abstracts","author":"Newman","year":"1994","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"2","key":"2026040213265217000_ref107","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1037\/1082-989X.5.2.241","article-title":"Null hypothesis significance testing: a review of an old and continuing controversy","volume":"5","author":"Nickerson","year":"2000","journal-title":"Psychological methods"},{"key":"2026040213265217000_ref108","doi-asserted-by":"crossref","DOI":"10.1016\/B978-0-08-052029-2.50007-3","volume-title":"Usability engineering","author":"Nielsen","year":"1993"},{"key":"2026040213265217000_ref109","first-page":"335","article-title":"A comparison of reading paper and on-line documents","author":"O\u2019Hara","year":"1997","journal-title":"Proceedings of the SIGCHI conference on Human factors in computing systems"},{"issue":"4","key":"2026040213265217000_ref110","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1145\/159764.159763","article-title":"Groupwork close up: a comparison of the group design process with and without a simple group editor","volume":"11","author":"Olson","year":"1993","journal-title":"ACM Transactions on Information Systems"},{"issue":"4","key":"2026040213265217000_ref111","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1109\/MPRV.2011.84","article-title":"Fielding Usability Evaluation","volume":"11","author":"Oulasvirta","year":"2012","journal-title":"IEEE Pervasive Computing"},{"issue":"3","key":"2026040213265217000_ref112","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.ijhcs.2010.11.003","article-title":"What does it mean to be good at using a mobile device? An investigation of three levels of experience and skill","volume":"69","author":"Oulasvirta","year":"2011","journal-title":"International Journal of Human-computer Studies"},{"key":"2026040213265217000_ref113","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1145\/985692.985784","article-title":"WaveLens: a new view onto Internet search results","author":"Paek","year":"2004","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"5","key":"2026040213265217000_ref114","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1017\/S1930297500002205","article-title":"Running experiments on amazon mechanical turk","volume":"5","author":"Paolacci","year":"2010","journal-title":"Judgment and Decision Making"},{"issue":"2","key":"2026040213265217000_ref115","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1109\/TSE.1986.6312940","article-title":"A rational design process: How and why to fake it","author":"Parnas","year":"1986","journal-title":"IEEE Transactions on Software Engineering"},{"key":"2026040213265217000_ref116","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1145\/503376.503428","article-title":"Gestural and audio metaphors as a means of control for mobile devices","author":"Pirhonen","year":"2002","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"3642","key":"2026040213265217000_ref117","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1126\/science.146.3642.347","article-title":"Strong inference","volume":"146","author":"Platt","year":"1964","journal-title":"Science"},{"issue":"4","key":"2026040213265217000_ref118","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1145\/2163.2164","article-title":"The evaluation of text editors: methodology and empirical results","volume":"26","author":"Roberts","year":"1983","journal-title":"Communications of the ACM"},{"key":"2026040213265217000_ref119","volume-title":"Real world research: A resource for social scientists and practitioner-researchers","author":"Robson","year":"2002"},{"key":"2026040213265217000_ref120","volume-title":"The volunteer subject","author":"Rosenthal","year":"1975"},{"key":"2026040213265217000_ref121","volume-title":"Contrast analysis: Focused comparisons in the analysis of variance","author":"Rosenthal","year":"1985"},{"key":"2026040213265217000_ref122","volume-title":"Essentials of behavioral research: Methods and data analysis","author":"Rosenthal","year":"1991"},{"issue":"4","key":"2026040213265217000_ref123","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1207\/s15327051hci0903&4_2","article-title":"Exploratory sequential data analysis: foundations","volume":"9","author":"Sanderson","year":"1994","journal-title":"Human-computer Interaction"},{"key":"2026040213265217000_ref124","doi-asserted-by":"crossref","first-page":"1609","DOI":"10.1145\/1518701.1518947","article-title":"Correlations among prototypical usability metrics: evidence for the construct of usability","author":"Sauro","year":"2009","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"1","key":"2026040213265217000_ref125","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1145\/174630.174632","article-title":"Split menus: effectively using selection frequency to organize menus","volume":"1","author":"Sears","year":"1994","journal-title":"ACM Transactions on Computer-Human Interaction"},{"issue":"3","key":"2026040213265217000_ref126","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1037\/0022-3514.51.3.515","article-title":"College sophomores in the laboratory: Influences of a narrow data base on social psychology\u2019s view of human nature","volume":"51","author":"Sears","year":"1986","journal-title":"Journal of Personality and Social Psychology"},{"key":"2026040213265217000_ref127","volume-title":"Experimental and quasiexperimental designs for generalized causal inference","author":"Shadish","year":"2002"},{"key":"2026040213265217000_ref128","volume-title":"Designing the user interface","author":"Shneiderman","year":"1987"},{"key":"2026040213265217000_ref129","first-page":"1","article-title":"Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies","author":"Shneiderman","year":"2006","journal-title":"Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization"},{"issue":"11","key":"2026040213265217000_ref130","doi-asserted-by":"crossref","first-page":"1359","DOI":"10.1177\/0956797611417632","article-title":"False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant","volume":"22","author":"Simmons","year":"2011","journal-title":"Psychological Science"},{"issue":"9","key":"2026040213265217000_ref131","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1109\/TSE.2005.97","article-title":"A survey of controlled experiments in software engineering","volume":"31","author":"Sj\u0f08berg","year":"2005","journal-title":"IEEE Transactions on Software Engineering"},{"issue":"2","key":"2026040213265217000_ref132","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1145\/772047.772051","article-title":"Evaluating a scientific collaboratory: Results of a controlled experiment","volume":"10","author":"Sonnenwald","year":"2003","journal-title":"ACM Transactions on Computer-Human Interaction"},{"issue":"6","key":"2026040213265217000_ref133","doi-asserted-by":"crossref","first-page":"751","DOI":"10.1016\/j.ijhcs.2004.09.001","article-title":"Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts\u2019 law research in HCI","volume":"61","author":"Soukoreff","year":"2004","journal-title":"International Journal of Human-Computer Studies"},{"key":"2026040213265217000_ref134","volume-title":"The elements of style","author":"Strunk","year":"1959"},{"issue":"5","key":"2026040213265217000_ref135","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/2.675631","article-title":"Should computer scientists experiment more?","volume":"31","author":"Tichy","year":"1998","journal-title":"Computer"},{"key":"2026040213265217000_ref136","volume-title":"Beautiful evidence","author":"Tufte","year":"2006"},{"key":"2026040213265217000_ref137","volume-title":"Measuring the user experience: collecting, analyzing, and presenting usability metrics","author":"Tullis","year":"2008"},{"key":"2026040213265217000_ref138","volume-title":"Nonreactive measures in the social sciences","author":"Webb","year":"1981"},{"issue":"1","key":"2026040213265217000_ref139","doi-asserted-by":"crossref","first-page":"iii","DOI":"10.2307\/25148621","article-title":"Editor\u2019s comments: the rhetoric of positivism versus interpretivism: a personal view","volume":"28","author":"Weber","year":"2004","journal-title":"MIS quarterly"},{"issue":"2-3","key":"2026040213265217000_ref140","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1207\/S15327051HCI1523_2","article-title":"Let\u2019s stop pushing the envelope and start addressing it: a reference task agenda for HCI","volume":"15","author":"Whittaker","year":"2000","journal-title":"Human\u2013Computer Interaction"},{"issue":"8","key":"2026040213265217000_ref141","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1037\/0003-066X.54.8.594","article-title":"Statistical methods in psychology journals: Guidelines and explanations","volume":"54","author":"Wilkinson","year":"1999","journal-title":"American psychologist"},{"key":"2026040213265217000_ref142","doi-asserted-by":"crossref","DOI":"10.1515\/9780804767835","volume-title":"Building experiments: Testing social theory","author":"Willer","year":"2007"},{"key":"2026040213265217000_ref143","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1145\/1978942.1978963","article-title":"The aligned rank transform for nonparametric factorial analyses using only anova procedures","author":"Wobbrock","year":"2011","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"issue":"6","key":"2026040213265217000_ref144","doi-asserted-by":"crossref","first-page":"1224","DOI":"10.1109\/TVCG.2007.70515","article-title":"Toward a deeper understanding of the role of interaction in information visualization","volume":"13","author":"Yi","year":"2007","journal-title":"IEEE Transactions on Visualization and Computer Graphics"},{"key":"2026040213265217000_ref145","volume-title":"Case study research: Design and methods","author":"Yin","year":"2003","edition":"third ed."},{"issue":"5","key":"2026040213265217000_ref146","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/2.675630","article-title":"Experimental models for validating technology","volume":"31","author":"Zelkowitz","year":"1998","journal-title":"Computer"},{"key":"2026040213265217000_ref147","first-page":"177","article-title":"Human on-line response to target expansion","author":"Zhai","year":"2003","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"}],"container-title":["Foundations and Trends\u00ae in Human\u2013Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/fthci\/article-pdf\/5\/4\/299\/10974941\/1100000043en.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/www.emerald.com\/fthci\/article-pdf\/5\/4\/299\/10974941\/1100000043en.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T14:32:33Z","timestamp":1777473153000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.emerald.com\/fthci\/article\/5\/4\/299\/1324736\/Some-Whys-and-Hows-of-Experiments-in-Human"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,6,26]]},"references-count":147,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2013,6,26]]}},"URL":"https:\/\/doi.org\/10.1561\/1100000043","relation":{},"ISSN":["1551-3955","1551-3963"],"issn-type":[{"value":"1551-3955","type":"print"},{"value":"1551-3963","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,6,26]]}}}