{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,31]],"date-time":"2025-07-31T00:48:09Z","timestamp":1753922889385,"version":"3.37.3"},"reference-count":50,"publisher":"MIT Press","issue":"1","license":[{"start":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T00:00:00Z","timestamp":1698624000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1934792"],"award-info":[{"award-number":["1934792"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Web-based experiments are gaining momentum in motor learning research because of the desire to increase statistical power, decrease overhead for human participant experiments, and utilize a more demographically inclusive sample population. However, there is a vital need to understand the general feasibility and considerations necessary to shift tightly controlled human participant experiments to an online setting. We developed and deployed an online experimental platform modeled after established in-laboratory visuomotor rotation experiments to serve as a case study examining remotely collected data quality for an 80-min experiment. Current online motor learning experiments have thus far not exceeded 60 min, and current online crowdsourced studies have a median duration of approximately 10 min. Thus, the impact of a longer-duration, web-based experiment is unknown. We used our online platform to evaluate perturbation-driven motor adaptation behavior under three rotation sizes (\u00b110\u00b0, \u00b135\u00b0, and \u00b165\u00b0) and two sensory uncertainty conditions. We hypothesized that our results would follow predictions by the relevance estimation hypothesis. Remote execution allowed us to double (n = 49) the typical participant population size from similar studies. Subsequently, we performed an in-depth examination of data quality by analyzing single-trial data quality, participant variability, and potential temporal effects across trials. Results replicated in-laboratory findings and provided insight on the effect of induced sensory uncertainty on the relevance estimation hypothesis. Our experiment also highlighted several specific challenges associated with online data collection including potentially smaller effect sizes, higher data variability, and lower recommended experiment duration thresholds. Overall, online paradigms present both opportunities and challenges for future motor learning research.<\/jats:p>","DOI":"10.1162\/jocn_a_02080","type":"journal-article","created":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T14:32:52Z","timestamp":1698676372000},"page":"71-94","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":6,"title":["A Case Study of the Validity of Web-based Visuomotor Rotation Experiments"],"prefix":"10.1162","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8466-0038","authenticated-orcid":true,"given":"Megan C.","family":"Shyr","sequence":"first","affiliation":[{"name":"University of California, Davis"}]},{"given":"Sanjay S.","family":"Joshi","sequence":"additional","affiliation":[{"name":"University of California, Davis"}]}],"member":"281","published-online":{"date-parts":[[2024,1,1]]},"reference":[{"key":"2023120716074131900_bib1","doi-asserted-by":"publisher","first-page":"920","DOI":"10.1038\/s41562-020-01036-x","article-title":"An implicit memory of errors limits human sensorimotor adaptation","volume":"5","author":"Albert","year":"2021","journal-title":"Nature Human Behaviour"},{"volume-title":"Machine bias","year":"2016","author":"Angwin","key":"2023120716074131900_bib2"},{"key":"2023120716074131900_bib3","doi-asserted-by":"publisher","first-page":"e3001147","DOI":"10.1371\/journal.pbio.3001147","article-title":"Reexposure to a sensorimotor perturbation produces opposite effects on explicit and implicit learning processes","volume":"19","author":"Avraham","year":"2021","journal-title":"PLoS Biology"},{"key":"2023120716074131900_bib145","doi-asserted-by":"publisher","first-page":"1454","DOI":"10.1038\/nn.2229","article-title":"Estimating the sources of motor errors for adaptation and generalization","volume":"11","author":"Berniker","year":"2008","journal-title":"Nature Neuroscience"},{"key":"2023120716074131900_bib146","doi-asserted-by":"publisher","first-page":"e1006501","DOI":"10.1371\/journal.pcbi.1006501","article-title":"Conventional analysis of trial-by-trial adaptation is biased: Empirical and theoretical support using a Bayesian estimator","volume":"14","author":"Blustein","year":"2018","journal-title":"PLoS Computational Biology"},{"key":"2023120716074131900_bib4","doi-asserted-by":"publisher","first-page":"3836","DOI":"10.1152\/jn.00009.2015","article-title":"Flexible explicit but rigid implicit learning in a visuomotor adaptation task","volume":"113","author":"Bond","year":"2015","journal-title":"Journal of Neurophysiology"},{"key":"2023120716074131900_bib5","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1038\/s41539-020-0066-9","article-title":"Mechanisms of offline motor learning at a microscale of seconds in large-scale crowdsourced data","volume":"5","author":"B\u00f6nstrup","year":"2020","journal-title":"NPJ Science of Learning"},{"key":"2023120716074131900_bib6","doi-asserted-by":"publisher","first-page":"20.1","DOI":"10.1167\/8.4.20","article-title":"The statistical determinants of adaptation rate in human reaching","volume":"8","author":"Burge","year":"2008","journal-title":"Journal of Vision"},{"volume-title":"Google's algorithm shows prestigious job ads to men, but not to women","year":"2015","author":"Carpenter","key":"2023120716074131900_bib7"},{"key":"2023120716074131900_bib8","doi-asserted-by":"publisher","first-page":"e71627","DOI":"10.7554\/eLife.71627","article-title":"Motor memories of object dynamics are categorically organized","volume":"10","author":"Cesanek","year":"2021","journal-title":"eLife"},{"key":"2023120716074131900_bib9","doi-asserted-by":"publisher","first-page":"e57410","DOI":"10.1371\/journal.pone.0057410","article-title":"Evaluating Amazon's Mechanical Turk as a tool for experimental behavioral research","volume":"8","author":"Crump","year":"2013","journal-title":"PLoS One"},{"key":"2023120716074131900_bib10","doi-asserted-by":"publisher","first-page":"2603","DOI":"10.1016\/j.jbusres.2015.05.001","article-title":"Swapping bricks for clicks: Crowdsourcing longitudinal data on Amazon Turk","volume":"68","author":"Daly","year":"2015","journal-title":"Journal of Business Research"},{"key":"2023120716074131900_bib148","doi-asserted-by":"publisher","first-page":"1050","DOI":"10.1016\/j.cub.2014.03.049","article-title":"Environmental consistency determines the rate of motor adaptation","volume":"24","author":"Gonzalez Castro","year":"2014","journal-title":"Current Biology"},{"key":"2023120716074131900_bib11","doi-asserted-by":"publisher","first-page":"e1005023","DOI":"10.1371\/journal.pcbi.1005023","article-title":"The statistical determinants of the speed of motor learning","volume":"12","author":"He","year":"2016","journal-title":"PLoS Computational Biology"},{"key":"2023120716074131900_bib12","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1038\/s41586-021-04129-3","article-title":"Contextual inference underlies the learning of sensorimotor repertoires","volume":"600","author":"Heald","year":"2021","journal-title":"Nature"},{"key":"2023120716074131900_bib13","doi-asserted-by":"publisher","first-page":"2883","DOI":"10.1523\/JNEUROSCI.5359-07.2008","article-title":"Motor adaptation as a process of reoptimization","volume":"28","author":"Izawa","year":"2008","journal-title":"Journal of Neuroscience"},{"key":"2023120716074131900_bib149","doi-asserted-by":"publisher","first-page":"e1002012","DOI":"10.1371\/journal.pcbi.1002012","article-title":"Learning from sensory and reward prediction errors during motor adaptation","volume":"7","author":"Izawa","year":"2011","journal-title":"PLoS Computational Biology"},{"key":"2023120716074131900_bib14","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1016\/j.jns.2010.11.021","article-title":"Influence of age and gender in motor performance in healthy subjects","volume":"302","author":"Jim\u00e9nez-Jim\u00e9nez","year":"2011","journal-title":"Journal of the Neurological Sciences"},{"key":"2023120716074131900_bib15","doi-asserted-by":"publisher","first-page":"302","DOI":"10.3389\/fnins.2014.00302","article-title":"Does EMG control lead to distinct motor adaptation?","volume":"8","author":"Johnson","year":"2014","journal-title":"Frontiers in Neuroscience"},{"article-title":"UI events","year":"2022","author":"Kacmarcik","key":"2023120716074131900_bib16"},{"key":"2023120716074131900_bib17","doi-asserted-by":"publisher","first-page":"936","DOI":"10.1038\/s41562-018-0463-8","article-title":"Network constraints on learnability of probabilistic motor sequences","volume":"2","author":"Kahn","year":"2018","journal-title":"Nature Human Behaviour"},{"key":"2023120716074131900_bib18","first-page":"1381","article-title":"\u201cA Game of Thrones\u201d: When human behavior models compete in repeated Stackelberg security games","volume-title":"Proceedings of the 2015 international conference on autonomous agents and multiagent systems","author":"Kar","year":"2015"},{"key":"2023120716074131900_bib19","doi-asserted-by":"publisher","first-page":"e2204379119","DOI":"10.1073\/pnas.2204379119","article-title":"Motor learning without movement","volume":"119","author":"Kim","year":"2022","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2023120716074131900_bib20","doi-asserted-by":"publisher","first-page":"244","DOI":"10.1038\/nature02169","article-title":"Bayesian integration in sensorimotor learning","volume":"427","author":"K\u00f6rding","year":"2004","journal-title":"Nature"},{"key":"2023120716074131900_bib21","doi-asserted-by":"publisher","first-page":"863","DOI":"10.3389\/fpsyg.2013.00863","article-title":"Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t Tests and ANOVAs","volume":"4","author":"Lakens","year":"2013","journal-title":"Frontiers in Psychology"},{"key":"2023120716074131900_bib22","doi-asserted-by":"publisher","first-page":"e0130834","DOI":"10.1371\/journal.pone.0130834","article-title":"\u201cJust Another Tool for Online Studies\u201d (JATOS): An easy solution for setup and management of web servers supporting online studies","volume":"10","author":"Lange","year":"2015","journal-title":"PLoS One"},{"key":"2023120716074131900_bib23","doi-asserted-by":"publisher","first-page":"5178","DOI":"10.1109\/EMBC.2018.8513412","article-title":"Effects of mapping uncertainty on visuomotor adaptation to trial-by-trial perturbations with proportional myoelectric control","volume-title":"2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)","author":"Lyons","year":"2018"},{"key":"2023120716074131900_bib24","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1016\/j.jbef.2017.12.004","article-title":"Prolific.ac\u2014A subject pool for online experiments","volume":"17","author":"Palan","year":"2018","journal-title":"Journal of Behavioral and Experimental Finance"},{"key":"2023120716074131900_bib25","doi-asserted-by":"publisher","first-page":"184","DOI":"10.1177\/0963721414531598","article-title":"Inside the Turk: Understanding Mechanical Turk as a participant pool","volume":"23","author":"Paolacci","year":"2014","journal-title":"Current Directions in Psychological Science"},{"key":"2023120716074131900_bib26","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1016\/j.jesp.2017.01.006","article-title":"Beyond the Turk: Alternative platforms for crowdsourcing behavioral research","volume":"70","author":"Peer","year":"2017","journal-title":"Journal of Experimental Social Psychology"},{"key":"2023120716074131900_bib27","doi-asserted-by":"publisher","first-page":"645","DOI":"10.1093\/geronj\/41.5.645","article-title":"Motor performance of women as a function of age and physical activity level","volume":"41","author":"Rikli","year":"1986","journal-title":"Journal of Gerontology"},{"key":"2023120716074131900_bib28","doi-asserted-by":"publisher","first-page":"1030","DOI":"10.1152\/jn.01151.2005","article-title":"Distinct short-term and long-term adaptation to reduce saccade size in monkey","volume":"96","author":"Robinson","year":"2006","journal-title":"Journal of Neurophysiology"},{"key":"2023120716074131900_bib29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/ICHMS53169.2021.9582639","article-title":"Validation of the Bayesian sensory uncertainty model of motor adaptation with a remote experimental paradigm","volume-title":"2021 IEEE 2nd International Conference on Human\u2013Machine Systems (ICHMS)","author":"Shyr","year":"2021"},{"key":"2023120716074131900_bib30","doi-asserted-by":"publisher","first-page":"736","DOI":"10.1016\/j.tics.2017.06.007","article-title":"Crowdsourcing samples in cognitive science","volume":"21","author":"Stewart","year":"2017","journal-title":"Trends in Cognitive Sciences"},{"key":"2023120716074131900_bib31","doi-asserted-by":"publisher","DOI":"10.1101\/2023.01.18.524634","article-title":"Predictors of sensorimotor adaption: Insights from over 100,000 reaches","author":"Tsay","year":"2023","journal-title":"bioRxiv"},{"key":"2023120716074131900_bib32","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1152\/jn.00493.2020","article-title":"The effect of visual uncertainty on implicit motor adaptation","volume":"125","author":"Tsay","year":"2021","journal-title":"Journal of Neurophysiology"},{"key":"2023120716074131900_bib33","doi-asserted-by":"publisher","first-page":"e1010005","DOI":"10.1371\/journal.pcbi.1010005","article-title":"Interactions between sensory prediction error and task error during implicit motor learning","volume":"18","author":"Tsay","year":"2022","journal-title":"PLoS Computational Biology"},{"key":"2023120716074131900_bib34","doi-asserted-by":"publisher","first-page":"e76639","DOI":"10.7554\/eLife.76639","article-title":"Understanding implicit sensorimotor adaptation as a process of proprioceptive re-alignment","volume":"11","author":"Tsay","year":"2022","journal-title":"eLife"},{"key":"2023120716074131900_bib35","doi-asserted-by":"publisher","first-page":"20220415","DOI":"10.1098\/rspb.2022.0415","article-title":"Dissociable use-dependent processes for volitional goal-directed reaching","volume":"289","author":"Tsay","year":"2022","journal-title":"Proceedings of the Royal Society B: Biological Sciences"},{"key":"2023120716074131900_bib36","doi-asserted-by":"publisher","DOI":"10.6084\/m9.figshare.c.5923824.v1","volume-title":"Supplementary material from \u201cDissociable use-dependent processes for volitional goal-directed reaching.\u201d","author":"Tsay","year":"2022"},{"key":"2023120716074131900_bib37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.51628\/001c.26985","article-title":"Moving outside the lab: The viability of conducting sensorimotor learning studies online","volume":"5","author":"Tsay","year":"2021","journal-title":"Neurons, Behavior, Data Analysis, and Theory"},{"volume-title":"Length of studies","year":"2021","author":"Tulloch","key":"2023120716074131900_bib38"},{"key":"2023120716074131900_bib39","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: Fundamental algorithms for scientific computing in Python","volume":"17","author":"Virtanen","year":"2020","journal-title":"Nature Methods"},{"key":"2023120716074131900_bib40","doi-asserted-by":"publisher","DOI":"10.1101\/2022.09.14.508027","volume-title":"Advanced feedback enhances sensorimotor adaptation","author":"Wang","year":"2022"},{"key":"2023120716074131900_bib41","doi-asserted-by":"publisher","first-page":"655","DOI":"10.1152\/jn.90545.2008","article-title":"Relevance of error: What drives motor adaptation?","volume":"101","author":"Wei","year":"2009","journal-title":"Journal of Neurophysiology"},{"key":"2023120716074131900_bib42","doi-asserted-by":"publisher","first-page":"11","DOI":"10.3389\/fncom.2010.00011","article-title":"Uncertainty of feedback and state estimation determines the speed of motor adaptation","volume":"4","author":"Wei","year":"2010","journal-title":"Frontiers in Computational Neuroscience"},{"key":"2023120716074131900_bib43","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1177\/1745691611406923","article-title":"Statistical evidence in experimental psychology: An empirical comparison using 855 t tests","volume":"6","author":"Wetzels","year":"2011","journal-title":"Perspectives on Psychological Science"},{"key":"2023120716074131900_bib44","doi-asserted-by":"publisher","first-page":"1212","DOI":"10.1038\/81497","article-title":"Computational principles of movement neuroscience","volume":"3","author":"Wolpert","year":"2000","journal-title":"Nature Neuroscience"},{"key":"2023120716074131900_bib45","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1080\/00222895.1983.10735298","article-title":"Rapid visual feedback processing in single-aiming movements","volume":"15","author":"Zelaznik","year":"1983","journal-title":"Journal of Motor Behavior"},{"article-title":"Pointer Lock 2.0","year":"2022","author":"Zolghadr","key":"2023120716074131900_bib46"}],"container-title":["Journal of Cognitive Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/36\/1\/71\/2190758\/jocn_a_02080.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/36\/1\/71\/2190758\/jocn_a_02080.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,7]],"date-time":"2023-12-07T16:09:11Z","timestamp":1701965351000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/jocn\/article\/36\/1\/71\/117927\/A-Case-Study-of-the-Validity-of-Web-based"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,1]]},"references-count":50,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1,1]]},"published-print":{"date-parts":[[2024,1,1]]}},"URL":"https:\/\/doi.org\/10.1162\/jocn_a_02080","relation":{},"ISSN":["0898-929X","1530-8898"],"issn-type":[{"type":"print","value":"0898-929X"},{"type":"electronic","value":"1530-8898"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024,1,1]]}}}