{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T09:53:12Z","timestamp":1777715592944,"version":"3.51.4"},"reference-count":33,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[1998,4,1]],"date-time":"1998-04-01T00:00:00Z","timestamp":891388800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[1998,4]]},"abstract":"<jats:p>The days of specifying missions for mobile robots using traditional programming languages such as C++ and LISP are coming to an end. The need to support operators lacking programming skills coupled with the increasing diversity of robot run-time operating systems is moving the field toward high-level robot programming toolsets that allow graphical mission specification. This paper ex plores the issues of evaluating such toolsets as to their usability. We first examine how usability criteria are established and perfor mance target values are chosen. The methods by which suitable experiments are created to gather data relevant to the usability criteria are then presented. Finally, methods to analyze the data gathered to establish values for the usability criteria are discussed. The MissionLab toolset is used as a concrete example throughout the article to ground the discussions, but the methods and techniques are generalizable to many such systems.<\/jats:p>","DOI":"10.1177\/027836499801700405","type":"journal-article","created":{"date-parts":[[2007,3,11]],"date-time":"2007-03-11T03:05:13Z","timestamp":1173582313000},"page":"381-401","source":"Crossref","is-referenced-by-count":35,"title":["Evaluating the Usability of Robot Programming Toolsets"],"prefix":"10.1177","volume":"17","author":[{"given":"Douglas C.","family":"MacKenzie","sequence":"first","affiliation":[{"name":"Mobile Intelligence Corp., and Georgia Institute of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ronald C.","family":"Arkin","sequence":"additional","affiliation":[{"name":"College of Computing Georgia Institute of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[1998,4,1]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1177\/027836498900800406"},{"key":"atypb2","doi-asserted-by":"publisher","DOI":"10.1080\/095281397147068"},{"key":"atypb3","doi-asserted-by":"publisher","DOI":"10.1109\/70.294203"},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1994.tb00594.x"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1989.1.2.253"},{"key":"atypb6","volume-title":"MissionLab User Manual","author":"Cameron, J.M.","year":"1996"},{"key":"atypb7","volume-title":"Proc. CHI'88, Human Factors in Computing Systems","author":"Chin, J.P."},{"key":"atypb8","volume-title":"A colony architecture for an artificial creature. AI technical report 1151","author":"Connell, J.","year":"1989"},{"key":"atypb9","volume-title":"Adaptive execution in complex dynamic worlds. Computer science technical report YALEU\/CSD\/RR 672","author":"Firby, J.","year":"1989"},{"key":"atypb10","volume-title":"The GUI Style Guide","author":"Fowler, Susan L.","year":"1995"},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.1080\/10798587.1995.10750620"},{"key":"atypb12","volume-title":"SAUSAGES Users Manual","author":"Gowdy, J.","year":"1991"},{"key":"atypb13","volume-title":"SAUSAGES: Between planning and action. Technical report draft","author":"Gowdy, J.","year":"1994"},{"key":"atypb14","volume-title":"Developing User Interfaces","author":"Hix, Deborah","year":"1993"},{"key":"atypb15","volume-title":"The Icon Book","author":"Horton, William.","year":"1994"},{"key":"atypb16","volume-title":"Proc. AIAA\/NASA Conf. on Intelligent Robots in Field, Factory, Service, and Space (CIRFFSS '94","author":"Lee, Jaeho"},{"key":"atypb17","volume-title":"Proc. CHI'85, Human Factors in Computing Systems","author":"Lund, M.A."},{"key":"atypb18","doi-asserted-by":"publisher","DOI":"10.1109\/70.34764"},{"key":"atypb19","volume-title":"Configuration Network Language (CNL) User Manual","author":"MacKenzie, D.C.","year":"1996"},{"key":"atypb20","volume-title":"A design methodology for the configuration of behavior-based mobile robots","author":"MacKenzie, D.C.","year":"1997"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008807102993"},{"key":"atypb22","volume-title":"Proc. from Animals to Animation, Second Int. Conf. on Simulation of Adaptive Behavior (SAB92","author":"Mataric, M.J."},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1109\/52.156902"},{"key":"atypb24","volume-title":"The Society of Mind","author":"Minsky, M.","year":"1986"},{"key":"atypb25","volume-title":"MissionLab Toolset","year":"1996"},{"key":"atypb26","doi-asserted-by":"publisher","DOI":"10.1145\/142750.142834"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1993.tb00543.x"},{"key":"atypb28","unstructured":"Rosenblatt, J.K. 1995. Damn: A distributed architecture for mobile navigation. AAAI Spring Symposium: Lessons Learned from Implementing Software Architectures for Physical Agents Menlo Park, CA: AAAI Press, pp. 167-178."},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.1989.118717"},{"key":"atypb30","volume-title":"Proc. Intelligent Robotics and Systems (IROS 95","author":"Stewart, D.B."},{"key":"atypb31","volume-title":"Proc. IEEE Int. Conf. on Systems, Man, and Cybernetics","author":"Tumell, M."},{"key":"atypb32","volume-title":"Proc. CHI '96: Common Ground\u2014Conference on Human Factors in Computing Systems","author":"Virzi, R.A."},{"key":"atypb33","volume-title":"Guidelines for Enterprise-Wide GUI Design","author":"Weinschenk, S.","year":"1995"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/027836499801700405","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/027836499801700405","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:15:35Z","timestamp":1777457735000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/027836499801700405"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1998,4]]},"references-count":33,"journal-issue":{"issue":"4","published-print":{"date-parts":[[1998,4]]}},"alternative-id":["10.1177\/027836499801700405"],"URL":"https:\/\/doi.org\/10.1177\/027836499801700405","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[1998,4]]}}}