{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,27]],"date-time":"2025-03-27T08:41:41Z","timestamp":1743064901467,"version":"3.40.3"},"publisher-location":"Boston, MA","reference-count":58,"publisher":"Springer US","isbn-type":[{"type":"print","value":"9781461375272"},{"type":"electronic","value":"9781461555292"}],"license":[{"start":{"date-parts":[[1998,1,1]],"date-time":"1998-01-01T00:00:00Z","timestamp":883612800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[1998,1,1]],"date-time":"1998-01-01T00:00:00Z","timestamp":883612800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[1998]]},"DOI":"10.1007\/978-1-4615-5529-2_13","type":"book-chapter","created":{"date-parts":[[2011,9,21]],"date-time":"2011-09-21T01:39:53Z","timestamp":1316569193000},"page":"311-347","source":"Crossref","is-referenced-by-count":1,"title":["Creating Advice-Taking Reinforcement Learners"],"prefix":"10.1007","author":[{"given":"Richard","family":"Maclin","sequence":"first","affiliation":[]},{"given":"Jude W.","family":"Shavlik","sequence":"additional","affiliation":[]}],"member":"297","reference":[{"key":"13_CR1","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1162\/neco.1995.7.4.639","volume":"7","author":"Y Abu-Mostafa","year":"1995","unstructured":"Abu-Mostafa, Y. (1995). Hints. Neural Computation, 7, 639\u2013671.","journal-title":"Neural Computation"},{"key":"13_CR2","first-page":"268","volume-title":"Proceedings of the Sixth National Conference on Artificial Intelligence","author":"P Agre","year":"1987","unstructured":"Agre, P., & Chapman, D. (1987). Pengi: An implementation of a theory of activity. In Proceedings of the Sixth National Conference on Artificial Intelligence, pp. 268\u2013272 Seattle, WA"},{"key":"13_CR3","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/B978-0-934613-41-5.50014-3","volume-title":"Proceedings of the Fourth International Workshop on Machine Learning","author":"C Anderson","year":"1987","unstructured":"Anderson, C. (1987). Strategy learning with multilayer connectionist representations. In Proceedings of the Fourth International Workshop on Machine Learning, pp. 103\u2013114Irvine, CA."},{"key":"13_CR4","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1109\/TSMC.1983.6313077","volume":"13","author":"A Barto","year":"1983","unstructured":"Barto, A., Sutton, R., & Anderson, C. (1983). Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, 13, 834\u2013846.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics"},{"key":"13_CR5","first-page":"539","volume-title":"Learning and Computational Neuroscience","author":"A Barto","year":"1990","unstructured":"Barto, A., Sutton, R., & Watkins, C. (1990). Learning and sequential decision making. In Gabriel, M., & Moore, J. (Eds.), Learning and Computational Neuroscience, pp. 539\u2013602. MIT Press, Cambridge, MA."},{"key":"13_CR6","doi-asserted-by":"publisher","first-page":"724","DOI":"10.1109\/72.159061","volume":"3","author":"H Berenji","year":"1992","unstructured":"Berenji, H., & Khedkar, P. (1992). Learning and tuning fuzzy logic controllers through reinforcements. IEEE Transactions on Neural Networks, 3, 724\u2013740.","journal-title":"IEEE Transactions on Neural Networks"},{"key":"13_CR7","volume-title":"Vision, Instruction, and Action","author":"D Chapman","year":"1991","unstructured":"Chapman, D. (1991). Vision, Instruction, and Action. MIT Press, Cambridge, MA."},{"key":"13_CR8","first-page":"92","volume-title":"Proceedings of the Ninth International Conference on Machine Learning","author":"J Clouse","year":"1992","unstructured":"Clouse, J., & Utgoff, P. (1992). A teaching method for reinforcement learning. In Proceedings of the Ninth International Conference on Machine Learning, pp. 92\u2013101 Aberdeen, Scotland."},{"key":"13_CR9","unstructured":"Crangle, C, & Suppes, P. (1994). Language and Learning for Robots. CSLI Publications, Stanford, CA."},{"key":"13_CR10","first-page":"37","volume-title":"Proceedings of the Eleventh International Conference on Machine Learning","author":"M Craven","year":"1994","unstructured":"Craven, M., & Shavlik, J. (1994). Using sampling and queries to extract rules from trained neural networks. In Proceedings of the Eleventh International Conference on Machine Learning, pp. 37\u201345 New Brunswick, NJ."},{"key":"13_CR11","volume-title":"Advances in Neural Information Processing Systems","author":"M Craven","year":"1996","unstructured":"Craven, M., & Shavlik, J. (1996). Extracting tree-structured representations of trained networks. In Advances in Neural Information Processing Systems, Vol. 8 Denver, CO. MIT Press."},{"key":"13_CR12","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/B978-1-55860-036-2.50024-2","volume-title":"Proceedings of the Sixth International Workshop on Machine Learning","author":"J Diederich","year":"1989","unstructured":"Diederich, J. (1989). \u201cLearning by instruction\u201d in connectionist systems. In Proceedings of the Sixth International Workshop on Machine Learning, pp. 66\u201368 Ithaca, NY."},{"key":"13_CR13","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1109\/64.79714","volume":"6","author":"T Dietterich","year":"1991","unstructured":"Dietterich, T. (1991). Knowledge compilation: Bridging the gap between specification and implementation. IEEE Expert, 6, 80\u201382.","journal-title":"IEEE Expert"},{"key":"13_CR14","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1207\/s15516709cog1402_1","volume":"14","author":"J Elman","year":"1990","unstructured":"Elman, J. (1990). Finding structure in time. Cognitive Science, 14, 179\u2013211.","journal-title":"Cognitive Science"},{"key":"13_CR15","doi-asserted-by":"publisher","first-page":"340","DOI":"10.1109\/69.382304","volume":"7","author":"P Frasconi","year":"1995","unstructured":"Frasconi, P., Gori, M., Maggini, M., & Soda, G. (1995). Unified integration of explicit knowledge and learning by example in recurrent networks. IEEE Transactions on Knowledge and Data Engineering, 7, 340\u2013346.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"13_CR16","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1080\/09540098908915644","volume":"1","author":"LM Fu","year":"1989","unstructured":"Fu, L. M. (1989). Integration of neural heuristics into knowledge-based inference. Connection Science, 1, 325\u2013340.","journal-title":"Connection Science"},{"key":"13_CR17","volume-title":"Automatic Refinement of Expert System Knowledge Bases","author":"A Ginsberg","year":"1988","unstructured":"Ginsberg, A. (1988). Automatic Refinement of Expert System Knowledge Bases. Pitman, London."},{"key":"13_CR18","first-page":"331","volume":"17","author":"D Gordon","year":"1994","unstructured":"Gordon, D., & Subramanian, D. (1994). A multistrategy learning scheme for agent knowledge acquisition. Informatica, 17, 331\u2013346.","journal-title":"Informatica"},{"key":"13_CR19","volume-title":"Neural Network Synthesis using Cellular Encoding and the Genetic Algorithm","author":"F Gruau","year":"1994","unstructured":"Gruau, F. (1994). Neural Network Synthesis using Cellular Encoding and the Genetic Algorithm. Ph.D. thesis, Ecole Normale Superieure de Lyon, France."},{"key":"13_CR20","first-page":"231","volume-title":"Cognitive Skills and their Acquisition","author":"F Hayes-Roth","year":"1981","unstructured":"Hayes-Roth, F, Klahr, P., & Mostow, D. J. (1981). Advice-taking and knowledge refinement: An iterative view of skill acquisition. In Anderson, J. (Ed.), Cognitive Skills and their Acquisition, pp. 231\u2013253. Lawrence Erlbaum, Hillsdale, NJ."},{"key":"13_CR21","first-page":"143","volume-title":"Machine Learning: Proceedings on the Tenth International Conference","author":"S Huffman","year":"1993","unstructured":"Huffman, S., & Laird, J. (1993). Learning procedures from interactive natural language instructions. In Machine Learning: Proceedings on the Tenth International Conference, pp. 143\u2013150 Amherst, MA."},{"key":"13_CR22","doi-asserted-by":"crossref","unstructured":"Kaelbling, L. (1987). REX: A symbolic language for the design and parallel implementation of embedded systems. In Proceedings of the AIAA Conference on Computers in Aerospace Wakefield, MA.","DOI":"10.2514\/6.1987-2822"},{"key":"13_CR23","doi-asserted-by":"crossref","unstructured":"Kaelbling, L. (Ed.). (1996). Special Issue on Reinforcement Learning. Kluwer Academic. Machine Learning 22.","DOI":"10.1007\/b102434"},{"key":"13_CR24","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1016\/S0921-8890(05)80027-2","volume":"6","author":"L Kaelbling","year":"1990","unstructured":"Kaelbling, L., & Rosenschein, S. (1990). Action and planning in embedded agents. Robotics and Autonomous Systems, 6, 35\u201348.","journal-title":"Robotics and Autonomous Systems"},{"key":"13_CR25","first-page":"235","volume-title":"Proceedings of the Seventh International Conference on Machine Learning","author":"J Laird","year":"1990","unstructured":"Laird, J., Hucka, M., Yager, E., & Tuck, C. (1990). Correcting and extending domain knowledge using outside guidance. In Proceedings of the Seventh International Conference on Machine Learning, pp. 235\u2013243 Austin, TX."},{"key":"13_CR26","first-page":"598","volume-title":"Advances in Neural Information Processing Systems","author":"Y Le Cun","year":"1990","unstructured":"Le Cun, Y., Denker, J., & Solla, S. (1990). Optimal brain damage. In Touretzky, D. (Ed.), Advances in Neural Information Processing Systems, Vol. 2, pp. 598\u2013605. Morgan Kaufmann, Palo Alto, CA."},{"key":"13_CR27","volume-title":"Lex & yacc","author":"J Levine","year":"1992","unstructured":"Levine, J., Mason, T., & Brown, D. (1992). Lex & yacc. O\u2019Reilly, Sebastopol, CA."},{"key":"13_CR28","first-page":"293","volume":"8","author":"L Lin","year":"1992","unstructured":"Lin, L. (1992). Self-improving reactive agents based on reinforcement learning, planning, and teaching. Machine Learning, 8, 293\u2013321.","journal-title":"Machine Learning"},{"key":"13_CR29","first-page":"182","volume-title":"Proceedings of the Tenth International Conference on Machine Learning","author":"L Lin","year":"1993","unstructured":"Lin, L. (1993). Scaling up reinforcement learning for robot control. In Proceedings of the Tenth International Conference on Machine Learning, pp. 182\u2013189 Amherst, MA."},{"key":"13_CR30","volume-title":"Learning from Instruction and Experience: Methods for Incorporating Procedural Domain Theories into Knowledge-Based Neural Networks","author":"R Maclin","year":"1995","unstructured":"Maclin, R. (1995). Learning from Instruction and Experience: Methods for Incorporating Procedural Domain Theories into Knowledge-Based Neural Networks. Ph.D. thesis, Computer Sciences Department, University of Wisconsin, Madison, WI."},{"key":"13_CR31","first-page":"195","volume":"11","author":"R Maclin","year":"1993","unstructured":"Maclin, R., & Shavlik, J. (1993). Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11, 195\u2013215.","journal-title":"Machine Learning"},{"key":"13_CR32","first-page":"694","volume-title":"Proceedings of the Twelfth National Conference on Artificial Intelligence","author":"R Maclin","year":"1994","unstructured":"Maclin, R., & Shavlik, J. (1994). Incorporating advice into agents that learn from reinforcements. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pp. 694\u2013699 Seattle, WA."},{"key":"13_CR33","doi-asserted-by":"publisher","first-page":"311","DOI":"10.1016\/0004-3702(92)90058-6","volume":"55","author":"S Mahadevan","year":"1992","unstructured":"Mahadevan, S., & Connell, J. (1992). Automatic programming of behavior-based robots using reinforcement learning. Artificial Intelligence, 55, 311\u2013365.","journal-title":"Artificial Intelligence"},{"key":"13_CR34","first-page":"77","volume":"I","author":"J McCarthy","year":"1958","unstructured":"McCarthy, J. (1958). Programs with common sense. In Proceedings of the Symposium on the Mechanization of Thought Processes, Vol. I, pp. 77\u201384. (Reprinted in M. Minsky, editor, 1968, Semantic Information Processing. Cambridge, MA: MIT Press, 403-409.).","journal-title":"Proceedings of the Symposium on the Mechanization of Thought Processes"},{"key":"13_CR35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1287\/mnsc.28.1.1","volume":"28","author":"G Monahan","year":"1982","unstructured":"Monahan, G. (1982). A survey of partially observable Markov decision processes: Theory, models, and algorithms. Management Science, 28, 1\u201316.","journal-title":"Management Science"},{"key":"13_CR36","volume-title":"Machine Learning: An Artificial Intelligence Approach","author":"DJ Mostow","year":"1982","unstructured":"Mostow, D. J. (1982). Transforming declarative advice into effective procedures: A heuristic search example. In Michalski, R., Carbonell, J., & Mitchell, T. (Eds.), Machine Learning: An Artificial Intelligence Approach, Vol. 1. Tioga Press, Palo Alto."},{"key":"13_CR37","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1613\/jair.30","volume":"1","author":"N Nilsson","year":"1994","unstructured":"Nilsson, N. (1994). Teleo-reactive programs for agent control. Journal of Artificial Intelligence Research, 1, 139\u2013158.","journal-title":"Journal of Artificial Intelligence Research"},{"key":"13_CR38","volume-title":"Computational Architectures Integrating Neural and Symbolic Processes","author":"D Noelle","year":"1994","unstructured":"Noelle, D., & Cottrell, G. (1994). Towards instructable connectionist systems. In Sun, R., & Bookman, L. (Eds.), Computational Architectures Integrating Neural and Symbolic Processes. Kluwer Academic, Boston."},{"key":"13_CR39","first-page":"361","volume-title":"Proceedings of the Ninth International Conference on Machine Learning","author":"C Omlin","year":"1992","unstructured":"Omlin, C., & Giles, C. (1992). Training second-order recurrent neural networks using hints. In Proceedings of the Ninth International Conference on Machine Learning, pp. 361\u2013366 Aberdeen, Scotland."},{"key":"13_CR40","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1016\/0004-3702(94)90028-0","volume":"66","author":"D Ourston","year":"1994","unstructured":"Ourston, D., & Mooney, R. (1994). Theory refinement combining analytical and empirical methods. Artificial Intelligence, 66, 273\u2013309.","journal-title":"Artificial Intelligence"},{"key":"13_CR41","first-page":"57","volume":"9","author":"M Pazzani","year":"1992","unstructured":"Pazzani, M., & Kibler, D. (1992). The utility of knowledge in inductive learning. Machine Learning, 9, 57\u201394.","journal-title":"Machine Learning"},{"key":"13_CR42","doi-asserted-by":"crossref","unstructured":"Riecken, D. (1994). Special issue on intelligent agents. Communications of the ACM, 37(1).","DOI":"10.1145\/176789.176801"},{"key":"13_CR43","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1080\/09540098908915640","volume":"1","author":"J Shavlik","year":"1989","unstructured":"Shavlik, J., & Towell, G. (1989). An approach to combining explanation-based and neural learning algorithms. Connection Science, 1, 233\u2013255.","journal-title":"Connection Science"},{"key":"13_CR44","unstructured":"Siegelmann, H. (1994). Neural programming language. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pp. 877\u2013882 Seattle, WA."},{"key":"13_CR45","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1016\/S0020-7373(05)80130-0","volume":"35","author":"S Suddarth","year":"1991","unstructured":"Suddarth, S., & Holden, A. (1991). Symbolic-neural systems and the use of hints for developing complex systems. International Journal of Man-Machine Studies, 35, 291\u2013311.","journal-title":"International Journal of Man-Machine Studies"},{"key":"13_CR46","first-page":"9","volume":"3","author":"R Sutton","year":"1988","unstructured":"Sutton, R. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3, 9\u201344.","journal-title":"Machine Learning"},{"key":"13_CR47","doi-asserted-by":"crossref","first-page":"288","DOI":"10.7551\/mitpress\/3115.003.0040","volume-title":"From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior","author":"R Sutton","year":"1991","unstructured":"Sutton, R. (1991). Reinforcement learning architectures for animats. In Meyer, J., & Wilson, S. (Eds.), From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, pp. 288\u2013296. MIT Press, Cambridge, MA."},{"key":"13_CR48","first-page":"257","volume":"8","author":"G Tesauro","year":"1992","unstructured":"Tesauro, G. (1992). Practical issues in temporal difference learning. Machine Learning, 8, 257\u2013277.","journal-title":"Machine Learning"},{"key":"13_CR49","first-page":"930","volume-title":"Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence","author":"S Thrun","year":"1993","unstructured":"Thrun, S., & Mitchell, T. (1993). Integrating inductive neural network learning and explanation-based learning. In Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pp. 930\u2013936 Chambery, France."},{"key":"13_CR50","first-page":"71","volume":"13","author":"G Towell","year":"1993","unstructured":"Towell, G., & Shavlik, J. (1993). Extracting refined rules from knowledge-based neural networks. Machine Learning, 13, 71\u2013101.","journal-title":"Machine Learning"},{"key":"13_CR51","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/0004-3702(94)90105-8","volume":"70","author":"G Towell","year":"1994","unstructured":"Towell, G., & Shavlik, J. (1994). Knowledge-based artificial neural networks. Artificial Intelligence, 70, 119\u2013165.","journal-title":"Artificial Intelligence"},{"key":"13_CR52","unstructured":"Towell, G., Shavlik, J., & Noordewier, M. (1990). Refinement of approximate domain theories by knowledge-based neural networks. In Proceedings of the Eighth National Conference on Artificial Intelligence, pp. 861\u2013866 Boston, MA."},{"key":"13_CR53","first-page":"596","volume-title":"Proceedings of the Ninth National Conference on Artificial Intelligence","author":"P Utgoff","year":"1991","unstructured":"Utgoff, P., & Clouse, J. (1991). Two kinds of training information for evaluation function learning. In Proceedings of the Ninth National Conference on Artificial Intelligence, pp. 596\u2013600 Anaheim, CA."},{"key":"13_CR54","volume-title":"Learning from Delayed Rewards","author":"C Watkins","year":"1989","unstructured":"Watkins, C. (1989). Learning from Delayed Rewards. Ph.D. thesis, King\u2019s College, Cambridge."},{"key":"13_CR55","first-page":"279","volume":"8","author":"C Watkins","year":"1992","unstructured":"Watkins, C., & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279\u2013292.","journal-title":"Machine Learning"},{"key":"13_CR56","first-page":"335","volume-title":"Proceedings of the 1993 Connectionist Models Summer School","author":"A Weigend","year":"1993","unstructured":"Weigend, A. (1993). On overfitting and the effective number of hidden units. In Proceedings of the 1993 Connectionist Models Summer School, pp. 335\u2013342 San Mateo, CA. Morgan Kaufmann."},{"key":"13_CR57","first-page":"607","volume-title":"Proceedings of the Ninth National Conference on Artificial Intelligence","author":"S Whitehead","year":"1991","unstructured":"Whitehead, S. (1991). A complexity analysis of cooperative mechanisms in reinforcement learning. In Proceedings of the Ninth National Conference on Artificial Intelligence, pp. 607\u2013613 Anaheim, CA."},{"key":"13_CR58","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1016\/S0019-9958(65)90241-X","volume":"8","author":"L Zadeh","year":"1965","unstructured":"Zadeh, L. (1965). Fuzzy sets. Information and Control, 8, 338\u2013353.","journal-title":"Information and Control"}],"container-title":["Learning to Learn"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-1-4615-5529-2_13","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T05:11:31Z","timestamp":1712898691000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-1-4615-5529-2_13"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1998]]},"ISBN":["9781461375272","9781461555292"],"references-count":58,"URL":"https:\/\/doi.org\/10.1007\/978-1-4615-5529-2_13","relation":{},"subject":[],"published":{"date-parts":[[1998]]}}}