{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T17:54:15Z","timestamp":1754157255952,"version":"3.41.2"},"reference-count":24,"publisher":"Emerald","issue":"1","license":[{"start":{"date-parts":[[2008,3,28]],"date-time":"2008-03-28T00:00:00Z","timestamp":1206662400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2008,3,28]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-heading\">Purpose<\/jats:title><jats:p>The purpose of this paper is to report upon research into developing a biologically inspired target\u2010tracking system (TTS) capable of acquiring quality images of a known target type for a robotic inspection application.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Design\/methodology\/approach<\/jats:title><jats:p>The approach used in the design of the TTS hearkens back to the work on adaptive learning by Oliver Selfridge and Chris J.C.H. Watkins and the work on the classification of objects by Zdzislaw Pawlak during the 1980s in an approximation space\u2010based form of feedback during learning. Also, during the 1980s, it was Ewa Orlowska who called attention to the importance of approximation spaces as a formal counterpart of perception. This insight by Orlowska has been important in working toward a new form of adaptive learning useful in controlling the behaviour of machines to accomplish system goals. The adaptive learning algorithms presented in this paper are strictly temporal difference methods, including Q\u2010learning, sarsa, and the actor\u2010critic method. Learning itself is considered episodic. During each episode, the equivalent of a Tinbergen\u2010like ethogram is constructed. Such an ethogram provides a basis for the construction of an approximation space at the end of each episode. The combination of episodic ethograms and approximation spaces provides an extremely effective means of feedback useful in guiding learning during the lifetime of a robotic system such as the TTS reported in this paper.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Findings<\/jats:title><jats:p>It was discovered that even though the adaptive learning methods were computationally more expensive than the classical algorithm implementations, they proved to be more effective in a number of cases, especially in noisy environments.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Originality\/value<\/jats:title><jats:p>The novelty associated with this work is the introduction of an approach to adaptive adaptive learning carried out within the framework of ethology\u2010based approximation spaces to provide performance feedback during the learning process.<\/jats:p><\/jats:sec>","DOI":"10.1108\/17563780810857121","type":"journal-article","created":{"date-parts":[[2008,4,12]],"date-time":"2008-04-12T07:13:46Z","timestamp":1207984426000},"page":"46-68","source":"Crossref","is-referenced-by-count":3,"title":["Adaptive learning by a target\u2010tracking system"],"prefix":"10.1108","volume":"1","author":[{"given":"Daniel","family":"Lockery","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James F.","family":"Peters","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","reference":[{"key":"key2022030720243707000_b1","doi-asserted-by":"crossref","unstructured":"Jaakkola, T., Jordan, M.I. and Singh, S.P. (1994), \u201cConvergence of stochastic iterative dynamic programming algorithms\u201d, Neural Computation, Vol. 6, pp. 1185\u2010201.","DOI":"10.1162\/neco.1994.6.6.1185"},{"key":"key2022030720243707000_b2","doi-asserted-by":"crossref","unstructured":"Kaelbling, L.P., Littman, M.L. and Moore, A.W. (1996), \u201cReinforcement learning: a survey\u201d, Journal of Artificial Intelligence Research, Vol. 4, pp. 237\u201048.","DOI":"10.1613\/jair.301"},{"key":"key2022030720243707000_b3","unstructured":"Knudsen, B. (2006), CC5X C Compiler, B. Knudsen Data, available at: www.bknd.com\/cc5x\/index.shtml."},{"key":"key2022030720243707000_b4","unstructured":"Linux (2003), Linux RedHat 9.0 Manuals, available at: www.redhat.com\/docs\/manuals\/linux\/RHL\u20109\u2010Manual\/ (accessed 1 February 2007)."},{"key":"key2022030720243707000_b5","unstructured":"Lockery, D. (2007), \u201cLearning with ALiCE II\u201d, MSc thesis, Electrical and Computer Engineering, University of Manitoba, Manitoba."},{"key":"key2022030720243707000_b6","unstructured":"Microchip Technology Inc. (2006), MPLAB Integrated Development Environment, Microchip Technology Inc., Chandler, AZ, available at: www.microchip.com."},{"key":"key2022030720243707000_b7","unstructured":"Orlowska, E. (1982), Semantics of Vague Concepts. Applications of Rough Sets, Report 469, Institute for Computer Science, Polish Academy of Sciences, Warszawa."},{"key":"key2022030720243707000_b24","doi-asserted-by":"crossref","unstructured":"Orlowska, E. (1985), Semantics of Vague Concepts, In: Dorn, G. and Weingartner, P. (Eds) Foundations of Logic and Linguistics. Problems and Solutions, Plenum Press, London, pp. 465\u201082.","DOI":"10.1007\/978-1-4899-0548-2_20"},{"key":"key2022030720243707000_b8","unstructured":"Pawlak, Z. (1981), Classification of Objects by Means of Attributes, Report 429, Institute for Computer Science, Polish Academy of Sciences, Warszawa, March."},{"key":"key2022030720243707000_b12","doi-asserted-by":"crossref","unstructured":"Peters, J.F. (2005), \u201cRough ethology: toward a biologically\u2010inspired study of collective behaviour in intelligent systems with approximation spaces\u201d, Transactions on Rough Sets III, Springer LNCS 3400, Berlin, pp. 153\u201074.","DOI":"10.1007\/11427834_7"},{"key":"key2022030720243707000_b10","unstructured":"Peters, J.F., Henry, C. and Gunderson, D.S. (2006a), \u201cBiologically\u2010inspired approximate adaptive learning control strategies: a rough set approach\u201d, International Journal of Hybrid Intelligent Systems, Vol. 3, pp. 1\u201014."},{"key":"key2022030720243707000_b9","unstructured":"Peters, J.F., Henry, C. and Ramanna, S. (2005), \u201cRough Ethograms: study of intelligent system behaviour\u201d, in K_lopotek, M.A., Wierzcho'n, S. and Trojanowski, K. (Eds), New Trends in Intelligent Information Processing and Web Mining (IIS05), pp. 17\u2010126."},{"key":"key2022030720243707000_b11","unstructured":"Peters, J.F., Borkowski, M., Henry, C., Lockery, D., Gunderson, D. and Ramanna, S. (2006b), \u201cLine\u2010crawling bots that inspect electric power transmission line equipment\u201d, Proc. 3rd Int. Conf. on Autonomous Robots and Agents (ICARA 2006), pp. 39\u201044."},{"key":"key2022030720243707000_b13","unstructured":"Press, W.H., Teukolsky, S.A., Vetterling, W.T. and Flannery, B.P. (2002), Numerical Recipes in C++: The Art of Scientific Computing, 2nd ed., Cambridge University Press, Cambridge, pp. 278\u201090."},{"key":"key2022030720243707000_b14","unstructured":"Rummery, G.A. and Niranjan, M. (1994), \u201cOn\u2010line Q\u2010learning using connectionist systems\u201d, Technical Report CUED\/F\u2010INFENG\/TR 166, Engineering Department, Cambridge University, Cambridge, September."},{"key":"key2022030720243707000_b15","doi-asserted-by":"crossref","unstructured":"Selfridge, O.G. (1984), \u201cSome themes and primitives in ill\u2010defined systems\u201d, in Selfridge, O.G., Rissland, E.L. and Arbib, M.A. (Eds), Adaptive Control of Ill\u2010defined Systems, Plenum Press, London.","DOI":"10.1007\/978-1-4684-8941-5"},{"key":"key2022030720243707000_b16","doi-asserted-by":"crossref","unstructured":"Singh, S.P. and Sutton, R.S. (1996), \u201cReinforcement learning with replacing eligibility traces\u201d, Machine Learning, Vol. 22, pp. 123\u201058.","DOI":"10.1007\/BF00114726"},{"key":"key2022030720243707000_b17","doi-asserted-by":"crossref","unstructured":"Singh, S.P., Jaakkola, T., Littman, M.L. and Szepesvari, C. (2000), \u201cConvergence results for single\u2010step on\u2010policy reinforcement\u2010learning algorithms\u201d, Machine Learning, Vol. 38 No. 3, pp. 287\u2010308.","DOI":"10.1023\/A:1007678930559"},{"key":"key2022030720243707000_b19","unstructured":"Sutton, R.S. (1996), \u201cGeneralization in reinforcement learning: successful examples using sparse coarse coding\u201d, Advances in Neural Information Processing Systems, Vol. 8, The MIT Press, Cambridge, MA, pp. 1038\u201044."},{"key":"key2022030720243707000_b20","doi-asserted-by":"crossref","unstructured":"Sutton, R.S. and Barto, A.G. (1998), Reinforcement Learning: An Introduction, The MIT Press, Cambridge, MA.","DOI":"10.1109\/TNN.1998.712192"},{"key":"key2022030720243707000_b18","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., Barto, A.G. and Anderson, C.W. (1983), \u201cNeuronlike adaptive elements that can solve difficult learning problems\u201d, IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC\u201013 No. 5, pp. 834\u201047.","DOI":"10.1109\/TSMC.1983.6313077"},{"key":"key2022030720243707000_b21","doi-asserted-by":"crossref","unstructured":"Tinbergen, N. (1963), \u201cOn aims and methods of ethology\u201d, Zeitschrift f'ur Tierpsychologie, Vol. 20, pp. 410\u201033.","DOI":"10.1111\/j.1439-0310.1963.tb01161.x"},{"key":"key2022030720243707000_b22","unstructured":"Watkins, C.J.C.H. (1989), \u201cLearning from delayed rewards\u201d, PhD thesis, King's College, Cambridge University, Cambridge, supervisor: Richard Young."},{"key":"key2022030720243707000_b23","doi-asserted-by":"crossref","unstructured":"Watkins, C.J.C.H. and Dayan, P. (1992), \u201cTechnical note: Q\u2010learning\u201d, Machine Learning, Vol. 8, pp. 279\u201092.","DOI":"10.1007\/BF00992698"}],"container-title":["International Journal of Intelligent Computing and Cybernetics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/17563780810857121","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/17563780810857121\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/17563780810857121\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T23:44:09Z","timestamp":1753400649000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/ijicc\/article\/1\/1\/46-68\/130308"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,3,28]]},"references-count":24,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2008,3,28]]}},"alternative-id":["10.1108\/17563780810857121"],"URL":"https:\/\/doi.org\/10.1108\/17563780810857121","relation":{},"ISSN":["1756-378X"],"issn-type":[{"type":"print","value":"1756-378X"}],"subject":[],"published":{"date-parts":[[2008,3,28]]}}}