Content area
Full Text
Appl Intell (2015) 43:487498 DOI 10.1007/s10489-015-0665-y
Multi-criteria expertness based cooperative method for SARSA and eligibility trace algorithms
Esmat Pakizeh1 Mir Mohsen Pedram2 Maziar Palhang3
Published online: 19 April 2015 Springer Science+Business Media New York 2015
Abstract Temporal difference and eligibility traces are of the most common approaches to solve reinforcement learning problems. However, except in the case of Q-learning, there are no studies about using these two approaches in a cooperative multi-agent learning setting. This paper addresses this shortcoming by using temporal difference and eligibility traces as the core learning method in multi-criteria expertness based cooperative learning (MCE). The experiments, performed on a sample maze world, show the results of an empirical study on temporal difference and eligibility trace methods in a MCE based cooperative learning setting.
Keywords Cooperative learning Reinforcement
learning Multi-criteria expertness Knowledge transfer
Temporal difference Eligibility traces
[envelopeback] Mir Mohsen Pedram mailto:[email protected]
Web End [email protected]
Esmat Pakizeh mailto:[email protected]
Web End [email protected]
Maziar Palhang mailto:[email protected]
Web End [email protected]
1 Research and Science Branch, Islamic Azad University,Tehran, Iran
2 Data Mining Laboratory, Electrical and Computer Engineering Department, Kharazmi University, Tehran, Iran
3 Artificial Intelligence Laboratory, Department of Computer Engineering, Isfahan University of Technology, Isfahan, Iran
1 Introduction
The capability of cooperation in multi-agent systems is critical in achieving better performance and results in higher efficiency and faster learning compared to individual learning, due to more resources of knowledge and information [1]. Agents benefit from exchanging information during the cooperative learning process, in order to achieve better individual and overall system performances.
One of the best ideas that improves the cooperative learning among agents is expertness based cooperative Q-learning which is named WSS [2]. In WSS, expertness measures are used to extract proper knowledge of agents during the cooperating processes and cooperation can be performed based on each of the defined expertness measures. Multi-Criteria Expertness cooperative Q-learning (MCE) [3, 4] is a new developed cooperative learning method that takes advantages of worthy information about different experiences of agents in a cooperative multi-agent system. Because of using variety of expertness measures, MCE has a rich content that provides better results. In fact, this method has high abilities to use more knowledge and information in contrast to existing methods.
Nowadays, most of researches in multi-agent cooperative learning...