Home » MOL2NET » Section 03: Computational Sciences (Applied to Social, Legal, and Life Sciences) » Paper

[] Efficient RL Algorithm by Combing AC with Dual Piecewise Model Learning

Shan Zhong 1 , 2 , 3 * email , Quan Liu 4 , 5 , 6 * email and Qiming Fu 6 , 7 , 8 email
1 School of Computer Science and Technology, Soochow University, Suzhou, Jiangsu, 215006
2 School of Computer Science and Engineering, Changshu Institute of Technology, Changshu, 215500
3 Jiangsu Province Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu, 215006
4 School of Computer Science and Technology, Soochow University, Suzhou, Jiangsu, 215006
5 Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing, 210000
6 Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012
7 Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012
8 College of Electronic & Information Engineering, Suzhou University of Science and Technology, Jiangsu, Suzhou, 215006
* Author to whom correspondence should be addressed.
24 January 2017
221 views
0/5 rated ( 0 ratings )

Abstract

As classic methods for handling continuous action space problem for continuous action space problem in RL, the actor-critic (AC) algorithm and its variants still fail to be sample efficiency. Therefore, we propose a method based on learning two linear models for planning. The two linear models refers to state-based piecewise model and action-based piecewise model, which are determined by the divisions for the state and action space, respectively. Through division, the models are learned more accurately. To accelerate the convergence, the sample near the goal is saved and used to learn the model, the value and the policy to balance the distribution of the samples. On two classic RL benchmarks with continuous MDPs, the proposed method shows the ability of learning an optimal policy by combing both models, and it also outperforms the representative methods in terms of convergence rate and sample efficiency.

Keywords

reinforcement learning; model learning; planning; linear approximation

Cite this article as

Zhong, S.; Liu, Q.; Fu, Q. Efficient RL Algorithm by Combing AC with Dual Piecewise Model Learning. In Proceedings of the MOL2NET, International Conference on Multidisciplinary Sciences, 25 December 2016–25 January 2017; Sciforum Electronic Conference Series, Vol. 2, 2016 ; doi:10.3390/mol2net-02-03895

Presentation

Comments on Efficient RL Algorithm by Combing AC with Dual Piecewise Model Learning